Let's go ahead and talk about PyCon indexes.
An index is the highest level
organizational unit of vector data in PyCon.
It accepts and stores vectors, serves
queries over the vectors it contains and does other vector operations over its contents.
There are currently serverless indexes and pod -based indexes.
Pods are pre -configured units of hardware for running a PyCon service.
With serverless indexes, you don't configure or manage any compute or
storage resources.
Instead, serverless indexes scale
automatically based on usage and you pay only for the amount of data stored and
operations performed with no minimums.
With pod -based indexes, you choose one
or more pre -configured units of hardware called pods.
Depending on the pod type, pod size and the number of pods used, you get a
different amount of storage and higher or lower latency and throughput.
In this section, we'll use the free starter plan which is suitable for trying
out and for small applications.
Currently, on the free starter plan, you
get one project with up to 5 serverless indexes and up to 100 namespaces per index.
This allows you to store nearly 300 ,000 vectors which is enough for tasks and
small projects.
If you need more, you'll have to upgrade
to a paid plan.
We are authenticated to Pinecon, so let's
see some information about all indexes.
Call PC .ListIndexes to get a complete
description of all indexes in a project.
Note that this is a list of dictionaries
and you can access the name of the first index using PC .ListIndexes of 0.
And you can access its name by accessing the value of the name key.
This is the name of my index.
This is equivalent to PC .DescribeIndex
and the name of the index, Langchain.
To display all index names, you can also
call the names method.
PC .ListIndexes .Names.
Let's go ahead and create a Pinecon index.
I'll import the serverlessSpec class from
the Pinecon library.
It is necessary to deploy a serverless index.
From Pinecon, import serverlessSpec.
Since I am using the free plan, I am
allowed to have up to 5 indexes.
In the Pinecon web console, check that
you don't have more.
In this example, I will remove the
existing index called Langchain and recreate it.
I will create a new index called Langchain.
indexName equals Langchain.
I will create it only if it doesn't already exist.
So if indexName not in PC .ListIndexes .Names, I am printing a message for the user.
It will be an F string.
Creating index and indexName in curly braces.
Now, I am calling PC .CreateIndex with the following arguments.
Name equals indexName.
Dimension equals 1536.
This is the default dimension for text embedding 3small, one of the recommended
OpenACE embedding models.
Metric equals cosine.
This is the algorithm used to calculate the distance between vectors.
And the spec equals serverlessSpec of cloud equals AWS and the region equals US
East 1.
Check for other available regions in the
Pinecon documentation.
And I am printing the message index created.
Now, the else branch.
If the index already exists, print
indexName in curly braces already exists.
Very well.
I am running the code.
It is creating the index.
The index was created and I can check it on the Pinecon web console.
I am refreshing the page.
Take a look, the index is here.
Let's go ahead and take a look at how to delete an index.
So, indexName equals lengthchain.
First, I will check that the index
already exists.
If indexName in PC .list .indexes .names,
I am printing a message and deleting the index.
Deleting index and the indexName in curly braces.
And I am calling the deleteIndex method with the indexName as an argument.
And done.
Else, if the index doesn't exist, I am
just printing a message.
index, indexName does not exist.
The name of the variable is index underlined name.
I am running the code.
It is deleting the index.
And I am refreshing the web page.
There are no indexes available.
I am creating the index again.
It is creating it.
To perform any operation with an index, you must first select it.
To select an index, use PC .index and provide the indexName as an argument.
This method will return an object of type index.
Here, I am selecting the index and
displaying some statistics about it, including the dimension and total vector count.
index .describeIndexStacks.
This operation provides statistics about
the contents of an index, such as the vector count per namespace, the number of
dimensions and the index fullness.
Keep in mind that serverless indexes
automatically scale as needed, so index fullness mainly applies to pod -based indexes.
That is all for now.
In this video, you have learned how to
work with PyCon indexes, creating, deleting and retrieving information and stacks.
Next up, we will dive into working with vectors.