Embeddings are the core of building LLMs applications.
Let's do a short recap of Embeddings.
Text Embeddings are numeric representations of text and are used in
actual language processing and machine learning tasks.
Text Embeddings can be used to measure the relatedness and similarity between
two pieces of text.
Relatedness is a measure of how closely
two pieces of text are related in meaning.
In this context, the distance between two
Embeddings or two vectors measures their relatedness which translates to the
relatedness between the text concepts they represent.
Text concepts are words and phrases.
Similar Embeddings or vectors represent
similar concepts.
There are two common approaches to
measuring relatedness and similarity between Text Embeddings.
Cosine similarity and Euclidean distance.
We'll find these often used when working
with Embeddings.
Some examples of how Text Embeddings can
be used to measure relatedness and similarity are text classification which
is the task of assigning a label to a piece of text, text clustering which is
the task of grouping together pieces of text that are similar in meaning, and
question answering which is the task of answering a question posed in natural language.
We'll now take a quick break.
In the next video we'll discuss vector databases.
These databases are designed for storing and querying high dimensional vectors efficiently.