In this project, we will build a question -answering system using the OPL stack.
OPL stands for OpenAI, Pinecon, and LangChain.
GPT models are great at answering
questions, but only on topics they have been trained on.
What if you want GPT to answer questions about topics it hasn't been trained on?
For example, about recent events after September 2021 for GPT 3 .5 or GPT 4, or
about your non -public documents.
GPT models can learn new knowledge in two
ways, fine -tuning on a training set or model inputs.
Fine -tuning on a training set is the most natural way to teach the model
knowledge, but it can be time -consuming and expensive.
It also builds long -term memory, which is not always necessary.
Model inputs means inserting the knowledge into an input message.
For example, we can send an entire book or PDF document to the model as an input
message, and then we can start asking questions on topics found in the input message.
This is a good way to build short -term memory for the model.
When we have a large corpus of text, it can be difficult to use model inputs
because each model is limited to a maximum number of tokens, which in most
cases is around 4000.
We cannot simply send the text from a 500
-page document to the model because this will exceed the maximum number of tokens
that the model supports.
The recommended approach is to use model
inputs with embedded -based search.
Embeddings are simple to implement and
work especially well with questions.
The pipeline is as follows.
First, we prepare the search data, and we'll do that once per document.
This means, load the data into Lanchain documents, split the documents into short
and mostly self -contained sections called chunks, embed the chunks into
numeric vectors using an embedding model such as OpenAI's text embedding ADA002,
and save the chunks and the embeddings to a vector database such as Pinecon,
Chroma, Milvus, or Quadrant.
After we've prepared the document, we can
do the search.
Given a user query, generate an embedding
for the question using the same embedding model.
Using the questions embedding and the
chunk embeddings from the first step, rank the vectors by similarity to the
questions embedding using cosine similarity or Euclidean distance.
The nearest vectors represent chunks similar to the question.
The third step is to ask.
You insert the question and the most
relevant chunks obtained in step 2b, here, into a message to XGPD model.
And the XGPD model will return an answer.
In this project, we'll build a complete
question -answering application on custom data that follows the pipeline I've just described.
This technique is also called retrieval augmentation because we retrieve relevant
information from an external knowledge base and give that information to our LLM.
The external knowledge base is our window into the world beyond the LLM's training data.
Because practice is more valuable than 1000 words, let me show you the final
version of the application.
I have loaded, chunked, and embedded this
PDF file on React.
This document was published in March 2023
and was not included in the XGPD 4 training data.
By the way, in this context, React has nothing to do with the JavaScript
framework used to build frontends.
React is a general paradigm that combines
reasoning and acting advices to enable language models to solve various language
reasoning and decision -making tasks.
Now, I want to ask questions about this document.
It's like having someone in front of me who knows everything about React and can
respond to any questions.
The first question is, what is reasoning
and acting in LLMs?
And I've got the answer.
I'm asking the second question.
What is chain of thought?
All these questions are from this document.
And I've got a pretty good answer.
And the last question.
What is React prompting?
Very well.
I've got good answers to all my questions.
Of course, instead of React, we can load a document about anything.
A medical pathology, a treatment plan, legal documents, accounting documents,
and more.
Ok.
We have a lot of work ahead, so let's get started building this application step by step.