Adding Context for a Large Language Model

00:00 And now you can build the project you’ve been thinking about all this time, adding context to an LLM. Here are the project’s goals: use an LLM to simulate a customer success agent at a car dealership, store and search car reviews using ChromaDB, use relevant reviews to provide context to the LLM, and generate insights based on real review data using RAG. What’s RAG?

00:25 Glad you asked. Retrieval-augmented generation, or RAG, is a system to improve LLM responses. A RAG system retrieves documents that are most relevant to a query.

00:36 This provides the LLM with information it never saw while it was being trained, the purpose being to reduce hallucinations and improve response accuracy, which helps generate context-informed answers for better decision-making.

00:52 Your RAG system will look like this. You start with a query. The query is passed into an embedding function and used to query ChromaDB.

01:01 ChromaDB has been loaded with documents that have been processed with the same embedding function you used for your query, and it’s crucial that the same embedding function is used throughout.

01:11 This returns relevant documents like you saw in the previous lesson. Then, the documents and the query are both fed into the LLM, augmenting the LLM’s existing knowledge base with new information related to the query, hopefully leading to better responses.

01:27 Here are the steps you’ll take to build your RAG system. First, you’ll set up the project. Then, explore the data and load it into ChromaDB, finally bringing everything together by connecting to an LLM.

01:40 Excited? Me too. Let’s go!