Connecting to an LLM

00:00 Starting the REPL one last time. To set a baseline, you’ll first want to see a naive response from Gemini without reviews added to the context. Starting with imports, import json, from google import genai, from google.genai import types.

00:19 And you’re going to need to load your API key, like so. with open("config.json", mode="r") as json_file:, as json_file, config_data = json.load(json_file).

00:35 Now config_data is a dictionary with your API key in it. With that, you can instantiate a client for making calls to the LLM. gemini = genai.Client() with api_key set to config_data .get("gemini-secret-key"), matching the name in your config.json.

00:54 Create a variable context, the string, "You are a customer success employee at a large car dealership." And a variable question, also a string, "What's the key to great customer satisfaction? Under 200 words." LLMs tend to be a little verbose, so "Under 200 words" keeps things snappy.

01:17 And let’s see what Gemini says. response = gemini.models.generate_content(), model="gemini-2.5-flash". As of March 2026, this is one of the available free LLMs offered by Google.

01:36 config=types.GenerateContentConfig(), system_instruction=context.

01:45 contents=question. The config argument is being used to supply context. This isn’t part of the conversation, but informs how the LLM should handle questions.

01:56 And contents is the question you defined above. And no error, so it looks like it worked. print(response.text) to see what the LLM said.

02:05 "The key to great customer satisfaction is making every interaction personal and truly understanding the individual behind the transaction." Okay, this isn’t bad advice, but it’s a little generic and it doesn’t really have anything to do with cars either.

02:21 So let’s see how the LLM does with some relevant context. Starting in a fresh REPL session again, imports, import os, import json, import chromadb.

02:34 from chromadb.utils import embedding_functions. from google import genai. from google.genai import types.

02:49 Set os.environ["TOKENIZERS_PARALLELISM"] to the string "false". This will silence a specific warning from working with embeddings. Define some constants.

02:59 CHROMA_PATH equals "car_review_embeddings".

03:04 EMBEDDING_FUNC_NAME equals "multi-qa-MiniLM-L6 -cos-v1". COLLECTION_NAME equals "car_reviews".

03:16 Grab your config.json again. with open("config.json", mode="r") as json_file:. config_data = json.load(json_file).

03:30 Set up Gemini and ChromaDB.

03:34 gemini = genai.Client(). api_key=config_data.get("gemini -secret-key"). client = chromadb.PersistentClient(), passing in CHROMA_PATH.

03:48 And embedding_func = embedding_functions.SentenceTransformerEmbeddingFunction() with model_name=EMBEDDING_FUNC_NAME.

04:02 CHROMA_PATH here matches the database you set up in the previous lesson. As does the EMBEDDING_FUNC_NAME.

04:10 Next, get the collection. collection = client.get_collection(), name=COLLECTION_NAME, embedding_function=embedding_func.

04:25 And this time you’ll define context as a multi-line string. context equals "You are a customer success employee at a large car dealership.

04:36 Use the following car reviews to answer questions." Pair of curly brackets. You’ll be adding those reviews in a minute. Define question as a multi-line string.

04:49 "What's the key to great customer satisfaction based on detailed positive reviews? In under 200 words." And get some reviews for the query.

05:03 good_reviews = collection.query(). query_texts is a list containing question. n_results=10. Just include documents.

05:16 And where rating is greater than or equal to 3. This ensures that the 10 relevant reviews are positive. Combine these reviews into one single string to interpolate them later.

05:29 reviews_str = ",".join(). good_reviews["documents"] at index 0. And take a quick look at the start of the reviews_str.

05:38 reviews_str first 160 characters. "Great value, awesome reliability. I normally don't do reviews, but this was worth writing. I own three pickups and I'm not brand dedicated because there are..." Because there’s what?

05:51 Oh well. Okay, moment of truth. Ask the LLM. response = gemini.models.generate_content().

06:02 Model again, "gemini-2.5-flash". config=types.GenerateContentConfig().

06:13 system_instruction=context.format(reviews_str). And contents=question. In this line here is where RAG happens. You format the context string, inserting the string of good reviews.

06:30 Sending it all off to the LLM. print(response.text). And hey, these results seem way more relevant. The LLM is able to use the reviews supplied to highlight value, quality, driving experience, features, and support.

06:49 That’s really cool. Since we’re already here, let’s try one more query. Redefine context. "You are a customer success employee at a large car dealership. Use the following car reviews to answer questions." Curly brackets.

07:10 Redefine question. "Which of these poor reviews has the worst implications about our dealership?

07:20 Explain why in under 200 words." This time get poor_reviews from collection.query(). query_texts is a list containing question.

07:33 n_results is 5. Just include documents. And filter to where rating is less than or equal to 3.

07:43 reviews_str = ",".join(). poor_reviews["documents"] at index 0.

07:51 And response = gemini.models.generate_content(). model="gemini-2.5 -flash". config=types.GenerateContentConfig().

08:09 system_instruction=context.format(reviews_str). Passing in those negative reviews for context. contents=question. Send that off to Google.

08:22 And print(response.text). "The review with the worst implications for our dealership is the Nissan and Infiniti review." Alright, this looks pretty relevant and accurate.

08:34 Just showcasing another great use case. Using ChromaDB with an LLM, you can query a large dataset and get actionable results from the AI. Very impressive indeed.

08:45 I could play around with this all day, and I hope you could too. But unfortunately, it’s almost time to say goodbye. First, a tour down memory lane. Let’s review and summarize what you’ve learned.