Looking at Next Steps

00:00 This would be a good time for me to refer you to the companion tutorial of this video course, which covers most of your next steps here in a bit more detail.

00:10 So please check out the supporting materials drop-down below this lesson. So now that you have a fully functioning bot that persists the index, what else can you do?

00:21 So firstly, you could improve functionality. For example, you can ask to use a different LLM. So the default LLM has a tendency to have hallucinations. So if you use gpt-4o-mini, then you can reduce those hallucinations, and you need to add three lines of code.

00:41 So firstly, to the from llama_index.core import section, at the very end, you can add Settings. You then also need to import OpenAI from llama_index.llms.openai.

00:57 And then the third line of code to add is the last one here on screen, which is Settings.llm = OpenAI. And then you set the model to gpt-4o-mini as a string, and then pick the temperature as something low, because the lower that number, the lower the chances of hallucinations.

01:16 Now, this isn’t the only model that you can move to. You can also use Google’s LLM, which at the time of recording was free to use, as opposed to the OpenAI model, which is not.

01:28 Or you could use local LLMs. The other thing you can do is you can use async queries. So if you are building a real world example like web traffic, anything like that, then you are very likely to need async queries. Now, all this is also explained in the written tutorial on which this video is based. So I would highly recommend you have a read through that tutorial.

01:55 The other thing you could do is change the reader. So far, you have been using the SimpleDirectoryReader, which is great. It’s a workhorse that does a lot of things well, but there are specific readers for specific types of documents, and I’ve listed them here for you.

02:10 And it goes quite far. You have the web there as well, and YouTube transcripts, and all sorts of funky stuff. So LlamaIndex is a very powerful tool.

02:20 You could also use additional index types. So, so far, you have used the VectorStoreIndex, but there are different index types. And again, this is touched on in the written tutorial. Of course, security: always keep your API keys secure, and always consider data protection, because remember that the context data from your secret document is passed to the LLM.

02:44 So if this LLM goes on the internet, it might be that your data isn’t as secret as you had hoped. As a real life example, in the company I work for, that was a problem, and therefore a particular application was shut down.

02:59 So do check the LLM’s privacy policy. Of course, cost management: monitor your usage of costs closely, unless, of course, you have a free LLM, and then evaluation. That is one of the RAG steps that we haven’t performed in this course.

03:17 So do check the system responses for accuracy and correctness, of course, and for the speed of response.

03:25 You are almost done. I’d like to show you just one more thing, though, and that’s how to investigate document objects and the index in a bit more detail in the next lesson.