Considering Limitations and Gotchas

00:00 In this lesson, you’re going to be given some final considerations to ponder on, specifically about costs. Remember that the token costs you’re going to incur depend on a couple of different factors, namely the system prompt that you define, the number of tools that you define, and all of those accessory things.

00:19 It’s also going to increase with the roundtrips from tool calls, because each tool call represents an LLM saying that it wants the result from a given tool, Pydantic AI calling that function, then taking the response from the function and passing it to the LLM again.

00:38 So all of those tool calls will increase your costs, and then also the tokens you’re going to spend in building the structured output, because the LLM needs to produce the output in a specified structure.

00:51 Now, the validation that Pydantic AI is doing for you is very useful, it’s very convenient, but that also might mean you’re going to have more latency, because after the LLM gives you the response, you still have to parse it and validate it, and it also means you’re going to spend more tokens, especially if you set the output_retries to be a number larger than one.

01:15 And then as a final note, with time these things get better, and so hopefully with time you will feel this lesson less, but the ability of an LLM to build structured outputs varies widely with provider, the complexity of your prompts, and the complexity of the output model that you specify. That’s it for this lesson.

01:39 In the next lesson, you’re going to wrap up the whole course and take a look at possible next steps.