Resource mentioned in this lesson: The Walrus Operator: Python’s Assignment Expressions
Join us and get access to thousands of tutorials and a community of expert Pythonistas.
This lesson is for members only. Join us and get access to thousands of tutorials and a community of expert Pythonistas.
Modifying Routing Behavior
00:00
You can tell OpenRouter you want to use a specific model by defining it in the request payload. For example, setting model to anthropic/claude-sonnet-4.5 means OpenRouter will only route your request to a provider hosting that model. This comes with some immediate benefits: predictable behavior, easier to manage costs, and consistent results.
00:22
Let’s take a look. The ask_specific_model.py script is based on the script from the previous lesson, again with a few tweaks. Line 13 is updated to pick a specific model, openai/gpt-3.5-turbo, and you also have some error handling now, in lines 19-24.
00:41
if model := data.get('model'):. This use of the assignment expression does two things. It evaluates the right-hand side, data.get('model'), providing its value as the condition to if.
00:53
And then it creates a variable, model, that you can use inside of the if block. Then it prints an f-string reporting the model and its provider, followed by another f-string with the response contents.
01:05
If data.get('model') evaluates to something falsy, it reports that no model was found and prints the response. This is some basic error handling to account for possible outages or rate limits.
01:16
:= is also called the walrus operator, which is hilarious, and if you want to read more about it, look at this tutorial. The Walrus Operator: Python’s Assignment Expressions.
01:26 Okay, back to your regularly scheduled programming. Open up the shell and run the script.
01:33
python ask_specific_model.py. Scroll up a bit and you’ll see the model used was openai/gpt-3.5-turbo, naturally served by OpenAI.
01:46 Just as expected. But there is something important to note here.
01:50 Models may be served by multiple providers. Even serving the same model, they may have significant differences in cost, speed, or capacity. So instead of selecting providers manually, OpenRouter can optimize provider selection using a strategy of your choice.
02:05 Price, optimizing for the lowest cost provider. Throughput, which optimizes for the highest capacity provider. And latency, optimizing for the fastest possible response times.
02:15
And how can you set this preference? Let’s go back to the IDE. Now open the route_requests.py script.
02:23
The first thing you’ll notice is that the request logic has been extracted into a function, make_request(). First you define headers the same as before, but now the payload dictionary receives its values for model and messages from the same named function parameters.
02:37
In addition, the optional provider_config argument is where you’ll store your sorting preferences.
02:43
response.raise_for_status() will raise an error in case the HTTP request was not successful. If no error is raised, the JSON is returned.
02:52
Then you call make_request(), setting model to meta-llama/llama-3.1-70b-instruct. And for fun, ask AI to explain Star Wars this time. And then you pass the dict, {"sort": "price"} as the provider_config.
03:05 The rest of the script remains the same. Run the script and…
03:19 Once again, not so brief.
03:22
Scroll up and see… The model requested was served, meta-llama/llama-3.1, and it was served by DeepInfra. Evidently, the cheapest possible provider of this model.
03:34 And that’s how you can provide preferences to OpenRouter. Which is great, but it leaves you with one question. What if the model you’re looking for is unavailable on any provider?
Become a Member to join the conversation.
