Using Intelligent Routing

00:00 OpenRouter uses intelligent routing to automatically select the model for your prompt. Enable intelligent routing with the following. To the payload dictionary that you’ll use for your request, set the model key to the string openrouter/auto.

00:13 Then, OpenRouter will analyze your prompt, select an appropriate model, and route the request to the best available provider. If you don’t have strong preferences for models or providers, this is a great approach.

00:25 To see it in action, head back to your IDE.

00:30 Grab the ask_auto_model.py file from the course resources. Here you have a similar script to the one you used before, with some important changes.

00:38 Line 4 has changed. Now you’re targeting the chat completions endpoint, where you’ll send a prompt and get a model response. Because you’re sending data, you also add Content-Type application/json to the headers dictionary.

00:52 And the data you’re sending is in the payload dictionary. model is defined as openrouter/auto, meaning OpenRouter will choose the model for your request.

01:00 And the messages key holds a list of dictionaries. This is fairly standard for LLM chats. In this one, the role is user and the content is "Briefly explain Star Trek." And instead of calling requests.get(), you call requests.post(), targeting the OpenRouter API URL, passing in headers as headers, and the payload dict as the json argument.

01:24 You store the JSON response as a dict in data, and report the model that was used, as well as the response content, as nicely formatted f-strings.

01:34 Open up the shell, and before you run the script, unless you’re using the same shell as before, make sure to export your API key again.

01:42 And give it a go. python ask_auto_model.py.

01:52 Not exactly what I’d call brief.

01:56 Scroll up. And the model used is mistralai/mistral-7b-instruct-v0.1. Not necessarily the same model you’ll see if you’re following along at home.

02:08 Remember, because we set model to openrouter/auto, OpenRouter automatically selects one of the available models based on the prompt it received.

02:17 And the response is a very lengthy and thorough explanation of what Star Trek is.

02:24 Very cool. Of course, sometimes you do have preferences for which models to use. Learn how to apply those preferences in the next lesson.