Implementing Model Fallbacks

00:00 In life, when you can’t get your first choice, you should always have a fallback. This applies to model selection as well. Fallbacks improve reliability by trying multiple models in order.

00:10 OpenRouter accepts a list of model options, tries each model sequentially, and handles retry logic automatically. This will make your application resilient to outages, rate limits, and model failures.

00:23 Let’s take a look.

00:25 Back in VS Code, the last script you’ll run is fallback_models.py. It has the request logic moved to a function called make_request_with_fallback(), which now has a models_list parameter, along with messages as usual.

00:37 The headers have not changed, but now it builds a payload with a models, plural, key, with models_list as its value.

00:45 messages remains the same. It returns the result of sending the POST request to the chat completion endpoint, with headers as headers, and the json argument as payload. Get your response by calling make_request_with_fallback(), passing in a list of models to models_list.

01:00 openai/gpt-5, openai/gpt-3.5-turbo, and openai/gpt-3.5-turbo-16k. And format messages just like before, this time asking what is perhaps the most contentious question of all time. Is Star Trek better than Star Wars? You might be able to figure out what I think, but let’s leave it to the AI.

01:21 The rest of the code is unchanged from the previous lesson.

01:26 The model provider and response will be printed unless no model is found.

01:30 Open up the terminal and run the script.

01:33 python fallback_models.py. And the result is gpt-3.5-turbo served by OpenAI, along with a very wishy-washy answer, as expected. And that’s how you can implement fallbacks.

01:48 You can see that GPT-5 wasn’t available, most likely it’s not on the free plan, so GPT-3.5 Turbo was used. This is a great way to ensure that even if your favorite model can’t be served, your production application doesn’t have to experience downtime.

02:01 So at this point, you’ve seen a few of the options and techniques for selecting models with OpenRouter. In the next lesson, you’ll look at some considerations to help guide your selections.