Implementing Model Fallbacks

00:00 In life, when you can’t get your first choice, you should always have a fallback. This applies to model selection as well. Fallbacks improve reliability by trying multiple models in order.

00:10 OpenRouter accepts a list of model options, tries each model sequentially, and handles retry logic automatically. This will make your application resilient to outages, rate limits, and model failures.

00:23 Let’s take a look.

00:25 Back in VS Code, the last script you’ll run is fallback_models.py. It has the request logic moved to a function called make_request_with_fallback(), which now has a models_list parameter, along with messages as usual.

00:37 The headers have not changed, but now it builds a payload with a models, plural, key, with models_list as its value.

00:45 messages remains the same. It returns the result of sending the POST request to the chat completion endpoint, with headers as headers, and the json argument as payload.

00:54 Get your response by calling make_request_with_fallback(), passing in a list of models to models_list. openai/gpt-5, openai/gpt-3.5-turbo, and openai/gpt-3.5-turbo-16k.

01:09 And format messages just like before, this time asking what is perhaps the most contentious question of all time. Is Star Trek better than Star Wars?

01:17 You might be able to figure out what I think, but let’s leave it to the AI. The rest of the code is unchanged from the previous lesson.

01:26 The model provider and response will be printed unless no model is found.

01:30 Open up the terminal and run the script.

01:33 python fallback_models.py. And the result is gpt-3.5-turbo served by OpenAI, along with a very wishy-washy answer, as expected. And that’s how you can implement fallbacks.

01:48 You can see that GPT-5 wasn’t available, most likely it’s not on the free plan, so GPT-3.5 Turbo was used. This is a great way to ensure that even if your favorite model can’t be served, your production application doesn’t have to experience downtime.

02:01 So at this point, you’ve seen a few of the options and techniques for selecting models with OpenRouter. In the next lesson, you’ll look at some considerations to help guide your selections.