LiteLLM

LiteLLM is an open-source AI gateway from BerriAI that gives applications a single OpenAI-compatible interface to more than 100 large language model providers:

The Python library exposes a completion() function that takes the same message format as the official OpenAI client. Switching from OpenAI to Anthropic, Gemini, or a locally hosted Ollama model means changing a model string rather than rewriting call sites. LiteLLM normalizes streaming responses and provider errors into OpenAI-style equivalents.

The same project ships a self-hosted proxy server aimed at platform teams that want one key in front of every LLM they use. The proxy issues virtual keys per team or project, tracks spend against budgets, enforces rate limits, and retries or falls back across deployments. Callbacks forward request traces to observability platforms such as MLflow and Langfuse.

LiteLLM routes requests but doesn’t host models, so you bring your own provider credentials and keep your existing billing relationships. Hosted gateways like OpenRouter take the opposite trade and bill every provider through one account, while Portkey and Kong AI Gateway compete with the LiteLLM proxy on governance features rather than on provider coverage.

The core project is MIT-licensed and can run air-gapped. A paid enterprise tier adds single sign-on, JWT authentication, and audit logs.

Official website: litellm.ai

Related Resources

Course

Leverage OpenAI's API in Your Python Projects

Learn how to use the ChatGPT API with Python's openai library to send prompts, control AI behavior with roles, and get structured outputs.

intermediate ai api

For additional information on related topics, take a look at the following resources:

• How to Integrate Local LLMs With Ollama and Python (Tutorial)
• How to Use Ollama to Run Large Language Models Locally (Tutorial)
• Pydantic AI: Build Type-Safe LLM Agents in Python (Tutorial)
• Build an LLM RAG Chatbot With LangChain (Tutorial)
• Connecting LLMs to Your Data With Python MCP Servers (Course)
• How to Integrate Local LLMs With Ollama and Python (Quiz)
• How to Use Ollama to Run Large Language Models Locally (Quiz)
• Building Type-Safe LLM Agents With Pydantic AI (Course)
• Pydantic AI: Build Type-Safe LLM Agents in Python (Quiz)
• First Steps With LangChain (Course)
• Build an LLM RAG Chatbot With LangChain (Quiz)