Software:LiteLLM
| Developer(s) | BerriAI |
|---|---|
| Initial release | 2023 |
| Repository | https://github.com/BerriAI/litellm |
| Written in | Python and TypeScript |
| Type | LLM gateway |
| License | MIT License |
| Website | www |
LiteLLM is an open-source large language model (LLM) gateway developed by BerriAI. It provides a unified OpenAI-compatible application programming interface (API) for accessing and routing requests to different model providers. LiteLLM can be deployed as a self-hosted proxy server or used as a Python library within applications.
Functionality
LiteLLM provides a common interface for models and services from providers including OpenAI, Anthropic, Google Gemini, Amazon Bedrock, Microsoft Azure, and Hugging Face. It translates requests and responses between provider APIs into a consistent OpenAI-compatible format, allowing applications to switch between models without separately implementing each provider's interface.[1][2]
The software also provides model routing, load balancing, fallbacks, authentication, rate limiting, usage monitoring, and cost controls. Its proxy server can act as a centralized API gateway through which multiple applications or users access AI models.[2]
LiteLLM also supports the Model Context Protocol (MCP), an open protocol developed by Anthropic for connecting AI applications to external tools and data sources.[3]
Security
In March 2026, malicious versions of LiteLLM were distributed through the Python Package Index (PyPI) as part of a software supply-chain attack. TechCrunch reported that AI recruiting company Mercor was among the organizations affected by the compromise.[4]
Compatibility
Supported models
LiteLLM supports more than 100 large language models through a common OpenAI-compatible interface. Notable supported model families include:[2]
Compatible AI coding software
LiteLLM can be used as an inference gateway for AI-assisted software development tools, routing their model requests through the LiteLLM proxy. Documented integrations include:[5]
- Claude Code
- Cline[6]
- Codex CLI
- OpenCode
- Pi
See also
- List of AI-assisted software development tools
- OpenRouter – proprietary LLM gateway
References
- ↑ MSV, Janakiram (May 15, 2025). "LiteLLM: An open-source gateway for unified LLM access". https://www.infoworld.com/article/3975290/litellm-an-open-source-gateway-for-unified-llm-access.html.
- ↑ 2.0 2.1 2.2 "LiteLLM - Getting Started". BerriAI. https://docs.litellm.ai/.
- ↑ "Introducing the Model Context Protocol". November 25, 2024. https://www.anthropic.com/news/model-context-protocol.
- ↑ Singh, Jagmeet (March 31, 2026). "Mercor says it was hit by cyberattack tied to compromise of open source LiteLLM project". https://techcrunch.com/2026/03/31/mercor-says-it-was-hit-by-cyberattack-tied-to-compromise-of-open-source-litellm-project/.
- ↑ "LiteLLM Proxy CLI". BerriAI. https://github.com/BerriAI/litellm/blob/main/litellm/proxy/client/cli/README.md.
- ↑ "Configure LiteLLM Provider". https://docs.cline.bot/enterprise-solutions/configuration/remote-configuration/litellm/admin-configuration.
External links
Template:Large language models Template:AI-assisted software development
 |  |
