Developer tools
LLM Gateway alternatives
2 open-source projects that already do this job. Free to use - not free to run, which is what the numbers on the right are for.
The projects
BerriAI/litellm★55,829 GitHub starslicence unclearRust core AI gateway exposing OpenAI-compatible API with cost tracking, guardrails, load balancing, and logging.Open on GitHub ↗coaidev/coai★9,273 GitHub starsApache-2.0 licenceMulti-tenant unified LLM gateway with admin, billing, routing, cost management, and support for many providers.Open on GitHub ↗
What you give up either way
- Hosted SLAs, uptime guarantees and managed enterprise support
- Out-of-the-box SOC 2 attestation and vendor compliance evidence
- Any proprietary integrations, paid billing/credit flow and negotiated volume discounts
- Polished analytics dashboards and built-in model directory
An open-source project doing the core job proves a replacement is possible. It does not prove it matches LLM Gateway, and it does not prove anyone has run it for a month. How we decide.