One API to Rule ALL LLMs — 24K★ Open-Source Middleware Unifies 30+ AI Providers

#ai #python #devops #opensource

Summary

One API (24K★, MIT license) consolidates 30+ LLM providers — OpenAI, Claude, DeepSeek, Gemini, local Ollama — behind a single OpenAI-compatible endpoint. Deploy with Docker once, add all your API keys, and every tool/app suddenly works with any model. Load balancing, budget control, channel failover, and usage analytics are built in.

The Problem It Solves

Problem	Your Current Fix	With One API
Different SDKs per provider	Install 10 packages	One OpenAI SDK
Key management chaos	Spread across env files	Centralized key vault
Budget tracking	Manual spreadsheet	Built-in quota
Cost optimization	Hard to compare	Log all costs

Architecture

Your App (OpenAI SDK) → One API Server → OpenAI/Claude/DeepSeek/Gemini/Ollama/30+ more

The Go backend translates standard OpenAI format to each provider's native format.

Supported Providers

Category	Providers
Major AI	OpenAI, Anthropic Claude, Google Gemini, Mistral
Chinese LLMs	DeepSeek, Alibaba Qwen, Baidu ERNIE, Zhipu GLM, MiniMax
Open Source	Ollama (local), vLLM, LocalAI
Other	Amazon Bedrock, Azure OpenAI, Together, Groq

Key Features

Unified API — All providers speak OpenAI format. Your existing code works without changes.

Channel Management — Load balancing across API keys, failover, priority settings.

Budget Control — Per-user quotas, monthly limits, token counting.

Usage Analytics — Dashboard: requests/minute, cost breakdown, model popularity.

Quick Start

docker run -d --restart always \
  -p 3000:3000 \
  -v /home/ubuntu/data:/data \
  --name one-api \
  justsong/one-api

# Open http://localhost:3000 → add API keys
# Use http://localhost:3000/v1 as OpenAI base URL

Practical Use Case

Set DeepSeek as primary (cheap), Ollama as local fallback, Claude for complex edits. One endpoint, zero friction.

FAQ

Q: Production-ready? A: Yes. 24K★ stars, 100+ contributors.

Q: Supports streaming? A: Yes. SSE streaming for all providers.

Q: MIT license? A: Use anywhere, modify, sell. No restrictions.

Q: Performance overhead? A: Minimal. Go backend adds ~5ms latency.

24K★ and growing. One endpoint to rule them all.

DEV Community