The Problem
Managing multiple AI model providers is a mess. Each has its own API, pricing, and quirks.
I got tired of juggling keys and decided to build one gateway to rule them all.
What I Built
An OpenAI-compatible API gateway that routes requests across 43 models from 13 providers—transparently, with cost tracking and smart routing.
Live Demo (Cost Calculator): https://csx0574--calculator.modal.run
Features
- 43 models, 13 providers — OpenAI, Anthropic, Google, Meta, Mistral, Zhipu, DeepSeek, Minimax, Groq, Fireworks, Novita, Kampute, XAI
- OpenAI-compatible endpoint — Drop-in replacement for your existing OpenAI code
- Smart routing — Choose by cost, speed, or balance
- Cost tracking — See exactly how much each request costs
Quick Start
git clone https://github.com/csx0574/ai-multi-model-gateway.git
cd ai-multi-model-gateway/gateway
pip install -r requirements.txt
export OPENAI_API_KEY=sk-...
python gateway.py
Use It Like OpenAI
import openai
openai.api_base = "https://csx0574--gateway.modal.run/v1"
openai.api_key = "user-api-key"
response = openai.ChatCompletion.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Hello!"}]
)
Smart Routing
Pass mode in the request to let the gateway choose:
| Mode | Behavior |
|---|---|
cost |
Cheapest model that gets the job done |
speed |
Fastest response |
balanced |
Best cost/quality trade-off |
Why This Matters
Chinese AI providers (Zhipu GLM, DeepSeek, Minimax) offer surprisingly competitive pricing—sometimes 10x cheaper than OpenAI for comparable quality. But they're scattered and hard to integrate.
This gateway unifies them under one OpenAI-compatible API.
Top comments (0)