How I Built an AI API Gateway That Cuts Costs by 80% Without Changing Code

A few months ago, I was building a side project that relied heavily on OpenAI's API. The bills were adding up fast. I needed a cheaper alternative, but I didn't want to rewrite my code for every new provider.

That's when I discovered Chinese LLMs like Zhipu GLM-4. The quality was comparable to GPT-3.5, but the cost was significantly lower — about 80% less per token.

The catch? Each provider has its own API format, authentication method, and SDK. Switching providers meant rewriting code.

So I built a unified API gateway.

The Solution

I used an open-source tool called One API to create a gateway that translates standard OpenAI-style requests into calls to Chinese LLM providers.

Here's the architecture:

Developer → API Gateway → One API → Zhipu AI

The flow is simple:

1. Developer sends a request in OpenAI format
2. API Gateway (aitokenshub.net:3000) receives it
3. One API translates it to the target provider's format
4. Zhipu GLM-4 processes the request and returns the result

The best part: the response comes back in the exact same format as OpenAI. Your code doesn't change.

Quick Demo

Here's how to test it with a simple curl command:

bash
curl http://aitokenshub.net:3000/v1/chat/completions \
  -H "Authorization: Bearer your_token_here" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-4-flash",
    "messages": [{"role": "user", "content": "Hello, introduce yourself"}]
  }'