I run an API gateway to 15 Chinese AI models. Here's what I learned in a month of watching real usage.

#ai #api #llm

People don't care about your model count

I launched with "15 models, one endpoint" as the headline. DeepSeek,
Qwen, GLM, Moonshot Kimi. Four providers. Streaming. Function calling.
JSON mode. I thought the breadth was the selling point.

Turns out, 62% of all playground calls went to a single model: Kimi K3.
Not because users read the docs and decided K3 was best. Because the
playground defaulted to it, it returned impressive results, and nobody
bothered switching.

The model count matters less than "what's the first thing I see work."
One great default beats fifteen choices when someone is trying to
evaluate you in 30 seconds.

The free tier is doing double duty

500K free tokens per month. No credit card. I worried it'd be abused.
It was — but not the way I expected.

19 users pushed past 500K. Some hit 685K. The quota check had a race
condition that let concurrent requests slip through. I fixed it with
an atomic check inside the existing lock. Read, verify, deduct — all
in one critical section.

The lesson: a free tier isn't just marketing. It's a load test. It
exposes the concurrency bugs in your billing code before real money
is on the line.

The onboarding gap is real and measurable

68% of registered users never made a single API call. They got a key,
opened the dashboard, saw a usage bar and a list of models... and left.

The fix was a post-login onboarding page. API key on screen with a
copy button. A curl command. Python SDK code. 30 seconds from login
to first response. Dormancy dropped from 68% to 65%.

Three percentage points sounds small. It's 110 users total — that's
three or four people actually using the product who wouldn't have
before. At this stage, that's meaningful.

Bots will find your signup flow within days

The moment GitHub OAuth went live and registration got easier for
humans, it got easier for bots too. /auth/send-verification-code
went from quiet to 2,600 requests per day in a week.

Five lines of in-memory rate limiting — one request per IP per minute
— dropped it to zero. No Redis. No CAPTCHA. No external service. A
dictionary, a lock, and a timestamp.

Simple problems don't always need complex solutions.

Prompt libraries are onboarding for the imagination

The deepest insight: users weren't failing because the API was hard.
They were failing because a blank input box is a test most people
can't pass.

24 ready-to-use prompts, six categories, each recommending a model.
"Find bugs → Kimi K3." "Translate → Qwen Max." One click fills the
playground. It's the difference between "what should I ask" and
"oh, I can do that."