Why we built this
If you want to use Chinese frontier models — DeepSeek, Kimi, Qwen, MiniMax, Doubao — as an international developer, you quickly hit friction: every provider has its own auth scheme, its own billing quirks, and several require mainland-China payment methods or business verification. We run these models in production ourselves, so we built the gateway we wanted to have: one OpenAI-compatible endpoint, one API key, and automatic routing across 16 Chinese LLMs.
The result is HOUJIAYAN API (https://api.houjiayan.com). These are our architecture notes.
Architecture
Client (any OpenAI SDK)
│ https://api.houjiayan.com/v1
▼
Cloudflare (WAF / rate limiting / TLS)
▼
NewAPI (forked) — channels, API keys, quota, billing
│ │
▼ ▼
Custom smart Custom payment bridge (standalone service)
router ├─ WeChat Native Pay (APIv3)
(quality / └─ NOWPayments (USDT-TRC20)
balanced /
cost tiers)
│
▼
16 upstream model channels (11 enrolled in auto-routing)
Gateway: a fork of NewAPI.NewAPI is a mature open-source API gateway popular in the Chinese LLM ecosystem. We kept its channel abstraction, token management and quota accounting, and customized pricing so that rates stay exactly at upstream official pricing — no markup — with a live price table in the console. Rates go as low as ¥1.00 / 1M tokens.
Smart router: a custom stateless service.Beyond the standard `/v1/chat/completions` (where you can pin any of the 16 models directly), we expose `POST /v1/auto/chat/completions` with three routing tiers:
- `quality` (default): pick the highest-quality model for the task class;
- `balanced`: the quality/cost sweet spot;
- `cost`: cheapest adequate model, for batch workloads.
The router scores candidate channels by task features (context length, code vs. prose, expected output length) against a curated quality-tier table. Fourteen of the sixteen models participate in the auto pool. On upstream failure it degrades to the next-best channel.
Payment bridge: the hardest part.NewAPI doesn't handle payments, so we built a standalone pay\_bridge service:
- WeChat Native Pay APIv3 for domestic users (QR-code Native orders → signed callback → signed epay-style notify into the gateway);
- NOWPayments for USDT-TRC20 for users outside mainland China (IPN callback → FX conversion → credit).
The bridge talks to the gateway through a signed callback protocol so the gateway credits accounts natively — no double-entry writes.
Lessons learned
1. WeChat's new "platform public key" mode.New merchant accounts are forced onto public-key verification (key IDs prefixed with `PUB_KEY_ID_`) instead of the old platform-certificate chain. Nearly all tutorials and SDK examples still assume certificate mode. If your callback signature verification fails on a new merchant account, this is why.
2. `base_url` must end with `/v1`.The OpenAI SDK concatenates paths onto `base_url`, so `https://api.houjiayan.com` alone 404s. We added a gentle redirect at the edge, and put the correct snippet at the top of the docs:
python
from openai import OpenAI
client = OpenAI(api_key="sk-...", base_url="https://api.houjiayan.com/v1")
3. Usage accounting across providers.Upstreams report token usage inconsistently (some only at the end of a stream). We normalize accounting to gateway-parsed usage with a local estimation fallback for streams that never report it, and keep per-request audit detail.
Access & pricing (full disclosure)
- Endpoint: `https://api.houjiayan.com/v1` (OpenAI-compatible); auto-routing at `POST /v1/auto/chat/completions`
- Models: 16 Chinese LLMs directly addressable (DeepSeek V4 Pro/Flash, Kimi K3, MiniMax M3, Qwen3.6/3.7 series, Doubao Seed Code, etc.); 11 in the auto-routing pool
- Pricing: identical to upstream official pricing, no markup; live table in the console; from ¥1.00 / 1M tokens
- Sign-up credit: ¥6.6 (~$1) free trial credit; stays valid with ongoing usage; reclaimed only after 7 consecutive days with no API calls and no logins
- Top-up: WeChat Pay (min ¥50, 5.5% service fee); USDT-TRC20 (min $25, 5% service fee, non-mainland-China residents only)
- Refunds: WeChat top-ups refundable within 24h if unused (service fee excluded); crypto top-ups are non-refundable
- Docs: https://houjiayan.com/docs/
- Operator: Inner Mongolia Huozhong Intelligent Technology Co., Ltd.; support@houjiayan.com
What's next
Routing tiers are currently static + task-feature scoring. We plan to fold historical success rate, latency and user feedback into dynamic channel weights.Support more models and payment methods. Happy to compare notes with anyone running multi-upstream LLM gateways.
---
Top comments (3)
Great write-up — we run a similar multi-upstream gateway in production (SG-hosted, OpenAI-compatible, China frontier + OpenAI/Gemini). The thing your static tiers will hit: failover to the next-best channel silently kills prompt-cache hit rate, which can erase the cost win. Cache-aware routing + provider pinning matter more than the quality/cost labels. Happy to compare notes.
Great point — and you're right, we hit exactly that. After reading your comment we instrumented our router and confirmed the drift: classification drift and daily score-refresh drift were bouncing multi-turn sessions across providers, silently killing prompt-cache hits (DeepSeek cache-hit tokens run at roughly 1/10 the input price here, so it adds up fast on long sessions).
We've now shipped session pinning: conversation_id when the client provides one, falling back to a SHA-256 hash of the system + first-user prefix (hashes only — no prompt content ever stored). The arbitration rule is "cache wins inside the quality band, quality wins outside", with a 0.08 hysteresis so daily score updates don't flap sessions, and failover re-pins to whatever actually served with no failback. State is just in-memory LRU + SQLite WAL, no Redis. It's running in shadow mode this week while we measure drift rate and cache-hit deltas before flipping it on.
One limit we've documented honestly: for models where our downstream gateway picks the upstream channel internally, we can only pin at model granularity, not channel level.
Would genuinely love to compare notes — especially how you handle a pinned provider degrading mid-conversation: do you eat the cache miss and re-pin immediately, or tolerate the latency until the session ends?
Great write-up — running a similar multi-upstream gateway in prod (SG-hosted), and the session-pin-top via SHA-256 of system + first user turn is exactly the trick we landed on too (hash only, never store the prompt). A few things we learned the hard way: (1) treat prefix stability as an SLA and alert when hit-rate drops, not just when latency spikes; (2) the arbitration "cache wins inside the quality band, quality wins outside" with a ~0.08 latency floor is the right call — it stops daily score refreshes from silently re-pinning; (3) in-memory LRU + SQLite WAL for state is plenty. One question: in shadow mode, what drift rate between the shadow and live routing did you see before you trusted it?