The most expensive part of AI calls isn't the tokens
If you build on LLM APIs, you have probably hit the same wall: the model works great, but the cost won't sit still.
Not because the model is expensive — because the cost is unpredictable.
- Providers price by time of day. At peak, the unit price can double, and the budget you set at the start of the month is gone by the end.
- One supplier stutters and your production environment throws 5xx; retries don't save you.
- Everyone mixes public list prices, so when finance asks "did we actually save anything?" at month-end, you have no clean answer.
At Yingsuan we run an LLM API gateway, and these three problems are our daily work. This note does not dump parameters or leak secrets. It explains, in engineering terms, how we turned "unpredictable" into "predictable" — and the "stable-price computing" thinking behind it.
Note: every "saved" figure in this post means the book difference against the public retail peak price. It is a real, per-call number you can verify — not a promise of returns.
1. Admit it: the unit price moves
Major LLM providers generally use dynamic pricing: expensive when compute is tight at peak, cheap when it is abundant off-peak. That is optimal for them, but a nightmare for the caller — you never know which price the next call lands on.
Our first call: don't try to predict the price; keep the price swing outside the gateway.
Two layers:
- Aggregate multiple suppliers behind one OpenAI-compatible endpoint. The caller talks to one endpoint; behind it sit several sources.
- Time-aware scheduling: the gateway watches each source's tight/abundant state in real time, routing more off-peak and avoiding peaks intelligently — turning the peak/off-peak spread into real savings for the caller.
This layer pays off "passively" — you configure nothing; calls just get cheaper.
2. Stable-price lock: nail the unit price down
Aggregation saves money, but how much still floats with peaks. For many B2B, budget-driven teams, "save a bit" loses to "lock it down."
So we added a second layer: stable-price lock mode.
Once on, your unit price stays fixed no matter how the upstream swings. The bill looks like a utility bill — units used × fixed unit price, a number you can read at a glance on the last day of the month. Finance stops chasing you about "what share was peak this month."
One engineering trade-off: locking the price means the gateway absorbs the peak/off-peak pooling risk for you. We set the locked price inside a safe band — never at a loss, and always below the reference baseline — and the platform eats the volatility instead of passing it to you. That is exactly why we can say "savings stay positive."
3. Zero-perception switch: when upstream breaks, you don't
The other value of aggregation is availability.
When one supplier stutters, rate-limits, or briefly drops, the request is routed to a healthy node in milliseconds. To the business, it looks like "waited a few dozen milliseconds longer" — not "the API is down." We call it "zero-perception switch": not that nothing ever breaks, but that when it does, you don't feel it.
We keep this mechanism restrained: we don't publish exact thresholds or the routing order, because that is an attack surface. All you need to know externally is one thing — a single point of failure never becomes your failure.
4. Transparent: how much you saved, look it up yourself
We hate the "black-box saving" most.
So the gateway records every call's "paid / reference baseline / saved / save rate" and ships two outward tools:
- Savings ledger: see today's and this month's cumulative savings in real time.
- Monthly bill: aggregated by month, with period comparison; managers view the global picture with an admin key, developers view their own with their key (data isolated per key).
Key point: the reference baseline uses only the public retail peak price. We will never compare against our channel floor price — that is our house secret, not your bill. Every "saved" you see is an honest difference computed from public prices.
5. Compliance and proof: capability landed where it can be checked
This "stable-price computing" isn't a slide-deck concept. Behind it is our self-developed stable-price computing method (a time-aware dynamic pricing approach, patent application accepted and currently pending examination), and the LLM API gateway service plus AI compute scheduling service are already listed on the Yunnan Data Circulation Trading Platform with provincial + regional dual-node registration.
Servers are deployed in mainland China (Kunming hub), offering low-latency access to South and Southeast Asia, with a clear data outbound boundary.
Closing: keep the complexity, leave the predictability to you
A gateway builder has one plain duty: the upstream's complexity shouldn't become the caller's complexity.
Peak/off-peak swings, single points of failure, unreadable bills — we block those at the gateway. Cost predictability, transparent savings, failure-free feel — we leave those to you.
If AI API cost is eating you alive, come grab a free key at Yingsuan and run it. Get it working first, then decide whether to upgrade. You never lose.
- Free key: https://yingsuan.top/payment.html#free-trial?utm_source=devto&utm_medium=blog&utm_campaign=stable_price
- Savings ledger: https://yingsuan.top/savings.html?utm_source=devto&utm_medium=blog&utm_campaign=stable_price
- Monthly bill: https://yingsuan.top/monthly.html?utm_source=devto&utm_medium=blog&utm_campaign=stable_price
This is an engineering practice note with no return promises. Patent status follows public information from the China National Intellectual Property Administration (currently "pending examination"). Upstream policy follows each supplier's official publication.
Top comments (0)