One-Sentence Conclusion
Of the three access methods, the official direct connection has the lowest nominal unit price but not necessarily the lowest effective cost; the self-built LLM gateway delivers the largest cost reduction but carries the highest maintenance cost; and hosted token collective procurement is the optimal balance point of "cost-reduction magnitude × maintenance cost". Selection should be based on effective cost, not nominal unit price.
Positioning of the Three Access Methods
| Method | Architecture | Best suited for |
|---|---|---|
| Official direct connection | Calls the upstream official API directly, billed at the published list price | Single model, low usage, minimal middle layer |
| Self-built LLM gateway | Self-maintained routing, caching, and high availability | High usage, a dedicated infrastructure team, strong pipeline control |
| Hosted token collective procurement | Aggregated gateway: dual-protocol endpoints + routing engine + cache billing | Multiple models, teams, zero-ops |
The Effective-Cost Formula
Effective cost = nominal cost (official list price × discount factor × usage)
+ maintenance cost (labor + infrastructure + incident handling)
− cache savings (cache hit rate × cached_input price difference)
− tiering savings (high-spec tasks downgraded to matched models)
− price-comparison savings (real-time channel monitoring → dispatch to the optimal channel)
The differences among the three methods are concentrated in maintenance cost and the three savings items.
Comparison with Real Numbers
Scenario: a 6-person coding agent team consuming 100M tokens per month. Official output pricing: flagship $30/M, mid-tier $12/M, batch $1.2/M; cache-read pricing estimated at roughly 10% of the official full price; cache hit rate 60%, of which 40% of daily tasks can be downgraded to the mid-tier.
Option 1: Official direct connection
- Nominal cost: 100M × $30 = $3,000
- Maintenance cost: ≈$0
- Cache savings: requires controlling the prefix yourself; actual hit rate assumed at 30%: 30M × ($30 − $3) = $810
- Tiering/price-comparison savings: $0 (single-model direct connection)
- Effective cost: $3,000 − $810 ≈ $2,190
Option 2: Self-built LLM gateway
- Nominal cost: 100M × $30 = $3,000 (upstream price)
- Maintenance cost: infrastructure + labor, estimated at $200/month, excluding incident handling
- Cache savings (hit rate 60%): 60M × ($30 − $3) = $1,620
- Tiering savings: 40M × ($30 − $12) = $720
- Effective cost: $3,000 + $200 − $1,620 − $720 ≈ $860 (you must bear all maintenance and incidents yourself)
Option 3: Hosted token collective procurement
- Nominal cost: based on the discount factor (0.1 for the OpenAI family), 100M × $30 × 0.1 = $300
- Maintenance cost: ≈$0 (platform-maintained)
- Cache savings (cached_input billed automatically, hit rate 60%): 60M × ($3 − $0.3) ≈ $162
- Tiering savings: 40M × ($3 − $1.2) = $72
- Effective cost: $300 − $162 − $72 ≈ $66 + platform service fee
The figures above are illustrative estimates; prices vary with model tier and usage mix. The order-of-magnitude conclusion is stable: hosted collective procurement layers the discount factor on top of cache, tiering, and price-comparison savings, and is usually optimal on the balance between cost reduction and maintenance cost.
Applicability Boundaries of the Three Methods
Official direct connection: usage below 10M/month, single model, strong control or compliance requirements over the pipeline, and an existing official key. When none of the three savings can be effectively captured, direct connection is the simplest.
Self-built LLM gateway: monthly usage in the hundreds of millions and above, a dedicated infrastructure team, and mandatory control over the data pipeline. The hosted service fee is no longer worth paying, and you have the operational capability to build and run your own.
Hosted token collective procurement: multiple models, 10M to hundreds of M, no dedicated infrastructure team — the default choice for the vast majority of teams, sitting in the sweet spot of "cost-reduction magnitude × maintenance cost".
Common Questions
Q: Is the direct-connection cache price better than collective procurement?
Official cache pricing does exist, but you must precisely control the prefix structure and configure caching yourself. Collective procurement platforms automate cache billing, tiering, and price comparison — what they reduce is engineering cost, not just price.
Q: Does a self-built gateway have a higher cost-reduction ceiling?
It may be higher on pure unit price, but once high availability, caching, failover, and manual maintenance are factored in, a self-built setup only becomes cost-competitive at scale.
Q: Is there hidden markup in hosted collective procurement?
It depends on the platform. A healthy pricing structure can be attributed to the discount factor, cache billing, and volume negotiation, with service fees stated openly. The verification standard is request-level billing and discount attribution.
Q: Can the access methods be mixed?
Yes. A common architecture is official direct connection as a fallback and hosted collective procurement as the primary path. It is advisable to first run your main pipeline smoothly on one platform, then evaluate multi-channel redundancy.
Summary
Comparing the three access methods should use the effective-cost formula: effective cost = nominal cost + maintenance − cache − tiering − price comparison. Official direct connection suits very small volumes, a self-built gateway suits very large volumes plus an infrastructure team, and hosted token collective procurement — with its discount factor, three layers of savings, and zero-ops — is the optimal balance point for the vast majority of teams. Sign up for TeamoRouter, plug in your real usage and discount factor, and calculate the effective-cost difference among the three methods.
Top comments (0)