I Shipped DeepSeek, Qwen, Kimi, and GLM: A CTO's Field Notes

I gotta say, i Shipped DeepSeek, Qwen, Kimi, and GLM: A CTO's Field Notes

Six weeks ago I was staring at a $41,000 monthly OpenAI bill that had quietly tripled over a quarter. As CTO of a 14-person startup processing roughly 18 million LLM tokens a day for a mix of customer support automation, document parsing, and code review tooling, I had a problem: the unit economics of my product were getting worse every month, and I was one pricing change away from being unprofitable.

So I did what any reasonable engineer with a margin problem does. I spent a weekend routing traffic to four Chinese model families through a single unified endpoint, instrumented everything, and made a decision. Here's what actually happened when I put DeepSeek, Qwen, Kimi, and GLM into production and let real users hammer them.

Why This Matters When You're Shipping at Scale

The vendor lock-in question isn't theoretical anymore. If your entire stack assumes GPT-4o at $10.00/M output tokens, you're building a business on rented land with a landlord who can reprice whenever they want. I've watched three competitors get margin-crushed by inference cost spikes in the last year alone. That's not a risk I'm willing to take when I have 8,000 paying customers.

What I needed was a production-ready fallback that I could route to when costs spiked, with minimal code changes. Global API gave me a single OpenAI-compatible endpoint that exposes all four Chinese model families, which meant I could A/B test without rewriting my integration layer. That's the only sane way to evaluate models — in production, with real prompts, not in a Jupyter notebook.

The Numbers Side by Side

Here's the snapshot I sent to my CEO when I made the case to switch our default model:

Model Family	Price Range (output $/M)	Best Budget Pick	Best Overall Pick	Vision	Reasoning
DeepSeek	$0.25–$2.50	V4 Flash @ $0.25	V4 Flash @ $0.25	Limited	Strong
Qwen	$0.01–$3.20	Qwen3-8B @ $0.01	Qwen3-32B @ $0.28	Yes (VL, Omni)	Strong
Kimi	$3.00–$3.50	N/A — all premium	K2.5 @ $3.00	No	Best in class
GLM	$0.01–$1.92	GLM-4-9B @ $0.01	GLM-5 @ $1.92	Yes (GLM-4.6V)	Strong

Three things jumped out immediately. First, the floor price for capable models is absurdly low — a tenth of a cent per million tokens for Qwen3-8B. Second, DeepSeek V4 Flash at $0.25/M was 40x cheaper than what I was paying. Third, Kimi doesn't even pretend to compete on price; it's a reasoning specialist and you pay reasoning rates for that capability.

DeepSeek Became My Default in 48 Hours

I'll be direct: DeepSeek V4 Flash is the best cost-to-quality ratio I've tested, period. At $0.25/M output tokens, it handled 70% of my production traffic with no measurable quality regression on English content generation, code review, and structured extraction tasks.

The model lineup is tight and well-positioned:

• V4 Flash at $0.25/M — my workhorse
• V3.2 at $0.38/M — newer architecture, marginal gains
• V4 Pro at $0.78/M — when I need the extra quality for client-facing copy
• R1 at $2.50/M — the reasoner, reserved for math and multi-step logic
• Coder at $0.25/M — specialized for code tasks

What sold me was the speed. V4 Flash was pushing around 60 tokens/second in my benchmarks, which is fast enough that latency wasn't a customer-facing issue. My p95 went from 1.8 seconds on GPT-4o to 1.1 seconds on V4 Flash for equivalent tasks. Faster and cheaper. That's the kind of win you don't argue with.

The honest downside: vision is limited, and if you need Chinese language nuance (poetry, idioms, regional variation), GLM and Kimi will outclass it. I had to route a small slice of traffic to other providers for those cases. Not a dealbreaker, but not a single-vendor solution either.

Qwen Solved a Model Selection Problem I Didn't Know I Had

Here's the thing nobody tells you about running a multi-model architecture: the cognitive overhead of picking the right model per request is real. Qwen's lineup is so wide that it basically eliminates that problem for most use cases.

The spread is wild. Qwen3-8B at $0.01/M is so cheap I use it for tasks I would have dismissed as "not worth an API call" a year ago — simple classification, intent detection, basic summarization where quality bar is moderate. Qwen3-32B at $0.28/M handles my general-purpose load. Qwen3-Coder-30B at $0.35/M is a solid code model. Qwen3-VL-32B and Qwen3-Omni-30B at $0.52/M cover image and multimodal. And if I'm willing to pay enterprise rates, Qwen3.5-397B at $2.34/M is a legitimate reasoning model.

The reason Qwen matters for production thinking is the omni-modal story. Audio, video, image, text — all through one vendor with consistent API patterns. If you're building something that needs to handle a mixed-media input pipeline, Qwen collapses three vendor relationships into one.

The catch is naming. Qwen3.5 versus Qwen3.6 versus Qwen3-32B versus Qwen3-Coder-30B is genuinely confusing, and I had to build a translation table in my routing config to keep it straight. Alibaba ships models fast enough that documentation lags behind releases. Also, some of the mid-range models feel slightly overpriced relative to DeepSeek's equivalent tier. Qwen3.6-35B at $1/M is steep when V4 Flash is doing the same job for a quarter of the cost.

Kimi Is a Premium Tool, Not a Default

I want to be honest about Kimi. At $3.00–$3.50/M output, it is the most expensive model family in this comparison. K2.5 at $3.00/M is the headline model, and I am not routing baseline traffic to it.

What Kimi does is reason. The Moonshot team has tuned these models for multi-step problems, complex logic, and tasks where you'd otherwise need to chain multiple LLM calls together. On my internal reasoning eval — the kind of thing where I throw it a 400-word policy document and ask it to extract every conditional rule — Kimi scored meaningfully higher than anything else in the test.

I use it for two specific things: compliance review on financial documents, and code migration planning where I need the model to reason about dependency chains. Both are high-value, low-volume use cases where the per-token cost is justified by the quality delta. Everything else goes to cheaper models.

The other thing to know: no vision, no multimodal. It's a text model that does text really well. If your product needs image understanding, Kimi isn't in the conversation.

GLM Earned Its Place in the Routing Table

Zhipu's GLM lineup surprised me. I went in expecting a budget option and found something more interesting.

GLM-4-9B at $0.01/M is, alongside Qwen3-8B, the cheapest viable model in this entire comparison. I use it for high-volume classification work — routing incoming tickets, tagging support conversations, basic content moderation. The cost is so low I stopped optimizing the prompts.

GLM-5 at $1.92/M is the flagship, and it's solid. Not the best at any one thing, but genuinely good at most things, with particular strength on Chinese-language content. If you're serving a Chinese-speaking market or doing any cross-language work, GLM is the answer. The CJK benchmark scores are notably stronger than DeepSeek or Qwen on tasks involving classical references, formal business Chinese, and regional idiom.

GLM-4.6V handles vision, which rounds out the lineup. The only model I'm slightly disappointed by is GLM's code generation — it's competent but not best-in-class. For code-heavy workloads, I'm still routing to DeepSeek or Qwen.

The Production Routing Config That Cut Our Bill 80%

After three weeks of testing, I landed on this architecture. The principle: match model capability to task value, not to task complexity. A complex-sounding task that's low-stakes doesn't need a premium model.

• Tier 1 — Trivial tasks (60% of traffic): Qwen3-8B or GLM-4-9B at $0.01/M
• Tier 2 — Standard workloads (25% of traffic): DeepSeek V4 Flash at $0.25/M or Qwen3-32B at $0.28/M
• Tier 3 — High-quality output (10% of traffic): DeepSeek V4 Pro at $0.78/M or Qwen3.5-397B at $2.34/M
• Tier 4 — Reasoning or Chinese nuance (5% of traffic): Kimi K2.5 at $3.00/M or GLM-5 at $1.92/M

Weighted average output cost: roughly $0.31/M tokens across 18 million tokens daily. My previous blended rate on GPT-4o was $8.40/M. The math is not subtle.

The other thing I get from this architecture is failover. When DeepSeek had a 40-minute outage two weeks ago, I shifted Tier 2 traffic to Qwen3-32B with a single config change. No customer-facing downtime. That's the vendor lock-in insurance I was looking for.

The Code: How I Actually Wired It Up

Here's the core pattern I use. The OpenAI SDK works against Global API's endpoint with no code changes other than the base URL, which is the entire point.

from openai import OpenAI

client = OpenAI(
    api_key="ga_xxxxxxxxxxxx",
    base_url="https://global-apis.com/v1"
)

response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "Summarize the key risks in this contract clause."}],
    temperature=0.3
)
print(response.choices[0].message.content)

For the reasoning tier, I swap the model string and I'm done:

# Tier 4: Complex reasoning, kimi k2.5
response = client.chat.completions.create(
    model="kimi-k2.5",
    messages=[{"role": "user", "content": "Walk through the dependency implications of upgrading React 18 to 19 in a Next.js 14 app with three shared component libraries."}],
    temperature=0.2
)

The router itself is a thin layer that inspects the request type, customer tier, and required capability flags, then dispatches to the right model. About 200 lines of code. The hardest part was writing the model selection logic, not the API integration.

The Lock-In Question, Answered Honestly

The thing I tell other CTOs who ask about this evaluation is: don't pick one vendor. Pick an abstraction layer.

Global API gave me the abstraction I needed. One endpoint, one auth pattern, one SDK call, and I can route to any of these four model families. When GLM drops a new flagship next quarter, or when Qwen ships something that beats DeepSeek on code, I change a config value and ship. My product roadmap is no longer hostage to a single provider's pricing decisions.

This is the real ROI. Not the 80% cost reduction — though that matters. The ROI is that I can move fast. If a competitor shows up with a model that does something I need, I can have it in production within a day. If a model family gets degraded, I shift traffic. If pricing changes, I rebalance. That's what production-ready means in 2026.

What I'd Do Differently If I Started Today

If I were starting this evaluation from scratch, I'd skip the benchmark theater. HumanEval and MBPP scores correlate loosely with what your users actually need. Run your real prompts through these models. Score the outputs against whatever quality signal matters to your product —