DEV Community: Hamimelon2026

I Tried GLM for the First Time. Here's the Minimal Setup That Actually Worked.

Hamimelon2026 — Fri, 28 Aug 2026 02:45:55 +0000

Everything I Found Assumed I Already Knew What GLM Was

I kept seeing GLM mentioned alongside models I already knew — DeepSeek, Qwen — and every time I looked it up, the documentation jumped straight into advanced parameters without ever showing a plain, working first call. So here's the version of that first call I wish I'd found.

GLM is a large language model family originally developed by a Tsinghua University research group, now maintained commercially as Z.ai. It's OpenAI-API-compatible, which meant I didn't need a new SDK — just a different base URL and model name.

Getting an API Key

Sign up on Z.ai's platform (or through BigModel, the mainland-facing endpoint) and generate a key from your account dashboard. Same basic flow as most other LLM providers at this point — nothing GLM-specific to worry about here.

pip install openai python-dotenv

The Minimal Working Call

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()

client = OpenAI(
    api_key=os.getenv("GLM_API_KEY"),
    base_url="https://api.z.ai/api/paas/v4"
)

response = client.chat.completions.create(
    model="glm-5.3",
    messages=[
        {"role": "user", "content": "Explain what a hash map is in two sentences."}
    ],
)

print(response.choices[0].message.content)

That's the whole first call. If you've used any other OpenAI-compatible provider, this will look identical except for the base_url and model string.

The One Thing That Actually Tripped Me Up

The current GLM models default to an extended "thinking" mode — the model generates an internal reasoning trace before answering, which adds latency you might not expect for a simple prompt. On GLM-5.3 specifically, this can't be fully turned off (older versions like 5.2 allowed a disabled setting; 5.3 replaced that with graduated effort levels instead). If you're just testing your first call and it feels slower than you expected, that's likely why — not a network issue, not a bug in your code.

response = client.chat.completions.create(
    model="glm-5.3",
    messages=[{"role": "user", "content": "Explain what a hash map is in two sentences."}],
    extra_body={"thinking": {"type": "enabled", "effort": "low"}},
)

For a simple factual prompt like the one above, effort: low gets you close to the fastest response the model can give — worth setting explicitly rather than leaving it on the default if latency matters for what you're building.

A Slightly More Realistic Example

Most tutorials stop at a single hardcoded prompt, which doesn't tell you much about actually building something. Here's a small wrapper that takes a variable prompt and handles the basic case of an empty or malformed response:

def ask_glm(prompt, effort="low"):
    try:
        response = client.chat.completions.create(
            model="glm-5.3",
            messages=[{"role": "user", "content": prompt}],
            extra_body={"thinking": {"type": "enabled", "effort": effort}},
        )
        content = response.choices[0].message.content
        if not content:
            return "No response content returned."
        return content
    except Exception as e:
        return f"Request failed: {e}"

print(ask_glm("Summarize the plot of a story about a lighthouse keeper in one sentence."))

Nothing sophisticated — just enough structure to actually build on top of, rather than a single throwaway print() call.

Where I Took This After the First Call

Once I had this working, I wanted to see how GLM's output compared to a couple of models I was already using for a small side project, without setting up a completely separate client and auth flow for each one. I ended up testing GLM through RouteAI alongside those other models — same request format shown above, just a different base_url and model name per test. That's a convenience note for anyone doing model comparisons, not a requirement for getting GLM working on its own; the code above runs fine against Z.ai's endpoint directly.

If You're Setting This Up Yourself
Start with the direct endpoint before adding anything else — get one working call before worrying about comparisons or routing
If your first call feels slow, check your thinking/effort settings before assuming something's broken
The OpenAI-compatible format means most of what you already know about calling other LLM APIs transfers directly — don't expect to relearn much

TL;DR: GLM's API is OpenAI-compatible, so getting started is mostly a different base_url and model name. The main gotcha for first-timers is the default thinking mode adding unexpected latency — set an explicit effort level if speed matters for your prompt. Full minimal example above.

Worth exploring if this is relevant to your stack: www.fastrouteai.com

I Ran the Same 40 Prompts Through Qwen2.5 and Qwen3. Here's the Script and Results.

Hamimelon2026 — Wed, 26 Aug 2026 08:38:53 +0000

Why I Didn't Just Trust the Benchmarks

Qwen3's published benchmarks look like a clean win over Qwen2.5 — real gains on MMLU-Pro, MATH, and coding tasks, plus a much larger training set (roughly 36 trillion tokens versus Qwen2.5's 18 trillion) and support for far more languages. On paper, swapping in Qwen3 should have been an easy call for my project.

I still wanted my own numbers, because a general benchmark tells you what a model can do on average, not what it does on the specific, sometimes-weird prompts your actual application receives. So I wrote a small script to run the same prompt set through both models and log the results side by side.

The Eval Script

Nothing fancy — this loops through a list of test prompts, calls both models with each one, and writes the outputs to a CSV for manual review.

import csv
import time
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://dashscope.aliyuncs.com/compatible-mode/v1"  # Qwen's own endpoint
)

test_prompts = [
    "Classify this support message: 'My payment failed twice this week.'",
    "A user says their invoice total doesn't match what they were quoted. What category does this fall under, and what follow-up question would you ask?",
    # ... add your own prompts here
]

models_to_compare = ["qwen2.5-72b-instruct", "qwen3-235b-a22b", "qwen3-235b-a22b-instruct"]

def run_eval(prompts, models):
    results = []
    for model in models:
        for prompt in prompts:
            response = client.chat.completions.create(
                model=model,
                messages=[{"role": "user", "content": prompt}],
                temperature=0,
            )
            results.append({
                "model": model,
                "prompt": prompt,
                "output": response.choices[0].message.content,
            })
            time.sleep(0.5)  # basic pacing to avoid rate limits
    return results

results = run_eval(test_prompts, models_to_compare)

with open("qwen_comparison.csv", "w", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=["model", "prompt", "output"])
    writer.writeheader()
    writer.writerows(results)

I deliberately kept temperature=0 to reduce randomness between runs — you want the comparison to reflect model differences, not sampling noise.

What I Actually Found

Out of 40 prompts specific to my ticket-classification use case, plain Qwen2.5-72B-Instruct and Qwen3-235B-A22B (the default, hybrid-thinking variant) landed close to each other overall — Qwen3 won clearly on the more ambiguous, multi-step prompts, but on short, unambiguous ones, Qwen2.5's answers came back faster and were occasionally more directly usable without extra parsing.

The bigger difference showed up when I added qwen3-235b-a22b-instruct — the non-thinking Instruct variant Qwen released separately — as a third comparison point. It matched Qwen2.5's speed on simple prompts while still outperforming it on the harder ones, which lines up with something I found afterward while reading about the initial Qwen3 release: early evaluations of the original hybrid-thinking Qwen3 models showed them underperforming Qwen2.5 on some agentic and instruction-following benchmarks, which is part of why the dedicated Instruct and Thinking variants exist as a separate release.

That's a specific, checkable claim for your own use case, not a general one — the point isn't "Qwen3-Instruct is better," it's that "Qwen3" isn't one model, and comparing against the wrong variant will give you a misleading result.

A Few Things Worth Watching in Your Own Comparison
Pin temperature=0 unless your task actually benefits from sampling variance — otherwise you're comparing noise, not models
Test the actual variant you'd deploy, not just whichever one your first pip install example happens to reference — Qwen3's hybrid, Instruct, and Thinking variants behave differently enough that lumping them together will skew your conclusion
Weight your prompt set toward your real traffic distribution — if 90% of your inputs are simple, a benchmark heavy on hard reasoning tasks won't tell you much about your actual cost/latency tradeoff
Where I Took This Next

Once I had this script working against Qwen's own endpoint, I wanted to run the same comparison against a couple of other providers, mostly out of curiosity about whether my results were Qwen-specific or held more generally. I ended up routing the same script through RouteAI instead of maintaining separate client configs per provider — the eval loop above didn't change at all, just the base_url and the model names in the list. That's a convenience thing, not a result — the comparison numbers above came from testing directly against Qwen's models.

TL;DR: Qwen3's benchmark gains are real, but "Qwen3" isn't a single model — the hybrid-thinking default, Instruct, and Thinking variants perform differently enough that comparing the wrong one against Qwen2.5 will mislead you. Full eval script above; test on your own prompts before deciding.

Here's the tool I referenced in this post: www.fastrouteai.com

How to Use DeepSeek's API: A Working Python Example From My First Side Project

Hamimelon2026 — Tue, 25 Aug 2026 06:41:21 +0000

The Tutorial I Wish I'd Had

I spent about forty minutes stuck on a 401 Unauthorized error the first time I tried to use DeepSeek's API. The fix took ten seconds once I found it. The forty minutes was me not knowing where to look.

I was building a small side project — a script that summarizes long PDFs into short study notes, mostly for going through research papers faster. DeepSeek came up as a cheap, capable option, and I figured I'd document the actual steps I took, errors included, since most "getting started" guides skip the part where things don't work the first time.

Step 1: Get an API Key

Sign up at DeepSeek's platform and generate an API key from your account dashboard. Keep it somewhere you won't accidentally commit to git — I use a .env file and python-dotenv, which is where my forty minutes actually went. (I'd put the key in the wrong environment variable name. Double-check this before you assume the API itself is broken.)

pip install openai python-dotenv

Yes, openai — DeepSeek's API is OpenAI-compatible, so you use the same SDK, just pointed at a different base URL.

Step 2: Your First Request

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()

client = OpenAI(
    api_key=os.getenv("DEEPSEEK_API_KEY"),
    base_url="https://api.deepseek.com"
)

response = client.chat.completions.create(
    model="deepseek-chat",
    messages=[
        {"role": "system", "content": "You are a helpful assistant that summarizes text concisely."},
        {"role": "user", "content": "Summarize this in three bullet points: [your text here]"}
    ],
    temperature=0.3
)

print(response.choices[0].message.content)

That's the whole thing for a basic call. If you're coming from OpenAI's own API, this will look almost identical — that's intentional, and it's why the SDK doesn't need to change.

Step 3: Handling Longer Documents

For my actual use case (summarizing PDFs), the text usually blows past a reasonable single-prompt length. I ended up chunking the document and summarizing each piece, then doing a final pass to combine them:

def summarize_chunk(text, client):
    response = client.chat.completions.create(
        model="deepseek-chat",
        messages=[
            {"role": "system", "content": "Summarize the following text in 2-3 sentences."},
            {"role": "user", "content": text}
        ],
        temperature=0.3
    )
    return response.choices[0].message.content

def summarize_document(chunks, client):
    partial_summaries = [summarize_chunk(chunk, client) for chunk in chunks]
    combined = "\n".join(partial_summaries)

    final_response = client.chat.completions.create(
        model="deepseek-chat",
        messages=[
            {"role": "system", "content": "Combine these partial summaries into one coherent summary."},
            {"role": "user", "content": combined}
        ],
        temperature=0.3
    )
    return final_response.choices[0].message.content

This is intentionally simple — no retry logic, no async, nothing fancy. It's the version that got my prototype working, not the version I'd ship to production.

What I'd Add Before Using This for Real
Retries with backoff for rate limit errors (429s happen, especially at volume)
Token counting before sending, so you're not surprised by a request that's too long
Error handling around the response object, since response.choices[0].message.content will throw if the request failed silently
The Part After "It Works"

Once this was running, the next question I had was whether a different model would handle long documents better or cost less for my volume. Answering that with DeepSeek's SDK directly would've meant repeating steps 1–3 above for each new provider — new base URL, new auth pattern, occasionally a slightly different response shape to handle.

I ended up routing requests through RouteAI instead, which uses the same OpenAI-compatible format shown above — same client.chat.completions.create() call, just a different base_url and model name. That let me test the same chunking logic against a couple of other models without rewriting the functions above. Worth being clear about what this does and doesn't solve: it saved me integration time, not token cost — pricing still depends entirely on which model you pick.

If You're Just Getting Started

Get the basic request working first, with your own API key, before adding anything else. Everything above builds on that one call. The chunking function is specific to my use case (long documents) — if you're doing something simpler, like a chatbot or single-turn queries, you may not need it at all.

TL;DR: DeepSeek's API is OpenAI-compatible, so getting started is mostly pip install openai, swap the base_url, and use your DeepSeek key. Full working example above, including a basic document-chunking pattern for long text.

Linking the tool mentioned above: www.fastrouteai.com

Stop Comparing LLM APIs by Price Per Token. Compare Them by Price Per Solved Task

Hamimelon2026 — Thu, 20 Aug 2026 09:07:02 +0000

Model A costs half as much per token as Model B. Model A also needs almost twice the tokens to actually solve the task correctly. Guess which one was cheaper by the time the invoice came.

This is the gap nobody accounts for when they search "cheapest LLM API" and sort a comparison table by the input/output price columns. The table isn't wrong. It's just answering a narrower question than the one people think they're asking.

The Metric Everyone Optimizes, and What It Leaves Out

Price-per-token is the number every pricing page leads with, and it's the number every comparison article sorts by. It's also, on its own, an incomplete predictor of what a task will actually cost — because price-per-token tells you nothing about how many tokens a model needs to get you a usable answer.

Two models can have identical per-token pricing and produce wildly different real-world costs, because token count isn't fixed by the task — it's a function of the model's behavior on that task. A model that's verbose, that pads its answers with unnecessary preamble and repetition, that needs a longer, more carefully engineered prompt to produce reliable output, or that has a higher rate of producing an answer you have to retry — all of that shows up as more tokens consumed per successfully completed task, regardless of how cheap each individual token is.

"Cheapest LLM API" answered by price-per-token is answering "which model has the lowest sticker price." What most people actually want answered is closer to "which model gets me a correct, usable result for the least money" — and those are only the same question if every model needs exactly the same number of tokens to get there, which is almost never true.

A Concrete Way to See the Gap

Picture two models, both priced at output tokens, one at $1 per million and one at $2 per million — the first one looks twice as cheap on the pricing page. Now run the same task through both: a structured data extraction job, say, pulling five fields out of unstructured text.

The cheaper-per-token model produces a verbose response — some preamble explaining what it's about to do, the extracted data, then a summary restating what it just extracted — averaging 400 output tokens per task. The pricier-per-token model returns exactly the structured output requested, no preamble, no restatement — averaging 90 output tokens per task.

Run the actual math: the "cheap" model costs $0.0004 per task. The "expensive" model costs $0.00018 per task. The model with double the sticker price ends up costing under half as much per completed task, purely because of how differently the two models behave on the same prompt.

This isn't a hypothetical edge case — verbosity, formatting habits, and instruction-following precision vary meaningfully across models, and none of that variance shows up in a price-per-token comparison table.

Why This Gets Worse With Retries and Prompt Length

Two more factors compound the gap, and both get ignored by a pure per-token comparison:

Retry rate. If a model produces an unusable or malformed response some percentage of the time — invalid JSON, a hallucinated field, an instruction it didn't follow — every retry is additional tokens spent on a task that hasn't actually been solved yet. A model with a meaningfully higher retry rate can lose a per-token price advantage entirely once you account for the wasted attempts.

Prompt engineering overhead. Some models need a longer, more explicit system prompt to reliably follow formatting instructions — extra input tokens paid on every single call, forever, to compensate for something a different model does correctly with a shorter prompt. This cost is easy to miss because it gets baked into your prompt template once and then becomes invisible, but it's still being paid on every request.

Both of these push the real cost further from the number on the pricing page, and both push in the same direction: toward the per-token price being a weaker predictor of actual cost than it appears.

How to Actually Measure the Metric That Matters

The fix isn't complicated, it's just a different thing to measure: price per successfully completed task, calculated as (input tokens + output tokens + retry overhead) × price per token, averaged across a representative sample of your actual use case — not a generic benchmark, your specific task.

A rough way to approximate this without heavy tooling:

Pick 15-20 representative examples of your actual task.
Run them against each model candidate, recording total tokens consumed (including any retries needed to get a usable result).
Multiply by each model's per-token pricing.
Compare the resulting per-task cost, not the per-token rate.

This takes an afternoon, not a research project, and it's the only version of "cheapest" that actually predicts your bill. It also tends to produce genuinely surprising results — the model with the higher sticker price wins this comparison more often than pure price-per-token rankings would suggest, precisely because verbosity and retry rate are real, common differentiators that the pricing page doesn't capture.

Where This Connects to a Bigger Pattern

This is also a decent argument for why testing multiple models against your actual task matters more than picking one off a price comparison chart and committing — since the real cost differences here only show up empirically, per task, not on a spec sheet. Standardized, OpenAI-compatible access across multiple providers makes that kind of task-level comparison meaningfully easier to run, since testing model B doesn't require a separate integration from testing model A. RouteAI is one example of infrastructure built around that specific need — one interface across DeepSeek, Qwen, Kimi, GLM, and other models, which makes running the kind of per-task cost comparison described above a matter of switching a model parameter rather than rebuilding an integration for each candidate.

The Actual Takeaway

"Cheapest LLM API" measured by price-per-token is a real, useful number — it's just not the number that determines your actual bill. The number that determines your actual bill is price-per-completed-task, and the only way to know it is to actually run your specific task against your specific candidates and count.

The pricing page will tell you the price. It won't tell you the cost. Those turn out to be different things more often than the sorted comparison table would have you believe.

TL;DR: Comparing LLM APIs by price-per-token misses how many tokens each model actually needs to complete a task — verbosity, retry rates, and prompt-engineering overhead can make a "cheaper" model cost more per solved task than a pricier one. The metric that actually predicts your bill is price-per-completed-task: run 15-20 representative examples through your candidate models, count total tokens including retries, and multiply by each model's rate. Standardized access across multiple providers (like a gateway such as RouteAI) makes running this comparison easier, since testing a new model becomes a config change instead of a new integration.

Here's the tool I referenced in this post: www.fastrouteai.com

Why Every 'Free AI API' Has an Expiration Date, and Why That's Not Going to Change

Hamimelon2026 — Wed, 19 Aug 2026 06:54:22 +0000

Every free AI API you've ever used has an expiration date. Not a bug, not bad luck — math.

If you've been burned by a free tier that quietly got smaller, or a signup bonus that used to be automatic and now isn't, this isn't a story about that provider being shady. It's a story about unit economics, and once you see the numbers, the pattern stops feeling random.

Start With What Actually Costs Money

Running inference on a large language model costs real compute — GPU time, memory bandwidth, power, all of it metered per token processed. This isn't a fixed cost that amortizes to zero at scale the way, say, hosting a static website does. Every single token a free-tier user generates costs the provider a real, non-trivial amount of money, right now, today.

Compare that to a typical freemium SaaS product. A free user on a project management tool costs the company almost nothing marginal — maybe a sliver of database storage, some bandwidth. The company can subsidize millions of free users basically forever because the marginal cost per free user rounds to zero.

Free AI API access doesn't have that property. The marginal cost per free user is not zero — it's a real number that shows up on the provider's cloud bill every month, denominated in the same GPU-hours the paying customers are using.

This is the entire reason "free AI API" behaves differently from "free software," and it's worth sitting with, because it explains basically every free-tier policy decision you've ever been annoyed by.

The Three Ways Providers Make the Math Work

Given that free usage has a real, ongoing cost, providers only have a few structural options to make offering it sustainable:

Option 1: Cap it hard enough that the cost stays trivial. Low requests per minute, low daily token limits — small enough that even a large number of free users adds up to a rounding error on the infrastructure bill. This is the most common approach, and it's why free tiers tend to feel "just barely enough for testing."

Option 2: Treat it as a fixed acquisition cost, not an ongoing one. A one-time signup credit — spend it, it's gone — functions like a customer acquisition cost with a hard ceiling. The provider knows exactly what a free user costs them (the credit amount) and can model it like any other CAC number, unlike an open-ended free tier where cost scales with usage.

Option 3: Subsidize it deliberately, for a defined period, for a specific strategic reason. New model launches sometimes come with unusually generous free access — not because the unit economics suddenly changed, but because the provider is paying for market share and usage data with compute cost, the same way any venture-funded product buys growth. This kind of generosity is explicitly temporary by design, and providers are usually upfront that it won't last, even when users don't read that part.

Every free AI API tier you've ever encountered is doing one of these three things. None of them are "we've decided to give this away forever," because that option doesn't exist in this cost structure the way it does for software with near-zero marginal cost.

Why "It Got Worse" Isn't a Betrayal

This reframes something that otherwise feels like bait-and-switch. When a free tier's limits tighten, or a promotional period ends, or a "free" model gets sunset in favor of a paid one, it's tempting to read that as the provider reneging on a promise. Usually it's closer to the provider's unit economics catching up with a subsidy that was never meant to be permanent.

That's not an argument that providers are blameless in how they communicate this — plenty of free-tier terms are genuinely under-explained, and "free" gets marketed with more confidence than the fine print supports. But understanding why the tightening happens changes how you should plan around it. If you build assuming a generous free tier is a stable foundation, you're building on a subsidy with a countdown timer you can't see. If you build assuming it's temporary by nature, you plan differently from day one.

What This Means If You're Building on Free Tiers

A few practical implications follow pretty directly from the unit economics:

The more generous a free tier looks relative to the model's actual capability, the more temporary it probably is. Extremely generous free access to a genuinely strong model is very likely Option 3 — a strategic, time-limited subsidy — not a stable long-term offering.

Hard rate limits (Option 1) are the most durable kind of "free." They're durable precisely because they're small enough that the provider's cost stays trivial indefinitely. If you need something to actually rely on for a low-volume, ongoing use case, a modest permanent free tier is a safer bet than a generous but clearly promotional one.

Fixed signup credits (Option 2) should be treated as exactly what they are — a one-time trial, not a resource to budget around long-term. Plan your evaluation to fit inside that credit, not your production usage.

None of this is a reason to avoid free tiers. It's a reason to build your evaluation and prototyping process assuming the specific terms you're getting today are not guaranteed to be the terms you get in three months — regardless of which provider it is.

Where This Connects to a Bigger Pattern

This same unit-economics logic is part of why the market has started producing standardized, multi-provider access layers — services that sit in front of several model providers behind one consistent interface, so that when any single provider's free tier tightens (as the economics above make functionally inevitable, eventually, for all of them), your actual application code isn't the thing that has to change. RouteAI is one example of this category — not because it changes the underlying unit economics of running inference, nobody can, but because standardizing the interface layer means a change in any one provider's free-tier terms is a config update rather than a rewrite. Worth knowing about as a category, whether or not RouteAI specifically is the right fit for what you're building.

The Actual Takeaway

"Free AI API" isn't a stable category, and it was never going to be, because free inference has a real marginal cost that free software mostly doesn't. Every free tier you'll ever use is either capped small enough to be permanently sustainable, structured as a one-time credit, or deliberately subsidized for a limited window — and knowing which one you're looking at tells you almost everything about how much to plan around it.

The expiration date isn't always written down. But it's always there, and it's math, not malice.

TL;DR: Free AI API access isn't free to provide — inference has a real, ongoing compute cost, unlike near-zero-marginal-cost freemium software. That means every free tier is one of three things: hard rate-limited to stay permanently cheap, a fixed one-time signup credit, or a deliberate, temporary subsidy tied to a launch. None of these are stable long-term foundations except the hard-capped kind, and even that only stays free because it stays small. Plan your evaluation and prototyping around which category you're actually in, and consider a standardized access layer (like a multi-provider gateway) so that when any single provider's terms change, your code doesn't have to.

DeepSeek API Cost Jumped Overnight — The Python Script I Used to Measure It

Hamimelon2026 — Tue, 18 Aug 2026 03:24:48 +0000

My billing alert fired at 9:14 AM. Not from a bug. From a price change I hadn't seen coming.

I checked the dashboard expecting a rounding error. It wasn't one — my cost-per-request for one specific token tier had gone up by a factor I had to double-check twice, because the first read looked like a mistake.

Here's what actually happened, and the script I wrote to quantify it before I did anything else.

The trigger

On August 17 (Beijing time), DeepSeek's new peak/off-peak API pricing went live — announced August 13, effective at 00:00 that day. Peak hours are 9:00–12:00 and 14:00–18:00 Beijing time; off-peak is everything else, priced at exactly half the peak rate across every tier.

That "half off" framing is accurate, but it's a discount off a much higher baseline than what DeepSeek was charging when V4 Pro launched just five days earlier. Multiple reports pegged the increase on cache-hit input tokens at up to 1100% during peak hours compared to the original launch price — I haven't independently confirmed the exact launch-day number against an archived source, so treat that figure as reported rather than verified, but it's the number that's been driving the conversation.

What I could verify directly, from DeepSeek's own current pricing page:

Peak is exactly 2x off-peak on every row, which at least makes the math predictable once you know which hour you're calling from.

Script 1: measuring the actual damage

Before deciding whether this mattered for my project, I wanted a number, not a feeling. This is a stripped-down version of what I ran — plug in your own daily token volumes and the reported launch price to see your delta:

# cost_delta.py
# Rough estimate — the "launch_price" figures below are from
# third-party reporting on DeepSeek's Aug 12 introductory pricing,
# not an official archived source. Treat as directional, not exact.

def estimate_daily_cost(cache_hit_tokens, cache_miss_tokens, output_tokens, prices):
    """prices: dict with keys cache_hit, cache_miss, output ($ per 1M tokens)"""
    cost = (
        (cache_hit_tokens / 1_000_000) * prices["cache_hit"]
        + (cache_miss_tokens / 1_000_000) * prices["cache_miss"]
        + (output_tokens / 1_000_000) * prices["output"]
    )
    return round(cost, 4)

# Your actual daily usage — replace with real numbers from your logs
daily_usage = {
    "cache_hit_tokens": 4_000_000,
    "cache_miss_tokens": 500_000,
    "output_tokens": 300_000,
}

launch_price = {"cache_hit": 0.0035, "cache_miss": 0.42, "output": 0.83}  # reported, unverified
off_peak_price = {"cache_hit": 0.022, "cache_miss": 0.66, "output": 1.98}  # official, current
peak_price = {"cache_hit": 0.044, "cache_miss": 1.32, "output": 3.96}      # official, current

launch_cost = estimate_daily_cost(**daily_usage, prices=launch_price)
off_peak_cost = estimate_daily_cost(**daily_usage, prices=off_peak_price)
peak_cost = estimate_daily_cost(**daily_usage, prices=peak_price)

print(f"Estimated daily cost — launch pricing:  ${launch_cost}")
print(f"Estimated daily cost — off-peak (now):  ${off_peak_cost}")
print(f"Estimated daily cost — peak (now):      ${peak_cost}")
print(f"Worst-case increase: {round((peak_cost / launch_cost - 1) * 100, 1)}%")

Running this with my own numbers, the worst-case daily increase — if all my traffic happened to land in peak hours — came out well over 300%. That's not the sensational 1100% headline (that figure applies specifically to the cache-hit tier in isolation, not my blended daily cost), but it was more than enough to make me look at scheduling.

Script 2: making the provider swap boring

The second thing I fixed wasn't DeepSeek-specific. My integration had the base URL hardcoded:

client = OpenAI(base_url="https://api.deepseek.com/v1", api_key=DEEPSEEK_KEY)

I moved it to config, so a pricing change anywhere doesn't require a redeploy:

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["AI_GATEWAY_BASE_URL"],
    api_key=os.environ["AI_GATEWAY_API_KEY"],
)

For the gateway itself, I've been testing RouteAI — an OpenAI-compatible proxy sitting in front of DeepSeek, Qwen, Kimi, GLM, and a few others. I'm not claiming it's the cheapest option available; I haven't benchmarked every gateway, and pricing in this space shifts often enough that any "cheapest" claim would likely be stale within weeks. What it did do: turn "a provider changed pricing with five days' notice" from an emergency into a config edit. Worth testing against your own traffic pattern rather than taking that at face value.

What I'd actually recommend

Run the numbers on your own usage before reacting to headline percentages — the tier that moved the most (cache-hit input) may or may not be the tier your workload actually depends on. Then decide separately whether scheduling, provider abstraction, or just eating the cost makes sense for you.

TL;DR: DeepSeek's peak/off-peak pricing went live Aug 17 — peak is exactly 2x off-peak on every tier, and even off-peak is higher than the reported Aug 12 launch price. Widely cited reports put the cache-hit tier's peak increase at up to 1100% vs launch, though that specific figure isn't independently source-verified here. Run your own blended cost estimate (script above) before reacting to the headline number. Abstracting base_url/api_key into config — optionally via an OpenAI-compatible gateway like RouteAI — turns future pricing shocks into a config edit instead of a rewrite.

Here's the tool I referenced in this post: www.fastrouteai.com

The 'DeepSeek Agent' Label Is Doing a Lot of Work It Shouldn't Have To

Hamimelon2026 — Thu, 13 Aug 2026 07:18:50 +0000

Search "DeepSeek agent" and you'll find tutorials, starter repos, and no shortage of takes on how capable DeepSeek is "as an agent." I think that framing quietly misattributes where agent behavior actually comes from, and it's worth untangling, because it affects how people evaluate and build these systems.

Here's the claim I want to push back on: there's no such thing as a "DeepSeek agent" in the sense of the model itself being agentic. What most people mean by that phrase is a system — tool definitions, an orchestration loop, some form of memory or state — with DeepSeek (or any other model) plugged in as the reasoning component inside it. The model matters, but it's one part of the system, not the system.

What "agent" actually depends on

Strip away the branding and an "agent" built on top of a chat-completion-style API is usually some version of this loop:

Send the user's request plus a set of available tools (functions the model can choose to call) to the model.
The model responds with either a direct answer or a request to call a specific tool with specific arguments.
Your code executes that tool call and feeds the result back to the model.
Repeat until the model produces a final answer.

None of the tool definitions, the loop control, the error handling for failed tool calls, or the memory/context management across turns comes from the underlying LLM. Those are things you (or whatever framework you're using) build. The model's job in this loop is narrower than "being an agent" — it's deciding, at each step, whether to answer directly or request a tool call, and doing that reasonably well.

That's still a meaningful capability, and it does vary by model. But it's a specific, testable skill — not a general property that some models "have" and others don't.

Where model choice actually shows up

If the orchestration is doing most of the structural work, where does the choice of DeepSeek vs. another model actually matter? In my own testing, mostly in a few narrower places:

Tool selection accuracy — how often the model picks the right tool (or correctly decides not to call one) given ambiguous or overlapping tool descriptions.
Argument formatting consistency — whether the model reliably produces well-formed arguments for a tool call, especially for tools with several parameters.
Multi-step reasoning stability — how well the model tracks state and doesn't lose the thread across several tool-call round trips in a single task.

I've seen DeepSeek perform competitively on these dimensions in my own informal testing, and I've also seen it (like most models) occasionally call a tool it didn't need to, or misformat an argument on a longer chain. I'm not going to claim it's "the best" or "most reliable" agent backbone — I haven't run anything close to a rigorous benchmark, and results seem to depend heavily on how well the tool descriptions are written, which is itself something the developer controls, not the model.

That last point is worth sitting with: a lot of "this model is bad at tool calling" reports I've seen trace back to vague or overlapping tool descriptions, not an inherent model limitation. Before concluding a model can't "do agents" well, it's worth checking whether the tools it's failing to pick correctly were actually described clearly enough to disambiguate.

Why the framing matters beyond semantics

Calling something "a DeepSeek agent" (or a Qwen agent, or a GPT agent) makes it easy to compare models on the wrong axis — as if agent capability were a single score a model either has or doesn't. In practice, most of what makes an agent useful or fragile — the tool definitions, retry logic, how errors get surfaced back to the model, how much context gets carried between steps — is architecture decisions that stay constant no matter which model you swap in underneath.

This has a practical implication: if you're building on top of multiple models to compare "which one is a better agent," you get a much cleaner comparison if you hold the orchestration layer constant and only swap the model. I've done this using a single OpenAI-compatible gateway (RouteAI, in my case) to point the same tool-calling harness at DeepSeek and a couple of other models without rewriting the loop each time — which made it obvious how much of what I'd have otherwise attributed to "the model" was actually coming from my tool descriptions and loop logic.

The takeaway

"DeepSeek agent" is a convenient shorthand, but it's worth being precise about what it's shorthand for: a tool-calling and orchestration system that happens to use DeepSeek for the reasoning steps. The model's tool-selection and argument-formatting behavior is genuinely worth evaluating — just not as a stand-in for the whole system's quality, and not without first checking whether your own tool descriptions are doing the model any favors.

Curious whether others building agent systems across multiple models have found the same thing — that swapping the model changes less than expected once the orchestration layer is solid?

TL;DR: "DeepSeek agent" (like "GPT agent" or "Qwen agent") misleadingly attributes agent behavior to the model alone. Most of what makes an agent work — tool definitions, orchestration loop, memory — is architecture you build; the model's actual contribution is narrower: tool-selection accuracy and argument formatting. Evaluate that specific skill, not "agent capability" as a single score.

Here's the tool I referenced in this post: www.fastrouteai.com

OpenAI Compatible' Is Becoming an API Standard, Not a Marketing Label

Hamimelon2026 — Tue, 11 Aug 2026 06:36:42 +0000

A couple of years ago, "OpenAI compatible" was mostly a marketing phrase — a way for a smaller provider to say "we're kind of like OpenAI, but cheaper." Today it means something more specific and more useful: a growing number of providers, both open-source and commercial, expose the exact same request/response shape that OpenAI's Chat Completions API uses. Same JSON fields, same streaming format, same SDK compatibility.

I think that's worth paying attention to for a reason that has nothing to do with any single vendor: it's quietly turning into a de facto API standard for the LLM space, the same way SQL became a de facto standard for relational databases even though every database has its own extensions and quirks underneath.

Why a de facto standard matters more than a "better" API

Nobody designed the OpenAI Chat Completions format as an industry standard on purpose. It became one for a boring but powerful reason: it got adopted first, tooling was built around it (LangChain, LlamaIndex, countless internal SDKs), and switching costs for anyone not supporting it kept rising. Once enough of the ecosystem assumes a given request shape, matching it becomes the path of least resistance for new entrants — not because it's the best possible design, but because compatibility is worth more than a marginally cleaner API.

This is a familiar pattern outside AI too. SQL isn't universally loved as a language design, but "just support SQL" became the rational choice for new databases because the alternative was asking every user to rewrite their queries. REST won over more "correct" API styles for similar reasons. Standards compound: the more things support them, the more costly it becomes not to.

What this actually changes for developers

If you build on top of LLM APIs, the practical effect of this convergence is portability. In principle, code written against the OpenAI SDK format should be able to point at a different provider's endpoint by changing a base URL and an API key, without rewriting request-building or response-parsing logic. In practice, this holds reasonably well for the core chat completion flow, and less well for provider-specific features (function calling variants, extended context handling, multimodal inputs) that haven't fully converged yet.

That partial convergence is, I think, the more interesting story than "everyone supports OpenAI's format now." A handful of things I've noticed working across providers that expose an OpenAI-compatible endpoint:

Basic chat completion and streaming — generally solid
Standard parameter names (temperature, max_tokens, etc.) — generally solid
Function/tool calling — inconsistent enough that I still test it per-provider before relying on it
Vendor-specific extras (like some providers' native reasoning-trace fields) — not portable, and probably shouldn't be treated as if they will be

This matters for how teams should think about vendor lock-in. A lot of the "avoid lock-in" advice in this space treats OpenAI-compatible endpoints as a full guarantee of portability. A more accurate framing is: the compatibility layer meaningfully lowers switching costs for the common path, but doesn't eliminate them for anything provider-specific you've come to depend on.

Where this shows up in practice

A number of gateway-style services have emerged specifically to sit on top of this convergence — offering one OpenAI-compatible endpoint that routes to multiple underlying models. RouteAI is one example I've used for this; the useful part isn't the gateway itself so much as what it implies: enough providers now agree on a shared request format that building a router on top of it is even feasible. That wouldn't have made sense a couple of years ago when every provider's API looked meaningfully different.

I'd expect this convergence to keep going for the common cases (chat, streaming, basic tool calls) while staying genuinely fragmented for anything closer to the frontier of what a given model can do — long-context handling, native multimodal input, or model-specific reasoning features. Standards tend to solidify around the stable, well-understood parts of a technology and stay contested around the parts that are still evolving.

The takeaway

"OpenAI compatible" is worth treating as an emerging API convention rather than a vendor claim to be skeptical of. It doesn't make every LLM interchangeable, and it doesn't remove the need to actually test a model for your use case — but for the core request/response shape, it's increasingly safe to build against it, the way you'd build against SQL knowing full well that vendor-specific extensions exist underneath.

Curious whether others building on multiple LLM providers have hit the same split — solid on the basics, fragmented on the advanced features — or whether your experience has been different.

TL;DR: "OpenAI compatible" has evolved from a marketing phrase into a de facto API convention for LLMs, similar to how SQL became a standard for databases. Basic chat/streaming is portable across providers; advanced features (tool calling variants, provider-specific extras) still aren't — so treat compatibility as reducing switching costs, not eliminating them.

Here's the tool I referenced in this post: www.fastrouteai.com