DEV Community: Felix

I Built a Tool-Calling Agent on Qwen. Here's the Full Working Loop.

Felix — Wed, 26 Aug 2026 07:12:39 +0000

Tool Calling Is the Easy Part. The Loop Is Where It Gets Real.

My first "agent" wasn't actually an agent. It was one API call that returned a tool call, which I executed, and then I just... stopped, because I hadn't written anything to feed the result back in. It looked like it worked. It didn't — it just did one step and quit.

I was building a small assistant that could search a local notes database and check a to-do list, using Qwen's API. Here's the actual working setup, including the part I got wrong the first time.

Defining Tools

Qwen's function calling uses the same OpenAI-compatible schema most major providers use now — define your functions, pass them with your messages, and the model decides whether to respond with text or a structured tool call.

tools = [
    {
        "type": "function",
        "function": {
            "name": "search_notes",
            "description": "Search the user's saved notes for a keyword and return matching entries. Use this when the user asks to find or recall something they wrote down.",
            "parameters": {
                "type": "object",
                "properties": {
                    "keyword": {"type": "string", "description": "The term to search for"}
                },
                "required": ["keyword"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "check_todo_list",
            "description": "Return the user's current to-do list. Use this when the user asks what they need to do or what's pending.",
            "parameters": {"type": "object", "properties": {}}
        }
    }
]

The description fields do more work than they look like they should. My first version just said "Searches notes" and "Checks todos" — the model started picking the wrong tool for ambiguous queries because there wasn't enough signal to differentiate intent. Being explicit about when to use each tool, not just what it does, fixed most of the misfires once I had more than two tools competing.

The Part I Got Wrong: No Loop

Here's the broken version — the one that looks like it works because it doesn't error out:

response = client.chat.completions.create(
    model="qwen-plus",
    messages=[{"role": "user", "content": "What's on my to-do list, and do I have notes about Berlin?"}],
    tools=tools,
)

message = response.choices[0].message
if message.tool_calls:
    for call in message.tool_calls:
        result = execute_tool(call.function.name, call.function.arguments)
        print(result)  # dead end — the model never sees this

This runs the tool and prints the result. It never tells the model what happened, so if the query needs more than one tool call, or needs the model to actually respond based on the tool's output, it just stops. For a query needing two tools ("to-do list" and "Berlin notes"), this version might only execute one and call it done.

The Working Version: Actually Looping

messages = [{"role": "user", "content": user_input}]

while True:
    response = client.chat.completions.create(
        model="qwen-plus",
        messages=messages,
        tools=tools,
    )
    message = response.choices[0].message

    if message.tool_calls:
        messages.append(message)
        for call in message.tool_calls:
            result = execute_tool(call.function.name, call.function.arguments)
            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": str(result)
            })
    else:
        print(message.content)
        break

The difference: tool results get appended back into messages as a new turn, and the loop calls the API again with that updated context. The model sees what the tool returned and decides whether it has enough to answer or needs another tool call. This is the actual mechanism behind "agent" — not a mode you enable, just a loop that keeps going until the model stops asking for tools.

The Failure Mode Nobody Warns You About

Without a cap, this loop can run indefinitely if the model keeps deciding it needs "one more" tool call, especially if a tool returns something ambiguous or a call fails silently. Add a max-iteration limit:

MAX_ITERATIONS = 6
messages = [{"role": "user", "content": user_input}]

for _ in range(MAX_ITERATIONS):
    response = client.chat.completions.create(
        model="qwen-plus",
        messages=messages,
        tools=tools,
    )
    message = response.choices[0].message

    if message.tool_calls:
        messages.append(message)
        for call in message.tool_calls:
            result = execute_tool(call.function.name, call.function.arguments)
            messages.append({
                "role": "tool",
                "tool_call_id": call.id,
                "content": str(result)
            })
    else:
        print(message.content)
        break
else:
    print("Hit max iterations without a final answer.")

I found this the hard way — a test query looped six times on something that should have taken one, because a tool call returned an empty result and the model kept retrying instead of giving up gracefully.

Where This Actually Got Tested Further

Once the loop was solid, I wanted to check whether tool-selection accuracy held up if I swapped the underlying model — not because Qwen was underperforming, but because I was curious whether the loop logic above was actually model-agnostic or whether I'd built something that happened to work for one provider's quirks. I ran the same loop through RouteAI, an OpenAI-compatible gateway, against a couple of other models, and the loop code above didn't need to change — only the model argument did, since the request format stayed consistent. Worth being clear this wasn't the hard part of the project; the loop and tool descriptions were.

If You're Building Your First Agent
Get one tool working end-to-end, including the loop, before adding a second
Write descriptions that say when to use a tool, not just what it does
Always cap your iterations — an ungated loop will eventually run past what you expect

TL;DR: An "agent" is a loop, not a single API call — tool results have to get fed back into the conversation for the model to actually finish the task. Full working loop with an iteration cap included above; the tool descriptions matter more than most tutorials mention.

Worth exploring if this is relevant to your stack: www.fastrouteai.com

I Wrote a Script to Track My DeepSeek API Costs After the Weekend Pricing Change

Felix — Mon, 24 Aug 2026 09:41:45 +0000

The Bill That Made Me Curious

I don't normally check my API dashboard more than once a week. But last Sunday I did, out of boredom, and the number was lower than I expected for how much I'd been running.

Turns out DeepSeek quietly changed its peak/off-peak billing structure. According to their pricing docs, peak hours are now strictly Monday–Friday, 01:00–04:00 and 06:00–10:00 UTC. Everything else — including all of Saturday and Sunday — bills at off-peak rates, which run at roughly half of peak pricing for both input and output tokens.

I run most of my testing on weekends (day job during the week), so in theory this should matter a lot for me. But "in theory" isn't good enough when I'm the one paying the invoice. I wanted actual numbers from my actual usage, not a guess.

So I wrote a small script.

What the Script Does

Nothing fancy — it takes a log of my API calls (timestamp + input/output token counts) and calculates what I actually paid under the new peak/off-peak rules, split by weekday vs weekend.

import csv
from datetime import datetime, timezone

# Pricing per 1M tokens (cache miss), off-peak / peak
PRICING = {
    "deepseek-v4-flash": {
        "input_offpeak": 0.22, "input_peak": 0.44,
        "output_offpeak": 0.66, "output_peak": 1.32,
    },
    "deepseek-v4-pro": {
        "input_offpeak": 0.66, "input_peak": 1.32,
        "output_offpeak": 1.98, "output_peak": 3.96,
    },
}

def is_peak(dt_utc):
    # Peak: Mon-Fri, 01:00-04:00 and 06:00-10:00 UTC
    if dt_utc.weekday() >= 5:  # Sat=5, Sun=6
        return False
    hour = dt_utc.hour
    return (1 <= hour < 4) or (6 <= hour < 10)

def calc_cost(model, input_tokens, output_tokens, dt_utc):
    rates = PRICING[model]
    peak = is_peak(dt_utc)
    in_rate = rates["input_peak"] if peak else rates["input_offpeak"]
    out_rate = rates["output_peak"] if peak else rates["output_offpeak"]
    cost = (input_tokens / 1_000_000) * in_rate + (output_tokens / 1_000_000) * out_rate
    return cost, peak

def analyze_log(csv_path):
    weekday_cost, weekend_cost = 0.0, 0.0
    weekday_calls, weekend_calls = 0, 0

    with open(csv_path) as f:
        reader = csv.DictReader(f)
        for row in reader:
            dt = datetime.fromisoformat(row["timestamp"]).astimezone(timezone.utc)
            cost, peak = calc_cost(
                row["model"],
                int(row["input_tokens"]),
                int(row["output_tokens"]),
                dt,
            )
            if dt.weekday() >= 5:
                weekend_cost += cost
                weekend_calls += 1
            else:
                weekday_cost += cost
                weekday_calls += 1

    print(f"Weekday: {weekday_calls} calls, ${weekday_cost:.4f}")
    print(f"Weekend: {weekend_calls} calls, ${weekend_cost:.4f}")

if __name__ == "__main__":
    analyze_log("api_usage_log.csv")

Expected CSV format:

timestamp,model,input_tokens,output_tokens
2026-08-22T14:32:00+00:00,deepseek-v4-flash,1200,340
2026-08-23T09:15:00+00:00,deepseek-v4-flash,980,410

You'll need to export your own usage log — DeepSeek's dashboard lets you download call history, or you can log it yourself at request time if you're not already.

What I Actually Found

Running this against about six weeks of my own logs: my weekend calls were consistently cheaper per-token than my weekday calls even before this change (because some of my weekend hours already fell outside the old off-peak window by luck). After the update, the gap widened — my weekend cost-per-call dropped further since Saturday and Sunday are now unconditionally off-peak, no matter the hour.

For someone running a handful of batch jobs on weekends, that's a real, if modest, saving. Your mileage depends entirely on when you actually run your workload — if most of your usage happens on weekday afternoons, this change does nothing for you.

The Question I Couldn't Answer With This Script

Once I had this data, the obvious next question was: how would the same workload cost on a different model? The script above only works because I already know DeepSeek's pricing structure. Answering the same question for Qwen or GLM would mean writing a whole new pricing table and, more annoyingly, a whole new API integration to actually generate comparable token logs.

That's the part I didn't script around — I switched my project to call models through RouteAI, an OpenAI-compatible API gateway, mainly so I could point the same request format at different models without rebuilding my client code each time. It didn't change the cost-tracking logic above, but it meant I could actually go collect that data for other models instead of just wondering about it. Worth noting this only saved me integration time — the actual per-token pricing is still whatever each model provider sets.

If You Want to Run This Yourself
Export or log your own usage with timestamps in UTC (not local time — this bit me on my first run)
Add pricing tiers for whichever models you're using; I only included flash and pro here
If you're on a different provider, check whether they have time-based pricing at all before assuming this script applies

TL;DR: DeepSeek's new pricing makes weekends fully off-peak (previously only certain hours were). I wrote a Python script to calculate actual cost split between weekday/weekend usage from a call log — script included above, MIT-license it however you like.

Worth exploring if this is relevant to your stack: www.fastrouteai.com

'Cheap AI' Is a Moving Target — Here's the Framework for Evaluating It Correctly

Felix — Thu, 20 Aug 2026 08:13:11 +0000

$0.14 per million tokens looks cheap. $0.14 per million tokens, multiplied by a feature that goes viral next Tuesday, might not be.

Most "is this AI cheap" evaluations stop at the first number. That's the mistake I want to walk through, because the actual question was never "what does it cost right now" — it's "what shape does the cost take as usage grows," and those are very different questions that happen to share a price tag.

The Number Everyone Compares, and Why It's Incomplete

When people evaluate whether an AI API is "cheap," they almost always mean: what's the per-token or per-request price, compared to alternatives. That number is real and it matters — but it's a snapshot, not a trajectory. It tells you what one unit costs. It tells you nothing about what a thousand units cost, or a million, or what happens at the specific volume your actual product will hit if it works.

Cost only behaves the way that snapshot implies if it scales linearly forever — the 1,000th call costs exactly what the 1st call cost, with nothing changing in between. That's the assumption baked into "cheap AI" as most people evaluate it. It's also, in practice, rarely how these systems actually behave once you're operating at real scale.

Three Shapes Cost Actually Takes, Not Two

Cost as a function of usage tends to fall into a few recognizable shapes, and knowing which one you're dealing with matters more than the sticker price:

Linear. Cost per unit stays flat regardless of volume. The 1,000th call costs what the 1st call cost. This is what a simple per-token price implies, and it's genuinely how some providers price things — but it's worth verifying rather than assuming, because it's the exception more often than the default once you look closely.

Step function. Cost stays flat within a tier, then jumps at a threshold — a rate limit you exceed and get bumped to a more expensive plan, a concurrency cap that forces you onto premium infrastructure, a free-tier boundary you cross without noticing until the bill arrives. This is common with anything that has both a free and paid tier, and with time-based pricing structures where crossing from off-peak into peak hours effectively doubles your rate mid-workload.

Marginal-decreasing. Cost per unit actually drops as volume increases — committed-use discounts, volume pricing tiers, negotiated enterprise rates. Less common at small scale, but real once you're a big enough customer for a provider to want to keep you.

Almost nobody evaluating "is this cheap AI" is thinking in these terms. They're comparing a single number, implicitly assuming shape one, when the actual answer might be shape two — which means the "cheap" evaluation was accurate at low volume and actively misleading at the volume that matters.

Where This Actually Goes Wrong

Here's the concrete failure mode: a team evaluates a model at prototype-scale usage, sees a low per-token cost, concludes "this is cheap," and builds a cost model for the product around that number. Then the product works. Usage climbs. And somewhere along the way, they cross a threshold they didn't know existed — a concurrency limit that forces an upgrade, a rate structure where their peak-hours traffic pattern (because of course their users are active during business hours) now costs meaningfully more than the number they budgeted around.

None of this is deceptive pricing. It's usually clearly documented — but documented in a pricing page that gets read once, at evaluation time, at a volume that doesn't reflect where the product ends up. The gap between "cheap at the volume I tested" and "cheap at the volume I'll actually run" is exactly the gap this framework is trying to make visible before it becomes a surprise on an invoice.

This is also worth connecting to a specific, current example: several major model providers have recently introduced peak/off-peak pricing structures, where identical usage costs meaningfully more during business hours than at night. That's a step function hiding inside what looks, on the pricing page, like a single number — and it's a good illustration of why "what's the price" and "what's the cost curve" are genuinely different questions right now, not just a hypothetical concern.

How to Actually Evaluate "Cheap" Correctly

A few concrete things worth doing differently, given this framework:

Ask about the shape, not just the number, when you evaluate a provider. Is this price linear indefinitely? Is there a rate limit or concurrency threshold where the effective cost jumps? Does pricing vary by time of day or usage tier? These are usually documented, just rarely asked about upfront.

Model your cost at the volume you expect to hit if things go well, not the volume you're at during evaluation. A price that's cheap at prototype scale and a price that's cheap at the scale you're actually building for are different claims, and testing at the wrong scale is how teams get surprised.

Treat unpredictability as its own cost, separate from the price itself. A cost structure that varies by time of day or usage threshold requires monitoring and budgeting work that a flat, predictable rate doesn't — and that overhead is real, even when the headline price is competitive.

Standardized, flat-rate access reduces the step-function risk specifically. This is part of why unified access layers — services that sit in front of multiple model providers with flat, consistent pricing rather than tiered or time-based structures — have become a relevant category to know about, separate from any single provider's rate card. RouteAI is one example of this, offering flat per-token pricing without a peak-hour surcharge structure across several model providers — worth mentioning as a case study in what a linear-shaped alternative looks like, not as a claim that it's the cheapest option available, which would require a comparison this piece isn't making.

The Actual Point

"Cheap AI" isn't a property of a number on a pricing page. It's a property of how that number behaves as your usage grows into whatever your product actually becomes. Two providers can have an identical headline price and radically different actual cost outcomes at scale, depending entirely on the shape underneath that number — and the shape is almost never what gets compared when people search "cheapest AI API" or "cheap AI" and pick based on the first number they see.

The question worth asking isn't "what does this cost me today." It's "what does this cost me at the volume where it actually matters" — and that's a question the pricing page alone usually won't answer for you.

TL;DR: Comparing "cheap AI" options by per-token price alone assumes cost scales linearly, but real pricing often follows a step function (rate limits, peak-hour surcharges, tier jumps) or occasionally marginal-decreasing pricing (volume discounts) instead. Evaluate providers by asking about the shape of cost as usage grows, not just the sticker price, and model your cost at the volume you'll actually hit — not the volume you tested at. Standardized, flat-rate access layers (like RouteAI, among other options) can reduce step-function risk specifically, though that's a different claim from being the cheapest option outright.

Here's the tool I referenced in this post: www.fastrouteai.com

'Free AI API' Isn't a Pricing Tier — It's a Funnel. Here's How It Actually Works

Felix — Tue, 18 Aug 2026 09:23:23 +0000

Zero dollars, for a while, under conditions you didn't read.

That's the actual terms of almost every "free AI API" you'll find right now. Not a lie, exactly — just incomplete, in a way that shapes a lot of decisions before anyone notices it's happening.

I want to walk through why "free" in this market behaves less like a pricing tier and more like a funnel, and what that means for how you should actually evaluate these offers.

The Pattern, Once You Notice It

Search "free AI API" and you'll land on some combination of: a signup-gated trial credit, a rate-limited forever-free tier, or an open-source model you self-host. All three get marketed as "free," and all three have a different actual cost structure hiding underneath the word.

Signup trial credits are the most common. You get a fixed dollar amount — enough to build a prototype, rarely enough to run anything in production for long. The moment it runs out, you're a paying customer with an existing integration, which is exactly the point. This isn't a criticism — it's a completely standard acquisition mechanic, the API equivalent of a free trial on any SaaS product. But it's worth naming it accurately: it's not a pricing tier, it's a funnel stage.

Rate-limited free tiers are the second version — genuinely free indefinitely, but capped hard enough (low requests per minute, low concurrency) that they work for testing and personal projects, not for anything with real traffic. These are sustainable for the provider precisely because they're too small to represent meaningful cost or meaningful competition to the paid tier.

Self-hosted open-source models are the third, and the only one that's "free" in a literal, ongoing sense — you're just trading API cost for infrastructure cost, DevOps time, and the work of keeping up with model updates yourself. Free in one column, expensive in another.

None of these are bad options. They're just not the same thing, and treating them as interchangeable answers to "is there a free AI API" is where people get tripped up.

Why "Free" Behaves Like a Funnel, Not a Tier

A pricing tier is a stable offering — you can build a long-term plan around it. A funnel stage is designed to convert you into the next stage. The tell is in how these free options are structured: they're calibrated to be just enough to prove the product works, and just limited enough that any real usage pushes you past the edge.

This isn't unique to AI APIs — it's the same shape as freemium software, free cloud tiers, free CDN bandwidth. What's different about AI APIs specifically is how fast the terms of the funnel can move underneath you, because the underlying compute cost is still genuinely high and providers are actively experimenting with pricing structure.

A concrete, recent example: several major model providers have started introducing peak/off-peak pricing — cheaper rates at night, higher rates during business hours — as a way to manage daytime infrastructure load. One provider's recent repricing moved certain rates by several hundred percent between off-peak and peak windows on the same day. That kind of structural change doesn't just affect paid usage — it reshapes what the free tier's limits are worth, and how predictable "free enough for now" actually is going forward.

If your evaluation of "is this free API good enough" was done once, at signup, and never revisited, you're evaluating a snapshot of a funnel stage that's actively designed to change.

What This Actually Means for How You Should Evaluate "Free"

Reframing free-tier evaluation around the funnel model changes what questions are worth asking:

Not: "Is this free?" Instead: "What stage of the funnel is this free tier designed to move me through, and what does the next stage cost?"

Not: "How much free credit do I get?" Instead: "What's the actual rate limit and concurrency cap, and does that match my real traffic pattern — not my testing pattern?"

Not: "Which provider has the best free tier?" Instead: "If I build against this provider's free tier and it changes, how much of my integration breaks?"

That last question is the one people skip most often, and it's the one that actually costs money later. A free tier that requires rewriting your error handling, retry logic, and request format the moment you outgrow it is a more expensive "free" than one that lets you scale into paid usage without touching your code.

The Honest Nuance Here

None of this means free tiers are a trap to avoid — they're genuinely useful, and almost every serious project should start on one. The point isn't "don't use free APIs." It's that treating "free" as a stable, comparable pricing category leads to worse decisions than treating it as what it actually is: a funnel stage with a shelf life.

This is also, worth being direct about, part of why unified API gateways exist as a category — services like RouteAI, which sit in front of multiple model providers behind one OpenAI-compatible interface, are a response to exactly this problem. Not because they eliminate the funnel dynamic (they don't — someone's still paying for compute), but because standardizing the interface means that whichever free tier you start on, and whatever it turns into later, the actual code you've built doesn't have to be the thing that breaks.

I'm not making a "just use this instead" argument here — that's a separate decision with its own tradeoffs. The point of this piece is narrower: the next time you're comparing "free AI API" options, the question that actually predicts how this goes for you isn't "how much free stuff do I get." It's "what happens to my integration when this stops being free."

That's the question the marketing page never answers, and it's the one worth answering yourself before you build on top of any of them.

TL;DR: "Free AI API" almost always means one of three things — trial credits (a funnel stage), a rate-limited forever-free tier (too small to matter), or self-hosted open source (free API cost, real infrastructure cost). None of these are stable pricing tiers — they're calibrated to convert or contain you. The question worth asking isn't "is this free," it's "what breaks in my integration when this stops being free," and building against a standardized interface (whether that's a single provider's stable API or a gateway like RouteAI in front of several) is what keeps that answer cheap instead of expensive.

Here's the tool I referenced in this post: www.fastrouteai.com

DeepSeek API Price Just Changed (Aug 17): Peak/Off-Peak Rates, With Real Numbers

Felix — Mon, 17 Aug 2026 10:46:25 +0000

If you're calling the DeepSeek API and your bill looked different this morning, you're not misreading it. As of 00:00 Beijing time today (August 17), DeepSeek's new peak/off-peak pricing is live.

Here's the short version, then the actual numbers, then what I changed in my own setup.

What changed

DeepSeek announced the update on August 13. The mechanism is simple: peak hours are 9:00–12:00 and 14:00–18:00 Beijing time, off-peak is everything else, and off-peak pricing is set at half the peak rate.

That sounds like a discount structure. It mostly is — relative to peak. But peak pricing itself is meaningfully higher than the price DeepSeek was charging just five days earlier, when the official V4 Pro model launched with aggressive introductory rates. So the honest way to describe this is: DeepSeek didn't get cheaper at off-peak hours, it got more expensive at peak hours, and off-peak is the smaller price to pay.

Here's the official pricing, straight from DeepSeek's own pricing page, for both V4 Flash and V4 Pro:

Two things jump out once you see the real numbers side by side. First, peak is exactly 2x off-peak across every single tier for both models — the "half price off-peak" framing is literally accurate, not marketing rounding. Second, and more relevant if you're building anything serious: v4-pro's concurrency limit is 500, five times lower than v4-flash's 2500. If you're getting throttled during peak hours, that limit — not just the price — might be the bigger constraint on your architecture.

Compared to V4 Pro's introductory pricing when it launched on August 12 (cache-hit input was priced at a small fraction of a cent), even off-peak cache-hit input today is roughly 6x higher, and peak is over 10x higher. Cache-miss and output pricing moved up too, just less dramatically. If your app leans on cache hits (repeated system prompts, shared context, RAG pipelines reusing chunks), that's the number to watch — not the headline "price change" itself.

Why "just move workloads off-peak" isn't a full fix

The advice you'll see everywhere — including from DeepSeek's own docs — is to schedule non-real-time jobs during off-peak hours. That's genuinely good advice for batch jobs, evals, and anything async.

It's less useful if your product serves live traffic. Peak hours are 9–12 and 14–18 Beijing time specifically because that's when Chinese business-hour demand spikes — but if your users are in other timezones, or your app is interactive, "wait until after 6 PM Beijing time" isn't an option you can hand to a user waiting on a response.

So for anything latency-sensitive, off-peak scheduling helps your batch/background work, but it doesn't solve the problem for your live traffic.

What I actually changed

Two things, neither of which is "leave DeepSeek."

I split workloads by urgency. Anything that can tolerate a delay (nightly summarization, embedding refreshes, eval runs) now explicitly targets off-peak windows. That's a scheduling change, not a code change — just a cron adjustment.
I stopped hardcoding a single provider's base URL. This is the part that actually mattered. My integration used to call DeepSeek directly:

client = OpenAI(
    base_url="https://api.deepseek.com/v1",
    api_key=DEEPSEEK_KEY,
)

Since DeepSeek's API is OpenAI-compatible, swapping providers is mostly a base_url and api_key change — if you've abstracted that config out instead of hardcoding it, which I hadn't. I fixed that first, independent of any specific provider:

client = OpenAI(
    base_url=os.environ["AI_GATEWAY_BASE_URL"],
    api_key=os.environ["AI_GATEWAY_API_KEY"],
)

Once that was config-driven, I could route different request types to whichever model/provider combination made sense, without a redeploy every time a pricing page changes.

For the gateway piece specifically, I've been testing RouteAI — it's an OpenAI-compatible proxy in front of DeepSeek, Qwen, Kimi, GLM, and a few others. I'm not going to claim it's the cheapest option out there — I haven't benchmarked every gateway or reseller, and pricing across this space changes often enough that any "cheapest" claim would probably be outdated by the time you read this. What it did solve for me: I'm no longer locked into rewriting integration code every time a single provider adjusts pricing with five days' notice. Worth evaluating yourself against your own traffic pattern rather than taking my word for it.

If you only do one thing today

Pull your last week of DeepSeek usage logs and check what percentage of your token volume falls inside the 9–12 / 14–18 Beijing-time windows. That number tells you whether this pricing change is a rounding error or a real line-item for you — before you make any architecture decisions based on it.

TL;DR: DeepSeek's peak/off-peak API pricing went live Aug 17 (Beijing time) — peak is 9–12 and 14–18, exactly 2x off-peak across every tier, but even off-peak is higher than the Aug 12 launch price, especially for cached input tokens (~6x off-peak, ~10x+ at peak). V4 Pro's concurrency limit (500) is also 5x lower than V4 Flash's (2500) — worth checking if you're hitting throttling. Batch/async work can shift to off-peak; live traffic mostly can't. Abstracting your base_url/api_key behind config (optionally through an OpenAI-compatible gateway like RouteAI) makes future pricing changes a config edit instead of a rewrite.

More on how I set this up: www.fastrouteai.com

Most AI Security Advice Ignores the Most Common Leak: What an API Security Gateway Actually Protects

Felix — Thu, 13 Aug 2026 06:51:44 +0000

Search "AI security" and most of what comes back is about the model layer: prompt injection, jailbreaks, output filtering, data leakage through model responses. All real concerns. But in the LLM applications I've actually looked at — side projects, hackathon demos, more than a few early-stage products — the security issue I run into far more often isn't any of those. It's a hardcoded or frontend-exposed API key.

This isn't a novel problem. It's the same class of mistake web developers have been making with third-party API keys for over a decade. But it keeps happening specifically with LLM keys, and I think that's worth examining, because the term "API security gateway" gets used loosely enough that people assume it covers this by default — and it often doesn't, depending on what you're actually deploying.

The unglamorous, extremely common failure mode

The pattern looks something like this: a developer is building fast, wires an LLM API key directly into client-side code (a mobile app, a browser-based tool, a quick prototype) to get something working, and either never gets around to moving it server-side, or doesn't realize that a key embedded in shipped frontend code is trivially extractable — via browser devtools, decompiling a mobile binary, or just reading a public GitHub repo the key got committed to.

None of this requires a sophisticated attacker. It's closer to "someone opened the network tab" than "someone reverse-engineered a model." And unlike a lot of AI-specific attack classes that require some domain expertise to pull off, this one has been a solved problem in web security for years: don't put credentials for a paid third-party service in code that ships to the client. The fact that it's LLM API keys now, instead of payment processor keys or cloud storage credentials, doesn't change the underlying issue — it's the same mistake with a newer name attached.

What "API security gateway" should actually mean here

I think the term gets muddled because "security" in the AI context has come to mean two fairly different things:

Model-layer security — protecting against malicious inputs/outputs: prompt injection, jailbreak attempts, content moderation, data exfiltration through crafted prompts.
Access-layer security — protecting the credentials and access paths to the model API itself: keeping keys off the client, enforcing per-user or per-app rate limits, revoking access without redeploying an app.

A lot of "AI security gateway" marketing leans on the first category because it's the more novel, more interesting-sounding problem. But for most applications I've seen actually get breached or leak something, it's the second category — plain old access-layer hygiene — that was missing. A gateway that proxies requests server-side and never exposes a raw key to the client is solving a boring, well-understood problem, not a cutting-edge AI security problem. That's not a knock on it — boring and well-understood is exactly what you want from access control.

It's worth being precise about what this kind of gateway does and doesn't cover. Routing requests through a server-side proxy so keys never reach the client addresses key exposure specifically. It doesn't, by itself, address prompt injection, doesn't validate what a user is asking the model to do, and doesn't replace rate limiting or auth on your own application layer if you need that too. Anyone evaluating a gateway for "security" should be clear on which of these two categories they're actually trying to solve, because a product can genuinely deliver on one while saying nothing about the other.

Why this keeps happening despite being a known problem

If this is such a well-understood failure mode, why does it keep showing up? A few reasons that seem to hold up across the examples I've seen:

Speed pressure during prototyping. It's genuinely faster to call an API directly from the frontend than to stand up a backend proxy, and "I'll move this server-side later" is an easy thing to defer indefinitely.
LLM APIs feel different from "real" credentials to some developers, even though functionally they're identical to any other paid API key — usable by anyone who has them, billed to the account that issued them.
Multi-provider setups multiply the surface area. If a team is calling two or three different LLM providers directly, that's two or three sets of keys to keep server-side, which raises the chance at least one gets handled carelessly under deadline pressure.

That last point is where a unified gateway layer — something like RouteAI, which I've used to route to multiple models through one server-side endpoint rather than juggling separate provider keys — becomes relevant less as a "security product" and more as a way of reducing the number of places a key-handling mistake can happen. Consolidating access reduces surface area; it doesn't eliminate the need to actually keep that access server-side.

The takeaway

Before reaching for an "AI security gateway" to solve prompt injection or content-safety concerns — which are real and worth addressing separately — it's worth checking the more basic question: is any API key for an LLM provider currently reachable from client-side code, a public repo, or a decompiled build? That's the leak that's actually common, actually cheap to fix, and doesn't require solving anything AI-specific — just applying the same access hygiene that's been standard for API keys in general for years.

Curious how teams here are currently structuring this — server-side proxy for every LLM call, or is direct client-to-provider still common in what you've seen?

TL;DR: Most "AI security" discussion focuses on model-layer risks like prompt injection, but the more common real-world issue is basic access-layer hygiene — API keys exposed in frontend code or committed to public repos. An API security gateway is most reliably useful for solving that specific problem (keeping keys server-side), not as a general AI security solution.

Here's the tool I referenced in this post: www.fastrouteai.com

Why Comparing Chinese LLM APIs by Price Alone Misses the Point

Felix — Tue, 11 Aug 2026 03:04:18 +0000

Every few weeks a new spreadsheet shows up in my feed comparing LLM API providers by dollars-per-million-tokens. It's a useful first filter, but it's also a trap. Price tables answer "which model is cheapest today," not "which model will actually get the job done reliably in production." Those are different questions, and conflating them is how teams end up migrating providers three times in six months.

I want to propose a slightly different way to evaluate LLM APIs — one that treats price as one input among several, not the headline metric. I'll use my own informal comparison of Qwen and DeepSeek as a working example, but the framework matters more than the specific numbers.

The problem with price-first comparisons

Sticker price ($/1M input tokens, $/1M output tokens) is easy to compare and easy to put in a table, which is exactly why it dominates the conversation. But it quietly ignores a few things that determine your actual cost and experience:

Retry cost. If a cheaper model has a higher failure/timeout rate under load, your effective cost per successful completion can end up higher than a pricier, more stable alternative.
Latency variance. A model that's fast on average but spikes unpredictably under concurrent load is a worse fit for user-facing products than one that's consistently "good enough."
Regional routing and availability. Where the model is actually served from affects both latency and uptime for your specific user base — this rarely shows up in a pricing page.

None of this is unique to Chinese LLM providers, but it's especially relevant there right now because Qwen, DeepSeek, GLM, Kimi and others are iterating fast, and public benchmarks lag behind real-world deployment experience.

A rougher, three-dimension framework

Instead of "which is cheapest," I've found it more useful to ask three questions when evaluating any LLM API, including Qwen and DeepSeek:

Task fit — Is this model's strength (long-context reasoning, code generation, multilingual handling, etc.) actually aligned with what I'm building, or am I picking it because it's popular?
Consistency under real load — Not "what's the benchmark latency," but "what's the latency distribution when I send it a realistic concurrent workload for a few hours?"
Total cost to a successful response — Base price, adjusted for retries, timeouts, and any fallback calls you need to make.

None of these require expensive tooling. You can get a decent read on all three with a few hours of scripted testing against your own representative prompts.

Applying it: Qwen vs DeepSeek, informally

I ran a small, non-scientific comparison across a mix of tasks — code generation, long-document summarization, and Chinese-English translation — sending the same prompts to both models.

Some qualitative observations from that run (your mileage will vary depending on prompt style, region, and time of day):

On code generation tasks, DeepSeek's responses tended to be more consistently well-structured for my prompts, though Qwen was competitive on shorter snippets.
On long-context summarization, Qwen handled longer inputs without noticeable degradation more often than DeepSeek did in my sample — but my sample size was small enough that I wouldn't call this conclusive.
Latency was close enough between the two that, for my workload, task fit mattered more than raw speed.

I'm intentionally not publishing hard numbers here — a handful of test runs on my machine isn't a benchmark, and I'd rather be honest about that than dress it up as one. If you're making a real decision, run your own representative workload through both.

To keep the comparison fair, I ran both models through the same test harness using a single OpenAI-compatible API gateway (I used RouteAI for this, mainly because it let me switch between Qwen and DeepSeek without juggling separate SDKs or keys) so the only variable changing was the model, not my client code. Any gateway that gives you a consistent interface across providers works for this purpose — the point is isolating the variable you're actually trying to measure.

The takeaway

Price comparisons aren't wrong, they're just incomplete. If you're choosing between Qwen, DeepSeek, or any other LLM API, I'd suggest spending less time on the pricing page and more time running your own workload through a task-fit / consistency / total-cost lens. The cheapest model on paper isn't always the cheapest model in production — and it's rarely the right model just because it's cheapest.

Curious what framework other people use when evaluating LLM providers — do you weight latency consistency as heavily as I do, or is task fit the dominant factor for you?

TL;DR: Comparing LLM APIs by price alone ignores retry cost, latency consistency, and regional availability. A quick informal test of Qwen vs DeepSeek across code, summarization, and translation tasks showed each has different strengths — evaluate on task fit and real-world consistency, not just the pricing table.

Here's the tool I referenced in this post: www.fastrouteai.com

I Wasted 3 Days on Bad DeepSeek Prompts — Here's What Actually Works

Felix — Thu, 06 Aug 2026 09:02:23 +0000

A practical guide to DeepSeek prompt engineering, better AI responses, and writing prompts that work across different LLMs

The pain point

I'll be honest: when I first started using DeepSeek, I basically copy-pasted the same prompts I'd been using with ChatGPT for months. Same tone, same structure, same lazy one-liners like "write me a function that does X." It worked fine with ChatGPT. With DeepSeek? I got responses that were either too verbose, oddly formatted, or just missed the point entirely. I spent three days thinking DeepSeek was "worse" — until I realized the model wasn't the problem. My prompts were.

The core idea

DeepSeek is not a drop-in replacement for ChatGPT. It responds differently to role-setting, formatting instructions, and task boundaries. If you're prompting it the same way you prompt other models, you're leaving quality on the table — and probably blaming the wrong thing.

Why this happens

Most of us build prompting habits around one model (usually ChatGPT, since it's the default most people learn on) and assume those habits transfer. But different models are trained with different instruction-following patterns and different defaults for verbosity, formatting, and "helpfulness." DeepSeek tends to take instructions more literally — if you don't explicitly define the output shape, it fills the gap with its own assumptions, which often aren't the ones you wanted. What reads as "the model is dumb" is usually "the model did exactly what I asked, and I asked for the wrong thing."

The methods that actually fixed it

1. Give it an explicit role, every time

Wrong:

Write a function to validate emails.

Right:

You are a senior backend engineer. Write a Python function that validates email addresses using regex. Follow PEP8 style. Do not add explanations outside code comments.

Without a role, DeepSeek defaults to a generic, sometimes overly cautious tone. Setting the role narrows its behavior fast.

2. Define the output format, not just the task

Wrong:

Summarize this article.

Right:

Summarize this article in exactly 3 bullet points, each under 20 words. No intro sentence, no conclusion sentence.

Vague tasks get vague — or bloated — answers. DeepSeek will fill unspecified format space with its own defaults, which are rarely what you want for production use.

3. Break big tasks into staged prompts

Wrong:

Build me a full REST API for a to-do app with auth, database, and tests.

Right:

Step 1: Design the database schema for a to-do app with user auth. Output only the schema, no code yet.

(then follow up with endpoints, then auth, then tests — one stage per prompt)

One giant prompt forces the model to guess priorities. Staged prompts keep every output reviewable and correctable before you move forward.

4. Force it to flag uncertainty

Wrong:

Fix this bug in my code. [code]

Right:

Fix this bug in my code. If you're not sure what's causing it, list your top 2 hypotheses instead of guessing a fix. [code]

This one change alone cut my "confidently wrong" answers by a lot. Models will guess by default unless you explicitly tell them not to.

5. Test the same prompt across models

Wrong: Assuming a prompt that works on ChatGPT will behave the same on DeepSeek, Qwen, or anything else — and only finding out it doesn't after shipping.

Right: Run the same prompt against a couple of models side by side before committing to one. This is honestly the step that saved me the most time — I stopped guessing which model handled my prompt style best and started actually checking. I've been testing this through an OpenAI-compatible API, letting me swap between DeepSeek, Qwen, and other models without rewriting my whole integration each time — it's basically a config change instead of a rebuild.

A template you can copy right now

You are a [ROLE].
Task: [SPECIFIC TASK]
Output format: [EXACT FORMAT — length, structure, style]
Constraints: [WHAT TO AVOID]
If uncertain: [WHAT TO DO INSTEAD OF GUESSING]

Drop this into any DeepSeek prompt and fill in the brackets. It forces you to define the four things DeepSeek needs explicitly — role, format, constraints, and uncertainty handling — instead of letting it guess.

The takeaway

Three days of "DeepSeek isn't good enough" turned into ten minutes of "oh, I was prompting it wrong." The model wasn't the bottleneck — my ChatGPT-shaped habits were. If your outputs feel off, don't switch models yet. Switch your prompting approach first. It's the cheaper fix, and it's the one that actually worked for me.

(I used RouteAI for this experiment.)
https://www.fastrouteai.com

How I Built My First AI App with an OpenAI Compatible API

Felix — Tue, 04 Aug 2026 08:08:16 +0000

When I started experimenting with AI APIs, I thought the hardest part would be building the application logic.

It wasn't.

The frustrating part was switching between different AI providers.

Every API had different:

authentication methods
request formats
SDK styles
model naming systems

My small prototype slowly became a collection of provider-specific code.

A simple model change required changing multiple files.

That was when I started looking into OpenAI compatible APIs.

What is an OpenAI Compatible API?

An OpenAI compatible API follows the same request format as OpenAI's API.

For developers, this means:

existing OpenAI SDKs can often be reused
applications need fewer code changes
testing different models becomes easier

Instead of rewriting integration code every time, you can keep a consistent interface.

My First Example

Here is a simple Python example.

First install the OpenAI SDK:

pip install openai

Then connect to an OpenAI compatible endpoint:

from openai import OpenAI

client = OpenAI(
api_key="YOUR_API_KEY",
base_url="YOUR_OPENAI_COMPATIBLE_ENDPOINT"
)

response = client.chat.completions.create(
model="deepseek-chat",
messages=[
{
"role": "user",
"content": "Explain API gateways in simple words."
}
]
)

print(response.choices[0].message.content)

The interesting part is that the application code stays almost identical.

The endpoint changes, but the development workflow remains familiar.

Why This Matters for Indie Developers

For small teams, the biggest challenge is usually not creating another AI feature.

It is managing complexity.

A unified API approach can help developers:

experiment faster
compare different models
avoid rewriting integrations repeatedly
How RouteAI Fits In

While exploring different OpenAI compatible solutions, I found platforms like RouteAI that provide a unified gateway for accessing multiple supported AI models through a compatible interface.

For developers, this means you can focus more on building applications instead of maintaining multiple API integrations.

What I Learned

AI development is moving from "which model should I use forever?"

to:

"How quickly can I test and switch between models?"

A flexible API layer becomes more useful as the AI ecosystem grows.

TL;DR: OpenAI compatible APIs let developers use familiar API formats while experimenting with different AI models more efficiently.