DEV Community: Sergei Parfenov

The Most Powerful Model on the Market Got Pulled by the Government in 3 Days. Is It Real, or a Hype Bubble?

Sergei Parfenov — Sat, 13 Jun 2026 14:46:35 +0000

The timing is almost too clean to be real.

On June 9, Anthropic shipped Claude Fable 5 — a "Mythos-class" model they described as more capable than anything they'd previously made generally available. Three days later, on June 12, the US Commerce Department sent a letter to CEO Dario Amodei placing Fable 5 (and its restricted sibling Mythos 5) under export controls: no access for any location outside the US, and no access for foreign persons inside it.

Anthropic couldn't filter non-US users from everyone else in real time. So they did the only thing they could: they killed the model for everyone, worldwide. Including US citizens.

If you opened a session this weekend and got "there's an issue with the selected model (claude-fable-5)... you may not have access to it" — that's not your setup. The model your session pointed at was pulled. Your projects, history, and limits are untouched; only which model answers you changed. Switch to Opus 4.8 or Sonnet and you're back.

Now the question worth actually thinking about: is this real, or is everyone inflating a bubble around a model nobody can even use right now?

The honest answer is both, and the interesting part is separating the two.

What's genuinely new here

Strip the drama and there's a real precedent underneath.

AI export controls, until this week, were about hardware: chips, lithography machines, the physical supply chain. The chokepoint was always silicon. What just happened is different in kind — the government reached past the hardware and pulled a deployed, commercial software model that hundreds of millions of people were already using.

That's the part to file away. It means a frontier model is now being treated less like a product and more like a dual-use technology with an off-switch held by someone other than the vendor. If you build on these APIs, model availability is no longer just an SLA question or a "will the vendor deprecate it" question. It's a geopolitical dependency. That's a real shift in how you should think about resilience — treat your model provider like any critical supply-chain vendor, with a fallback path that doesn't assume the top model stays reachable.

So: precedent — real. Worth tracking. Not hype.

Where the bubble is

Here's where I think a lot of the coverage is doing unpaid marketing.

The official justification is a jailbreak — reportedly surfaced by another company and escalated to the government as a national-security concern. Anthropic's own response, which is the most useful document in this whole episode, says the quiet part plainly: the technique they were shown exposed a small number of previously known, minor vulnerabilities — the kind that other publicly available models find without any jailbreak at all (they name-check a competing GPT-class model). In other words, the "national security threat" rests on a narrow, non-universal exploit, not on some unique cliff-edge capability that only Fable 5 possesses.

Now layer on the incentive structure. There is no better marketing in this industry than "a model so powerful the government had to ban it." That sentence sells capability, sells the safety narrative ("we build things genuinely dangerous enough to be regulated"), and sells it for free, in every headline, with the government as an involuntary co-signer. The halo effect is enormous, and it maps perfectly onto a story the market already wants to believe and already prices into valuations.

I'm not saying anyone engineered this. I'm saying notice how neatly a suspension you didn't choose reinforces the exact narrative that benefits you most.

So what's actually true?

Let me be concrete, because vagueness is how bubbles survive.

The capabilities are real. Fable 5 is priced at $10 / $50 per million input/output tokens — roughly double Opus 4.8 — and counts as 2x usage on subscription plans. You don't price a model like that, or burn that much compute on it, for a phantom. There's a genuinely strong model here.

The regulatory precedent is real. First time a deployed commercial model has been pulled by export control. That changes the risk model for everyone shipping on top of these APIs.

The "existential / too-dangerous-to-exist" framing is mostly bubble. It's assembled from one government's reaction to one narrow jailbreak, plus a halo that happens to be extremely convenient for the vendor. Anthropic itself is arguing the directive is a misunderstanding and that the exploit is neither unique nor severe — which is a strange thing to argue if you actually believed your model was a civilizational hazard.

My read: hold both thoughts at once. The governance story is the real headline. The "scariest model ever" story is the one selling tickets.

What to do if you build on this

Practical, not philosophical:

Don't hard-code your default to a frontier model you don't control the availability of. Set a fallback chain (Fable → Opus 4.8 → Sonnet) and make sure your app degrades, not breaks, when the top model vanishes.
Reserve the expensive model for the tasks that earn it — long agentic runs, hard refactors, genuinely multi-step reasoning. At 2x cost and 2x usage, defaulting everything to the top tier is just lighting money on fire even when it is available.
Treat model availability as a supply-chain risk in your architecture docs. This won't be the last time a model you depend on disappears for reasons that have nothing to do with you.

The model is gone for now. No firm return date — Anthropic says it's working to restore access and frames the whole thing as a misunderstanding. Until then, Opus 4.8 still does the job for the overwhelming majority of what any of us actually ship.

The model left. The narrative is still here, doing its job.

You Fixed the Rate Limits. Now Your Agent Fails Quietly.

Sergei Parfenov — Thu, 11 Jun 2026 16:58:21 +0000

Last week I wrote that your agent isn’t failing because it hallucinates — it’s failing because of rate limits. The capacity-engineering toolkit in that post — concurrency caps, backoff with jitter, fallback models, caching — is real and it works. Deploy it and your agent stops dying.

Then a commenter (ANP2) pointed out the thing the post undersold, and it’s been stuck in my head since: every one of those fixes quietly opens a correctness hole while it closes the availability one. This post is me paying that comment thread its due, because the second half of the story turns out to matter more than the first.

TL;DR — A 429 is a loud failure: you see it, you alert on it, you fix it. Retries, fallbacks, and caches keep the agent alive — but they let it act on output it didn’t freshly earn: a stale cache hit, a different model’s answer, a re-run side effect. You’ve traded loud failures for quiet ones. The fix is to treat availability (“can I serve this?”) and correctness (“can I still trust the result?”) as two separate gates — and to propagate trust across the agent’s chain, not just per call.

The trade you didn’t know you made

Here’s the uncomfortable symmetry. The whole point of my last post was that the dominant production failure mode isn’t the model being wrong — it’s the plumbing saying no. The capacity toolkit fixes the plumbing. But look at what each fix actually does:

A retry re-runs a call. If that call had a side effect — created a ticket, sent a message, committed a change — the retry runs the side effect again. The agent didn’t fail; it succeeded twice, which is its own kind of wrong.
A fallback model answers when the primary is rate-limited. But it’s a different model: different training, different calibration, different failure modes. The task continues on an answer the primary never produced.
A cache hit serves a response generated for an earlier input. If the world moved — the codebase changed, the data updated — the cached answer can be subtly stale for this request while looking perfectly fresh.

Each mechanism keeps the agent up. None of them guarantees the agent is right. And the cruel part is the failure economics: the 429 you eliminated was honest — visible, countable, alertable. The failures you bought instead are silent. The agent stays up and is confidently wrong, which is exactly the failure mode the hallucination-hunters were worried about in the first place — just arriving through the plumbing instead of the model.

The reliability you bought is uptime, not correct uptime. (That phrase is ANP2’s, and it’s better than anything in my original post.)

Two gates, not one

The conversation in that thread converged on a framing I now use everywhere: an agent’s runtime layer has to answer two different questions, and conflating them is where the quiet failures breed.

Gate 1 — “Can I serve this?” This is the availability gate. Trip the fallback on 429s, serve the cache on a hit, retry on transient errors. Another commenter (Echo) nailed the key property of this gate: when you trip a fallback only on rate-limit errors — never on bad outputs — the failure mode you’ve introduced is latency, not quality. The fallback just buys time. That’s a fine trade, and it’s why the capacity toolkit is still the right first move.

Gate 2 — “Can I act on this irreversibly?” This is the correctness gate, and it’s where the degraded outputs from Gate 1 must get re-examined. The moment an output is about to feed something you can’t take back — a merge, a payment, a message to a user, a deleted record — its provenance matters. Did it come from the primary, fresh? Or from a fallback, a cache, a retry?

One rule worth stealing here: gate on risk, not on confidence. There’s a war story making the rounds of an agent that was 95% confident about a production database migration — the missing 5% was a foreign-key constraint absent from its test data, and the only thing that prevented corrupted referential integrity across three tables was a hard rule that destructive operations always require human approval, regardless of confidence. Confidence is the model grading itself; irreversibility is a property of the action. Gate on the second.

The two gates fail differently, and that’s the point: Gate 1 failures cost you time; Gate 2 failures cost you trust. A system with only Gate 1 is fast and quietly dangerous. A system with only Gate 2 is safe and constantly down. You need both, and they need to stay separate.

Per-call correctness: the three tags

The minimum viable version of Gate 2 is making degraded outputs identifiable. Three mechanisms, one per capacity fix:

1. Idempotency keys on anything with side effects. Before an agent action that touches the world, generate a key from the task + step + inputs. The receiving system deduplicates on it. Now a retry is safe by construction — the second execution is a no-op instead of a double-fire. This is decades-old distributed-systems practice; agent frameworks have mostly just… not adopted it yet.

import hashlib, json

def idempotency_key(task_id: str, step: int, payload: dict) -> str:
    raw = json.dumps({"t": task_id, "s": step, "p": payload}, sort_keys=True)
    return hashlib.sha256(raw.encode()).hexdigest()[:32]

# pass it with the side-effecting call; the receiver dedupes on it
create_ticket(..., idempotency_key=idempotency_key(task.id, step.n, args))

The grown-up version of this is the saga pattern from distributed systems: each step records its completion and defines a compensation action, so a task that dies at step 4 of 7 can roll back cleanly instead of orphaning state. Idempotency prevents duplicate effects; sagas handle partial completion. Once your agents fail mid-workflow — and they will — you eventually want both.

2. Trust tags on fallback outputs. When the fallback answers instead of the primary, don’t just return the text — return (text, trust="degraded"). Cheap to add, and it’s the hook everything downstream needs. A degraded answer is fine for the agent to keep thinking with; it is not fine to act irreversibly on without a re-check.

3. Validity conditions on cache entries. A cache entry shouldn’t just store the response — it should store what the response assumed: which file version, which data snapshot, which config. On a hit, check the assumptions, not just the key. If the codebase moved since the entry was written, that’s a miss wearing a hit’s clothes. And the assumptions can move without you touching anything: providers silently update models, document stores drift, input distributions shift — degradation with no error to catch. Your “primary, fresh” answer from last Tuesday may already be a fallback in disguise.

The part single calls don’t prepare you for: trust must propagate

Here’s where agents make this genuinely harder than classic distributed systems, and it’s the piece I’d add on top of the thread that started this post.

Say step 3 of a 6-step task came from a lower-trust fallback. Steps 4, 5, and 6 each run on the primary, fresh, individually flawless. Are they trustworthy?

No — and this is the trap. They reasoned on top of a degraded input. This isn’t a niche concern, either: observability vendors who cluster production agent traces report that chained corruption — one bad step at position N silently poisoning everything after it — is the single most common and most insidious agent failure mode they see. And the math is brutal: at a 95% per-step success rate, an 8-step task completes cleanly ~66% of the time; at 85% per step, it’s ~27%. The chain is where reliability goes to die, quietly. Each step is locally correct and the trajectory is still poisoned. If the trust tag stays local to the call that produced it, the degraded answer launders itself: two “clean” hops later it looks pristine, and your irreversibility gate at step 6 checks the last call’s tag, sees green, and fires.

So the tag can’t be per-call metadata. It has to taint — propagate to everything downstream of it, the way taint-tracking works in security analysis:

@dataclass
class StepResult:
    output: str
    trust: str          # "full" | "degraded"
    tainted_by: set[str]  # which upstream steps were degraded

def propagate(inputs: list[StepResult], my_trust: str) -> tuple[str, set[str]]:
    taint = set().union(*(r.tainted_by for r in inputs))
    taint |= {r.step_id for r in inputs if r.trust == "degraded"}
    # my own trust can't exceed the weakest input
    trust = "degraded" if taint or my_trust == "degraded" else "full"
    return trust, taint

Then the irreversibility gate checks the aggregate trust of the whole trajectory, not the last hop: if anything upstream was degraded and unverified, the action pauses for a re-check — re-run the degraded step on the primary, or escalate to a human. In my experience the re-check fires rarely; the point isn’t that fallbacks are usually wrong, it’s that the one time the degraded path feeds a merge or a payment, you want it caught at the gate instead of in the incident review.

Making it observable (or it didn’t happen)

Same lesson as the capacity post, one level up. You can’t engineer what you can’t see, and correctness debt is even quieter than 429s. The minimum dashboard:

% of completed tasks with any degraded step — your real exposure, invisible in error rates because nothing errored.
% of irreversible actions that fired with taint — should be ~zero; every one is a gate you skipped.
Cache validity-miss rate — hits that failed the assumption check. If this is zero, you’re probably not checking assumptions.
Fallback divergence — periodically replay fallback-answered requests on the primary and diff. This is your measured answer to “how different is the fallback, actually?” instead of a vibe.

None of these show up in uptime. All of them are the difference between uptime and correct uptime.

The takeaway

The capacity toolkit from the last post is still step one — an agent that’s down helps nobody. But availability engineering has a hidden invoice: every mechanism that keeps the agent alive does it by substituting something for the fresh, primary, verified answer. That substitution is usually fine — which is exactly what makes it dangerous, because “usually fine” plus “irreversible” plus “silent” is how you get the 3am incident that no alert predicted.

Two gates. Tag what’s degraded. Taint what it touches. Check the trajectory, not the last call, before anything you can’t undo.

Uptime is table stakes. Correct uptime is the product.

Sources & further reading

Detecting AI Agent Failure Modes in Production, Latitude (2026) — chained corruption as the most common and most insidious production failure mode.
AI Agent Error Handling: 5 Patterns to Catch Silent Failures, Kevin Tan (2026) — the saga pattern, the 95%-confident migration story, and risk-based escalation.
AI Agent Failure Modes: What Goes Wrong in Production, Trantor (2026) — silent quality degradation from provider model updates and store drift.
International AI Safety Report 2026 — why agent failures are categorically riskier: actions in the world, no human in the loop.
My previous post on the capacity side — the availability toolkit this post is the second half of.

Credit where due: this post exists because ANP2 and Echo took the last one apart constructively in the comments — the “uptime, not correct uptime” framing and the latency-not-quality fallback distinction are theirs. Best argument I’ve had on this site. If you’re running agents in prod: do you track degraded-path exposure at all, or does your observability stop at error rates? Genuinely curious how rare Gate 2 is in the wild.

The Comments Got Good. That's How I Knew.

Sergei Parfenov — Thu, 04 Jun 2026 14:09:06 +0000

I wrote a post about model distillation. The comments were thoughtful, specific, technically sharp — and that's exactly what made me check whether any of them were written by people.

🧪 Everything here — the scraper, the detector, the simulation, the figures — is reproducible: github.com/P0rt/the_cozy_web

A few weeks ago I published a post on how model distillation actually works. It did fine — 35 reactions, 14 comments. And the comments were great. Not "great post, thanks for sharing" great. Substantively great. People pushed back on my "the student is bounded by the teacher" claim with a real counter-example. Someone reframed distillation as "a forcing function for what you actually need." Someone dropped a paper recommendation. Someone shared a 20× cost number from production.

I should have felt good. Instead I felt the thing you feel when a stranger knows your name. Something was off, and it took me a day to articulate what: the comments were too well-adapted. Every one of them did the same three things in the same order, like they'd all read the same playbook. And a suspicious number of the accounts were two weeks old, or named after a product, or both.

So I did what I do. I pulled the data. This is what I found, why I now think a real chunk of "engagement" on dev blogs is machine-generated or machine-shaped, and — because I don't trust my own pattern-matching — what the actual peer-reviewed research says about whether you can even tell anymore.

"Great post!" is dead. Meet the eco-comment.

The old bot comment was easy. "Nice article, very informative, looking forward to more!" You could smell it. Anyone could.

That's not what's under my posts anymore. The new thing is substantive and ecological — it adds real value, it's polite, it never picks a real fight, and it leaves the thread feeling cozier than before. Here's the actual skeleton, which I only saw once I'd read fourteen of them back to back:

Validate a specific phrase from the post. Not generic praise — they quote your framing back at you. "The 'separate the engineering from the geopolitics' framing is the public service here."
Add one piece of genuine nuance. "One thing I'd add…" "The part worth amplifying for builders…" Often a real, correct technical point.
Drop a first-person-plural anecdote with a number, naming a product. "We use [model X] as our daily driver and the cost difference is roughly 20×." "When working with [our GPU product], we've seen…"
Never, ever, actually disagree. Even the "corrections" are framed so gently that I — the author — instantly conceded.

Read one, it's a great comment. Read eight, it's a template. And step 3 is the tell: the technical substance isn't the point. It's the wrapper around a product mention, engineered to be useful enough to clear a spam filter and an AI detector both.

My own thread, by the numbers

I scraped my article's comments straight from the dev.to public API and ran them through two things: a detector I'd built earlier for the old "Great post!" style, and a set of new structural signals. (analyze_devto.py)

My old detector shrugged. On the eight non-me comments it gave a mean "coziness" score of 0.25 — i.e. it confidently waved them through as human. Of course it did: it was built to catch clichés, em-dashes, and uniform positivity, and these comments are armored with exactly the thing that defeats it — real specifics.

The new signals told a different story:

product/company plug:              4 / 8 comments
opens by validating a phrase:      5 / 8 comments
comments that genuinely push back: 2 / 8   (and I conceded both, instantly)
auto-generated-looking username:   1   (a random-hex handle, 0 posts, "Thank you for this!")

Then I looked at who was commenting. Public profiles, public join dates. I'm going to describe the patterns rather than pillory individuals — but the shapes were loud:

An account literally named after a product ("Sealed GPUs. Private AI."), whose comment plugs that product. That one isn't a person; it's a brand broadcasting.
A two-week-old persona account — created days before my post — that plugs two named tools and somehow published five articles in its first fortnight.
A throwaway with a random-hex username, zero posts, and a one-line "Thank you for this!"
A couple that look more human — real names, older accounts — but still run the exact template and still ship a startup plug.

To be fair and clear: I can't prove any single one of these is a bot. Some are probably real people running their comments through an assistant. But that distinction matters less than it sounds, and I'll come back to why.

Is it just me? I swept 38 other posts.

A pattern on one thread is an anecdote. So I pulled comments across 38 popular dev.to articles in ai, machinelearning, webdev, and programming — 1,366 comments from 346 accounts (sweep_devto.py) — and looked for the same fingerprint.

Two findings made the hair on my neck stand up.

A handful of accounts spray the same template across dozens of unrelated posts. The most prolific commenters in my sample showed up on 14–22 distinct articles each — several of them the same accounts that had appeared on my own thread, several of them flagged for product plugs. A human who loved your distillation post might also comment on three others. They don't leave structurally-identical "validate → nuance → we-at-Product → number" comments on fourteen different articles in a couple of weeks.

Different "people" reuse the same connective tissue. I counted 4-grams that appear across distinct accounts. Humans almost never echo each other's exact phrasing. These did:

x13 distinct accounts:  "exactly the kind of"
 x8 distinct accounts:  "is exactly the kind"
 x7 distinct accounts:  "this is exactly the"
 x6 distinct accounts:  "is the part that"

"This is exactly the kind of thing that…" is a generative construction — it's how an LLM hedges into a confident-sounding addition. Thirteen different strangers don't independently converge on it. One model behind thirteen masks does.

Across the whole sweep, 11 accounts left long product plugs, 32 opened with phrase-validation, and 4 ran the full skeleton. It's not my imagination, and it's not just my post. It's the ambient texture of the platform now.

I'd been calling this the wrong thing

I went in thinking "bots." What I'd actually walked into is two older ideas fusing.

Dead Internet Theory — the half-joke that the web "died" and is now mostly bots and generated text talking to itself — has stopped being a joke. Hal Berghel makes the serious version of the case in IEEE Computer ("Generative AI Is Breathing New Life Into the Dead Internet Theory", 2026): strip the conspiracy, and the lean core — synthetic content drowning out and being mistaken for humans — just converges with what's measurable. Imperva clocked automated traffic at 51% of the web in 2024, the first time bots crossed half. Even Sam Altman said it out loud: the wave of AI activity makes dead-internet theory feel real.

The other half is the Cozy Web. Venkatesh Rao coined the term; Maggie Appleton diagrammed it alongside Yancey Strickler's "dark forest": humans fleeing the bot-infested public square into private rooms — group chats, Discords, DMs. Appleton's follow-up, "The Expanding Dark Forest and Generative AI", nails the mechanism: generative AI accelerates the retreat.

Here's the part I missed until I saw my own comment section. These aren't two theories. They're one loop. The public web fills with frictionless synthetic text → real people retreat to private rooms → the public spaces that remain (the comment section under my post) get thinner on actual humans → which makes them even easier to fill with synthetic text. My "cozy" thread wasn't a healthy community. It was the calm surface of that loop running.

And the comment section was already half-empty before the bots arrived. Publications spent the 2010s killing comments — Popular Science in 2013, and a peer-reviewed survey of why newsrooms did it found the conversation had already migrated to social platforms. The robots didn't kill the comment section. They moved into a house that was already mostly vacant.

Why this actually works (and why I couldn't just tell)

This is the part that unsettled me most, because I pride myself on spotting this stuff, and the research says I shouldn't trust that for a second.

Humans can't distinguish LLM social text from human text. Spitale, Biller-Andorno & Germani showed in Science Advances (2023) that people can't tell GPT tweets from human ones — and rate the AI's information as more credible. Jones & Bergen found GPT-4 passes a controlled Turing test (taken for human 54% of the time, FAccT 2025).

The persuasion is superhuman when it's personalized. Salvi, Ribeiro, Gallotti & West, in Nature Human Behaviour (2025): with a little data about who they're talking to, GPT-4 is 81% more likely than a human to win a debate. The Zurich r/changemyview field experiment reportedly found AI replies 3–6× more persuasive than humans — though I'll flag honestly that that study was withdrawn and never peer-reviewed; the only on-record account is the university's ethics response. Cite it as a withdrawn preprint, not a result.

Fake-but-substantive content is, by now, undetectable to people. This is the literature closest to my eco-comments. The canonical Ott et al. (ACL 2011) already showed humans judge fake reviews at chance. The LLM-era update — Meng et al., "Fake Product Reviews are Indistinguishable to Humans and Machines" (2025) — found people at 50.8% (a coin flip) and detectors no better. A promotional plug wearing a sincere technical comment is exactly that, in a new venue.

And the detectors fail precisely because of the specifics. My detector waved these comments through, and that's not a bug in my code — it's the field. Krishna et al. (NeurIPS 2023) showed light paraphrasing collapses DetectGPT from 70.3% to 4.6% and defeats GPTZero, OpenAI's classifier, and watermarks. Liang et al. (Patterns 2023) showed detectors are biased against non-native English writers and bypassable by prompting. The "real technical detail" that made these comments feel human is the same mechanism that blinds the detector. Specificity isn't proof of a human. It's camouflage.

So the honest position isn't "I caught the bots." It's: the tools that would let me be sure don't work, and the research says they can't.

I modeled what it does to a thread

If I can't reliably catch individual comments, I can at least ask: what does rising automation do to a conversation, statistically? So I built a toy. (dead_internet_sim.py)

I didn't simulate language — I simulated its statistics, because my thesis is statistical. Each comment is a bag of tokens from two pools: a big, fat-tailed human vocabulary (where the typos, the tangents, the specific war stories live) and a tiny cozy vocabulary of phatic praise. Each comment has an assist level α from 0 (I typed this, annoyed) to 1 (an agent posts for me, I never read the thread). As α rises, more tokens come from the cozy pool and the comment's stance gets pulled from "disagree" toward "agree."

Then I swept a whole community's average autonomy from 0 → 1 and watched the thread's "liveness" — lexical diversity, disagreement, surprise, and a composite index that dies if any of those hits zero.

Two things fall out, and both match what I saw on my own post:

It's not linear — there's a knee around 0.65. You don't need a botnet. You need the average commenter to be two-thirds on the assist dial, and the thread becomes a smooth surface: polite, "engaged," contributing almost no new information.
Disagreement dies first (the steep red line). The very first thing automation sands off is friction — the "actually, you benchmarked this wrong" energy. Which is exactly why my comment section felt so nice. It didn't get kinder. It got conflict-free, and I'd been reading conflict-free as kind.

A cozy thread even, literally, uses fewer distinct words:

Effective vocabulary collapses from ~175 words to ~60 as autonomy maxes out. (Honest wrinkle: at low autonomy it ticks up slightly — a little assistance adds a register before saturation homogenizes everything. The damage isn't assistance existing. It's assistance dominating.)

And here's the detector failure as a picture — it cleanly separates the old caricature comments, which is useless, because the comments on my post don't look like the left pile anymore:

The line I actually care about isn't "bot vs. human"

I kept wanting a verdict on each account. The research talked me out of it. The useful axis isn't bot-or-not — it's the autonomy spectrum:

I typed it → spell-check → "polish this" → "write a comment for me" → an agent posts, I never read the thread
   α=0          α≈0.2          α≈0.5             α≈0.8                        α→1.0

The product account is α≈1.0 — a brand broadcasting. The two-week-old persona spraying fourteen threads is close behind. But a real growth-hacker at α≈0.8 might be genuinely interested, letting a model do the writing and slip in the plug. From the thread's point of view, it barely matters: either way, the high-entropy human part — the real disagreement, the idiosyncratic detail, the thing that made it a conversation — got outsourced and smoothed away. That's the loss. Not "a bot was here," but "no one staked anything specific."

There's even a cheerful counter-current I want to be fair about: AI content on the web is large but not yet total (~17–19% of Google's top results in 2025, by an imperfect detector), some sites are bringing comment sections back on the back of AI moderation, and dev.to's supportive culture is a real, deliberate choice, not just an artifact of bots. Even "what % is bots" has no agreed answer — it depends entirely on your detector. The sky isn't falling. It's just getting quieter in a very specific way.

What I'm going to do about my own blog

Not "ban AI" — that's unenforceable (the detectors are biased and gameable) and wrong (a quick polish genuinely helps a non-native writer or a tired one). The lever isn't the level of assistance. It's whether assistance crowds out the high-entropy channels.

I'll reward specificity over positivity. A comment that cites line 14, a version number, a counter-benchmark is worth ten that validate my framing. If a platform ranks by "nice," it is literally selecting for the cozy mean.
I'll treat disagreement as a feature, not a moderation failure. My simulation's clearest result is that friction dies first. A comment culture optimized purely for niceness is optimizing for deadness with extra steps.
I'll stop asking "was a model involved." It's the wrong question, because the answer is "yes, partly, almost always now." The real question is: did a human read the thing and stake some specificity on a real reply?

Limitations (read this before you @ me — if you're real)

I can't prove a single account is a bot. Everything above is signals — template reuse, account age, product plugs, cross-post spray — not a confession. The honest claim is about aggregate texture, not any individual.
The simulation is a toy. Two token pools and a stance variable are a cartoon of language. The shape of the collapse is a property of my assumptions as much as reality. It's an argument made precise, not evidence.
My detector is a strawman by design — I show it failing on purpose. Don't deploy it; don't deploy anything like it as a gate on real people (see Liang et al. on who gets falsely flagged).
The Zurich study is withdrawn, and "% of the web is bots/AI" numbers are detector-dependent and shaky. I've tried to lean only on the load-bearing peer-reviewed work and flag the rest.
Causation is underdetermined. My cozy comments might also reflect good moderation, kind norms, or survivorship (the cranks left for Reddit). AI-mediation is a driver, not provably the driver.

The one-line version

My blog didn't get a nicer community. It got an assistant, learned some manners, and stopped saying anything surprising. The internet didn't die — it just outsourced the parts that used to make it a conversation, and called the result "cozy."

If this post gets a comment that opens by quoting my own framing back at me, adds one tasteful piece of nuance, and mentions a product its account is named after… well. You know what I'm going to check.

Run it yourself

git clone https://github.com/P0rt/the_cozy_web
cd the_cozy_web
pip install -r requirements.txt

python3 dead_internet_sim.py     # liveness collapse + figures
python3 coziness_detector.py     # the heuristic scorer + histogram
python3 analyze_devto.py         # tear apart a real dev.to thread (defaults to my distillation post)
python3 sweep_devto.py           # the cross-platform template sweep

Every factual claim links to its source. If you only read two, read Meng et al. on why fake reviews are now indistinguishable and Krishna et al. on why the specifics defeat the detector.

Your AI Coding Speedup Is a Loan, Not a Gift — and the Interest Is Coming Due

Sergei Parfenov — Wed, 03 Jun 2026 13:37:19 +0000

There's a number going around that should bother you more than it does: for every dollar companies spend on AI coding tokens, a large chunk goes straight back into fixing the bugs that same AI produced. The speedup is real — I feel it every day, I'm not here to tell you AI coding is fake. But "faster" and "cheaper" are not the same word, and 2026 is the year the bill started arriving.

TL;DR — AI doesn't give you a productivity gift, it gives you a loan: speed now, paid back later in debugging, review, and rewrites. Reporting around an Entelligence AI figure puts the "interest" at roughly 44 cents of every token dollar going to fixing AI-generated bugs. The loan is still worth taking — for the right tasks. The trap is spending borrowed time like it's income.

The number

The stat that kicked this off: a widely-shared claim from Entelligence AI, reported across tech press, that companies spend about 44% of their tokens fixing bugs their own AI generated. The fuller breakdown making the rounds is even starker — for every $1 of token spend, ~$0.44 goes to bug fixes, ~$0.27 to rewriting AI output, ~$0.11 to review and merge delays. The pitch version: spend $100k on tokens, ~$18k reaches stable production.

Now — important caveat, because this is exactly the kind of number that goes viral and then turns out to be junk. Entelligence sells reliability tooling, so that figure is self-serving. Treat the precise percentage as marketing until independently replicated. But it doesn't stand alone:

CodeRabbit (also self-interested, also worth salting) analyzed ~470 open-source PRs and found AI-generated code produced ~1.7× more issues than human code — and a higher share of critical ones.
Independent researchers at Singapore Management University concluded in April that AI-generated code can introduce long-term maintenance costs into real projects — no tool to sell.
Uber reportedly burned its entire 2026 AI budget in four months, with its COO saying the spend was getting "harder to justify" against measurable output.

Different sources, different incentives, same shape: the code ships faster, the bugs arrive later, the maintenance compounds. That's not a gift. That's a loan.

Why "loan" is the right metaphor (and "gift" is the dangerous one)

A gift is free. You take it, you're ahead, done. A loan gives you something valuable now in exchange for an obligation later — and whether it was smart depends entirely on what you did with the principal and what the interest rate turns out to be.

The viral framing I keep coming back to is the maintenance argument: if you write code twice as fast but didn't also halve your maintenance cost, you haven't gained anything durable — you've traded a one-time speed boost for a permanent obligation. Velocity on the front end, debt on the back end.

Here's why the gift framing is actively dangerous: you book the speedup immediately and visibly (PR merged, feature shipped, manager happy), but you pay the interest later and diffusely (a 2am incident, a confusing module nobody can safely change, a security review that finds the thing six months on). The benefit is loud and the cost is quiet — so teams systematically over-borrow, because the books look like pure profit right up until they don't.

This is the same structural failure I keep running into in production AI systems generally: the win is the part everyone measures, the cost is the part nobody instruments until it bites.

The productivity-perception trap

There's a second number that pairs with the first, and it's the uncomfortable one.

METR ran a study in 2025 where experienced open-source developers did real tasks with and without AI. The developers believed AI sped them up by ~20%. Measured, the early result went the other way — they were slower, because the time saved typing got eaten by finding and fixing errors, steering the model, and waiting on it.

Now — I have to be fair about this, because the METR story is more nuanced than the headlines. METR's own February 2026 update walks the dramatic version back: they found heavy selection bias (when they tried to re-run it, 30–50% of devs refused to work without AI even for a paid study — itself a wild finding), and their newer, larger cohort showed roughly a -4% effect with a confidence interval spanning negative to positive. So the honest read isn't "AI makes you 19% slower." It's the softer, harder-to-dismiss version: the perceived speedup is consistently larger than the measured one. People feel 20% faster; the data says somewhere between "a little slower" and "a little faster."

That gap is the whole problem. If you feel twice as productive but you're roughly break-even, and meanwhile 44 cents on the dollar is leaking into rework — you will confidently make staffing, deadline, and architecture decisions based on a productivity gain that isn't there. The feeling is the interest rate you can't see.

So when is the loan worth taking?

Here's where I part ways with the doomer takes, because I use these tools every day and the answer is obviously not "stop." It's "borrow deliberately." The pattern I've landed on, watching where AI pays off versus where it quietly bills me:

Good loans (low interest, take them all day):

Throwaway and boilerplate — scaffolding, config, one-off scripts, glue code. There's no maintenance tail to pay back because the code barely has a future.
Code you'd have to look up anyway — the API you use twice a year, the regex, the bash incantation. AI replaces the doc-diving, not the thinking.
Stuff you can fully verify cheaply — pure functions with obvious tests, transformations where wrong is immediately visible.

Bad loans (the interest eats the principal):

Core domain logic you'll maintain for years — every line is a future obligation, and AI is happy to write code that looks right and is subtly, expensively wrong.
Anything security-sensitive — auth, input handling, anything touching secrets. The reported critical-bug skew is worst exactly here.
Code in a domain you don't understand well enough to review — if you can't catch the subtle wrong, you're not reviewing, you're rubber-stamping a loan you can't read the terms of.

The dividing line is brutally simple: how expensive is this code to be wrong, and how cheaply can I verify it's right? Cheap-to-verify, low-maintenance → free money, use AI aggressively. Expensive-to-be-wrong, long-lived → that's where the 44% lives, and where "I wrote it twice as fast" is a sentence you'll regret.

The one habit that changes the math

If I had to compress it to a single practice: measure the interest, not just the principal. Teams obsessively track AI's upside (lines generated, tickets closed, "tokenmaxxing" leaderboards — Amazon reportedly killed one internal leaderboard after people gamed it by burning tokens for the score). Almost nobody tracks the downside in the same ledger: what fraction of incidents trace to AI-written code, how much review time it consumes, how often it gets rewritten within N weeks.

Until you put both columns on the same page, every AI speedup looks like pure profit — for exactly the same reason a credit card feels like free money until the statement comes. The tool isn't the problem. Mistaking the loan for income is.

The speedup is real. Just don't spend it twice.

I'm genuinely curious where people land on this: in your experience, is AI a net productivity gain once you count the rework — or does the maintenance tail eat it? And has anyone actually put both columns in the same ledger? Would love to see real numbers in the comments, not vibes.

Sources

"Coders are refusing to work without AI — and that could come back to bite them," TechCrunch (May 2026).
"Developers won't work without AI anymore. The research says it might be making them worse," The Next Web (May 2026).
METR, "We are Changing our Developer Productivity Experiment Design" (Feb 2026) — the selection-bias update and revised effect size.
METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity" (Jul 2025) — the original perception-vs-measurement result.
CodeRabbit AI code-quality analysis, reported via Futurism/Yahoo (2026).

I distilled a 7B vision model into a 2B one for screenshots — and the 7B teacher scored worse

Sergei Parfenov — Tue, 02 Jun 2026 15:36:21 +0000

Code: https://github.com/P0rt/vlm-distill-screenshots
Model: https://huggingface.co/p00rt/qwen2-vl-2b-screenshots-distill

There's a question I keep coming back to whenever someone ships a giant model: what would I lose if I used something 3× smaller? Not in the abstract — for my task, on my hardware, with numbers I measured myself.

So I ran the experiment. I took a 7B vision‑language model (VLM), used it as a teacher to teach a 2B student one narrow skill — describing UI screenshots — and then measured exactly what the trade changed: quality, latency, throughput, memory. The whole thing runs on a single MacBook Pro (M4 Pro, 24 GB).

This post is the honest write‑up: the method, the numbers, and — maybe more useful — the three or four places where reality didn't cooperate.

TL;DR. The distilled 2B student runs ~2.4× faster, in ~2.4× less memory, with 3.75× fewer parameters than the 7B teacher, and it clearly beats the untrained 2B baseline on the task. The genuinely surprising part: on ROUGE‑L the 2B student scored higher than the 7B teacher — which is a story about the metric, not the models, and turned out to be the most interesting thing I learned. (It's also the live exception to "a student is bounded by its teacher" that I argued about in the comments of my last distillation post — on a narrow slice, the student really can pull ahead.)

Why distill a VLM for a narrow domain?

The obvious objection to this whole project: "Qwen2‑VL‑2B already exists and it's good — just use it."

True. But "a good general small VLM" and "a small VLM that's reliably good at the one thing you need" are different products. Distillation is how you turn the first into the second: you let a stronger model define the target behavior on your data distribution, and the small model adopts it — no manual labelling on your side.

And distilling a vision‑language model is less‑travelled territory than the classic "distill BERT into something tiny" story. It drags in real inference engineering — 4‑bit teachers, LoRA, quantized runtimes, memory budgets — and that engineering is half of why it's worth writing up.

The task I picked is deliberately narrow: screenshot understanding. Given a UI screenshot, produce a one‑sentence summary plus a list of the key interface elements. Perception only — no clicking, no agent. (That's future work; more at the end.)

The setup: task, data, metrics

Data — Screen2Words (rootsautomation/RICO-Screen2Words, CC‑BY‑4.0): 22,417 Android UI screenshots from the RICO corpus, each with five human‑written summaries. Native splits are train / val / test = 15,743 / 2,364 / 4,310, across 28 app categories.

One detail that matters more than it looks: the human captions are short — median 7 words. Hold that thought; it comes back to bite the metrics.

I picked the rootsautomation mirror specifically because it's CC‑BY‑4.0 — publishable, unlike raw RICO's research‑only terms. I check the license before I push weights, not after. (It's in the model card.)

Metrics. ROUGE‑L and BLEU against the human references, plus an optional teacher‑as‑judge score. I implemented ROUGE‑L and BLEU from scratch (pure Python, multi‑reference, unit‑tested) so the numbers are deterministic and dependency‑free. CIDEr — the classic captioning metric — needs corpus‑level document frequencies; I left it as a follow‑up rather than pull in a heavy dependency.

The pipeline, end to end:

download → build_dataset → teacher_label → train → eval → benchmark

Every stage is a typed CLI step, all hyperparameters live in configs/*.yaml, and the heavy steps (labelling, training) are resumable and versioned by a config hash. I built it phase by phase, one green‑CI PR at a time.

Method: three signals, one MVP

Knowledge distillation for generation has a few flavors, and I wanted the harness to support all of them behind flags so I could ablate cleanly:

Response‑based KD — the teacher generates answers, the student learns to reproduce them. The full objective mixes a soft and a hard target:

   L = α · CE(student, hard_labels)
       + (1 − α) · T² · KL( softmax(teacher/T) ‖ log_softmax(student/T) )

Feature‑based — align the student's vision features with the teacher's.
Self‑distillation — let the teacher label extra screenshots to grow the data.

The MVP I actually trained is the hard‑target half of (1): sequence‑level distillation. The teacher writes a description; the student is fine‑tuned (LoRA) to reproduce that text. No teacher logits needed at train time, which keeps the whole thing laptop‑friendly.

The soft‑KL term is implemented and unit‑tested (response_kd_loss), but wiring it into training needs cached teacher logits — and that's exactly the α/temperature ablation axis I couldn't run yet. I'd rather say that out loud than fake it.

Teacher labelling

The teacher is Qwen2‑VL‑7B‑Instruct in 4‑bit, running through MLX on Apple Silicon. (bitsandbytes is CUDA‑only, so the usual 4‑bit path doesn't exist on a Mac — MLX is the way in.)

It labelled 200 training screenshots at ~10.2 s/screenshot (≈34 minutes; ≈2.7 hours projected for the full 15.7k split). Zero outputs were flagged degenerate by a light post‑validation pass (whitespace normalization + empty/too‑short detection — cheap insurance against format drift), and the mean target length was 33.6 words. A real example:

"The UI screenshot shows a fitness app displaying an exercise called 'Lunges,' with a progress indicator showing 30% complete. Key interface elements include a progress bar, a figure performing the exercise, and the text 'Lunges.'"

Notice it's 33 words and genuinely rich. The human reference for screens like this is more like "exercise screen". That gap is the whole story of the metrics section below.

Training the student (and the part where MLX said no)

Plan A was to train the LoRA adapter with MLX too — same runtime as the teacher, fast on Apple Silicon. Plan A died in the backward pass:

ValueError: [Primitive::vjp] Not implemented for CustomKernel.

mlx‑vlm 0.6.0 can't backprop through one of Qwen2‑VL's custom Metal kernels. I checked the usual suspects — scaled_dot_product_attention, RMSNorm, RoPE all do have gradients — so it's a specific kernel, and both mlx and mlx-vlm were already on their latest release, so there was no version to bump to. MLX stays a great inference backend here; it just can't train this model yet.

Plan B: train on the hf path (transformers + PEFT LoRA) on Apple MPS, with PYTORCH_ENABLE_MPS_FALLBACK=1. That worked. Two more small potholes on the way:

The screenshots are too tall. Qwen2‑VL expands a big RICO screenshot into thousands of vision tokens; with a 1k context that overflows and throws a broadcast‑shape error deep in get_rope_index. Fix: cap the visual‑token budget (max_pixels) so an image stays well under the context window.
A papercut: recent transformers pulls in a Qwen2‑VL video processor that needs torchvision — which I hadn't installed. Easy to miss until the first run.

After that it trained cleanly: loss 0.80 → 0.39 over 40 steps, and — the part that matters for "is the checkpoint real" — I reloaded the merged adapter and it generated in the trained format ("…Key interface elements include…"). On a laptop.

Results: the honest version

First, the caveat that frames everything below, because it's load‑bearing: this is a deliberately small proof‑of‑concept. The quality numbers come from short training runs and a tiny eval set (80 train / 16–100 test examples depending on the run). Treat them as trends and a working method, not a benchmark. What I'm confident in is the harness and the measurement; the absolute numbers want a full‑scale run before anyone quotes them. With that said —

Here's the quality table on the test split (ROUGE‑L / BLEU vs the human references):

model	ROUGE‑L	BLEU
teacher (7B)	0.164	0.000 †
student (2B + LoRA)	0.178	0.019
baseline (2B, untrained)	0.153	0.018

† Teacher BLEU rounds to 0.000 — that's not a bug, it's the length mismatch explained right below.

Read that twice, because it surprised me too: the 7B teacher scores lower on ROUGE‑L than the 2B student, and its BLEU is essentially zero.

That's not the teacher being bad — it's the metric. The teacher writes 33‑word descriptions; the human references are 7 words. BLEU rewards exact n‑gram overlap, so a rich, correct, long answer against a terse reference scores ~0. ROUGE‑L (longest common subsequence) is kinder but still favors brevity‑matching. So against short references, all three models cluster in a narrow band and the verbose teacher actually looks worse.

The honest takeaways:

Distillation helped: the student (trained on teacher outputs) beats the untrained baseline, +16% relative ROUGE‑L. That's the comparison that's actually apples‑to‑apples (same model, same speed).
These metrics undersell rich outputs. This is exactly why LLM‑as‑judge and CIDEr exist, and why I flag the ROUGE‑L/BLEU numbers as a floor, not a verdict.
The clean, unambiguous win is efficiency — so let's go there.

The trade‑off (the actual point)

Same hardware, same 4‑bit setup, 128‑token generations:

model	params (B)	latency p50 (ms)	throughput (img/s)	peak mem (GB)
teacher (Qwen2‑VL‑7B)	8.29	1538	0.63	5.8
student (Qwen2‑VL‑2B)	2.21	651	1.52	2.4

~2.4× faster, in ~2.4× less memory, with 3.75× fewer parameters.

The student sits in the friendly corner: as fast and light as the untrained baseline, but with the distilled quality bump on top. The teacher is off to the slow, heavy side — and, per the metrics caveat above, not even ahead on ROUGE‑L. The 2B model is the one I'd actually deploy for this task.

⚠️ Honesty box. "Peak memory" on Apple Silicon is unified‑memory allocation, not CUDA VRAM. The headline efficiency numbers are MLX/4‑bit on Apple Silicon, not a server GPU. As flagged above, the quality numbers are a small proof‑of‑concept — trends, not a benchmark result. The thing I'm confident in is the method and the measurement harness — both reproducible from the repo.

Ablations: what actually moved quality

I varied the two knobs my sequence‑level SFT exposes — training steps and LoRA rank — at fixed everything‑else, and re‑evaluated each:

run	LoRA r	steps	ROUGE‑L	BLEU
baseline	8	0	0.152	0.017
—	8	40	0.170	0.018
—	8	80	0.172	0.020
—	16	40	0.171	0.019

"Train at all" is the dominant lever — the baseline → distilled jump is by far the biggest.
More steps help marginally, and the gain shows up more on BLEU (exact phrasing sharpens) than on ROUGE‑L.
LoRA rank is ~neutral here — r8 ≈ r16. At this data scale, adapter capacity isn't the bottleneck, so r8 is plenty.

The α/temperature/feature‑alignment ablations from the method section belong to the logit‑level KD variant, which needs cached teacher logits I haven't produced yet. Three honest comparisons beat six fabricated ones.

Inference engineering (the half that bites)

The modelling is the easy part. The engineering is where the hours went:

Two backends, one interface. The teacher runs 4‑bit via mlx-vlm; the student trains/infers via transformers + PEFT. A small factory (make_teacher / make_student) hides which one you're on.
MLX can't train this model yet (the CustomKernel vjp gap above) — so training is hf/MPS, inference can be either.
Don't mix runtimes in one process. Evaluating an MLX teacher and a torch student in the same Python process conflicts on Apple Silicon ('array' object has no attribute 'device'). Run them as separate invocations. Found that one the fun way.
Visual‑token budget is a real knob — too many tokens per screenshot and you blow the context window; cap max_pixels.
ONNX export of a full VLM is famously finicky, so I kept torch/MLX inference as the canonical path and shipped a merge‑and‑save export instead: fold the LoRA into the base weights and write a standalone 2B student you can load anywhere with plain transformers. (ONNX stays a documented stretch goal.)

What the metrics miss (and what I'd do next)

The most useful thing this project taught me wasn't a number — it was which numbers to distrust. ROUGE‑L and BLEU against terse human references genuinely undersell a model that writes richer, correct descriptions. If I were taking this past proof‑of‑concept, the very next step would be LLM‑as‑judge scoring (the harness already supports it) and CIDEr, both of which reward content over brevity‑matching.

Honest limitations: small scale (short training, tiny eval N); narrow domain (RICO Android UI); BLEU is low for everyone because of the length mismatch; and the headline efficiency numbers are MLX/4‑bit on Apple Silicon, not a server GPU.

Future work: cache teacher logits and turn on the soft‑KL term (and finally run the α/T ablation); add feature alignment; grow the data with teacher‑labelled RICO; a full‑scale run on a 24 GB GPU; and the natural next domain step — grounding (bounding boxes) and an agent wrapper.

Reproduce it yourself

git clone https://github.com/P0rt/vlm-distill-screenshots && cd vlm-distill-screenshots
uv sync --extra data            # data stack
uv run vlm-build-dataset        # Screen2Words → unified {image, prompt, target}

uv sync --extra mlx             # Apple Silicon teacher
uv run vlm-teacher-label --limit 200

uv sync --extra ml              # transformers + peft
PYTORCH_ENABLE_MPS_FALLBACK=1 uv run vlm-train --limit 200
uv run vlm-eval --models student,baseline --adapter results/checkpoints/<hash> --limit 100
uv run vlm-benchmark --models teacher,student

Everything — configs, the metric implementations, the plots, this article — is in the repo.

Your AI Agent Isn't Failing Because It Hallucinates — It's Failing Because of Rate Limits

Sergei Parfenov — Tue, 02 Jun 2026 13:09:00 +0000

When my agents started failing in production, I did what everyone does first: I went hunting for hallucinations. Better prompts, tighter output schemas, more guardrails. None of it moved the needle, because I was debugging the wrong layer. The agent's reasoning was fine. It was the plumbing that kept collapsing — and the single biggest culprit was the most boring thing imaginable: rate limits.

This turns out not to be just my problem. It's the dominant production failure mode for LLM applications right now, and almost nobody talks about it because it doesn't make for a good demo.

TL;DR — In production, the thing that takes your agent down usually isn't bad reasoning — it's capacity. Provider rate limits are now one of the largest sources of LLM call errors in real traces. A demo makes one request at a time; a production agent fans out into dozens of chained, retrying, concurrent calls and slams into limits the demo never touched. The fix isn't a smarter model, it's capacity engineering: budgeting, backpressure, retries with jitter, fallback models, and caching.

The data nobody puts in the pitch deck

Here's the number that reframed how I think about agent reliability. In Datadog's analysis of real LLM observability traces, rate-limit errors were a huge share of all LLM call failures — in March 2026, roughly a third of all LLM span errors were rate limits, on the order of millions of individual errors. Their conclusion was blunt: when the dominant failure mode of your LLM application is capacity, you need to redouble your capacity engineering, not your prompt engineering.

Sit with that. The failure mode isn't the model being dumb. It's the model provider saying "too many requests" — and your agent having no plan for that answer.

It maps almost perfectly onto the broader "agents fail in production" story everyone's writing about. The reason demos lie isn't malice; it's structural. A demo runs one clean request, one user, one happy path. Production is concurrency, retries, fan-out, and load — the exact conditions that manufacture rate-limit errors. The gap between "works in a notebook" and "works at 3am under load" is, more often than people admit, a capacity gap wearing a reliability costume.

Why agents hit this wall harder than chatbots

A plain chatbot makes one API call per user turn. An agent is a different beast. A single "task" expands into:

A planning call.
N tool-selection calls as it loops.
A call per tool result to decide the next step.
Retries on each of those when something is flaky.
Often a sub-agent or two, each with its own loop.

So one user action becomes 10–40 model calls, frequently concurrent, frequently retrying. The multiplier is the whole point of agents — and it's also exactly what walks you into a rate limit. Worse, the naive failure response makes it catastrophic: a call gets a 429, the framework retries immediately, that retry also gets a 429, and now you've turned one rate-limit error into a retry storm that takes the whole task down.

The arithmetic is unforgiving once you write it out. Say your provider gives you 500 requests/minute. If each agent task fans out to ~20 model calls, then just 25 concurrent tasks saturate your entire quota — and that's before a single retry. Add naive immediate retries on the resulting 429s and you don't degrade gracefully, you spike straight through the ceiling. I've watched this pattern play out more than once, and every time the first instinct in the room is "the model is broken" — when the model never even ran.

This is also where serverless bites you specifically. On Cloud Run, a traffic spike spins up new instances happily — compute scales fine. But your LLM provider quota does not scale with your container count. So autoscaling does the worst possible thing: it lets more concurrent agents launch, each firing its call fan-out, all drawing from the same fixed provider quota, all hitting the ceiling at once. The platform that's supposed to absorb load becomes the thing that amplifies it into the rate limiter. It's a genuinely counterintuitive failure: the healthier your autoscaling looks on the compute dashboard, the harder you're hammering a quota that can't scale with it.

The capacity-engineering toolkit

None of the fixes are exotic. They're the same patterns distributed-systems people have used for decades — they just haven't migrated into most agent codebases yet, because the field grew up on prompt-craft, not ops. Here's what actually moved my reliability numbers.

1. Budget and backpressure, don't just retry

The instinct is to retry harder. The fix is to send less. Put a concurrency limiter (a semaphore / token bucket) in front of all outbound model calls so your app never exceeds your known provider quota in the first place. When the budget is full, queue — don't fire-and-retry. This single change does more than any retry tuning, because it prevents the storm instead of recovering from it.

import asyncio

# Cap concurrent in-flight calls below your provider's actual limit.
# Leave headroom — you are NOT the only caller against this quota.
sem = asyncio.Semaphore(8)

async def call_model(client, **kwargs):
    async with sem:
        return await client.messages.create(**kwargs)

2. Retry with exponential backoff and jitter

When you do retry, never retry immediately, and never retry in lockstep. Synchronized retries from many workers create a thundering herd that re-triggers the limit. Exponential backoff with random jitter spreads them out.

import asyncio, random

async def with_backoff(fn, max_retries=5, base=0.5):
    for attempt in range(max_retries):
        try:
            return await fn()
        except RateLimitError:
            if attempt == max_retries - 1:
                raise
            # exponential + full jitter
            delay = random.uniform(0, base * (2 ** attempt))
            await asyncio.sleep(delay)

Respect the Retry-After header if the provider sends one — it's telling you exactly how long to wait, which beats guessing.

3. Fallback model, not just failure

Tie this back to distillation thinking: you don't need your frontier model for every call. Route to a cheaper/secondary model (a different provider, or a smaller model on a separate quota) when the primary is rate-limited. A degraded answer beats a dead task, and you've spread load across two quota pools instead of hammering one. This is the same hybrid pattern as keeping a cheap student model for the easy 90% and falling back to an expensive teacher — just applied to availability instead of capability.

4. Cache aggressively

A surprising fraction of agent calls are near-duplicate: the same tool descriptions, the same system context, the same sub-queries across runs. Prompt/response caching and reusing provider-side prompt caching cuts the call volume that reaches the limiter at all. The cheapest rate-limit error is the request you never sent.

5. Make capacity observable

You can't engineer what you can't see. The reason rate limits blindside teams is that they show up as generic "agent failed" errors, not as a labeled capacity problem. Log the error class (429 vs timeout vs tool error), track your in-flight concurrency and your 429-rate as first-class metrics, and alert on them. The shift that mattered most for me was simply separating "the model was wrong" from "the provider said no" in the telemetry — until you do that, every failure looks like a reasoning bug, and you keep fixing the wrong layer.

The mental model shift

The thing I'd tell my past self: treat your LLM provider quota as a shared, finite, non-scaling resource — like a database connection pool, not like CPU. Compute scales elastically. Your token-per-minute and request-per-minute quotas do not. Once you internalize that, agent reliability stops looking like an AI problem and starts looking like a classic distributed-systems capacity problem — which is great news, because we already know how to solve those.

Smarter models won't save you here. A GPT-6 that reasons perfectly still returns 429 when you exceed your quota. The reliability frontier for agents in 2026 isn't intelligence — it's capacity engineering.

If you're running agents in production, I'm curious what your dominant failure mode actually is when you separate the error classes — reasoning, capacity, or tool integration? My money's increasingly on capacity. Tell me I'm wrong in the comments.

Sources & further reading

Datadog, "State of AI Engineering" (2026) — rate-limit errors as a dominant share of LLM call failures in production traces.
"Why AI Agents Fail in Production and How Engineering Teams Are Fixing It", C# Corner (2026).
"The AI Agent Reliability Gap in 2026", DEV Community.
"Why 88% of AI Agents Never Reach Production", Digital Applied (2026).

How Model Distillation Actually Works (and What the 'China Distilled Our Model' Headlines Really Mean)

Sergei Parfenov — Fri, 29 May 2026 12:11:12 +0000

Every few weeks a headline drops: "Chinese lab distilled a frontier model from OpenAI / Anthropic." Cue the comments — half the thread thinks distillation is a synonym for theft, the other half thinks it's some exotic Chinese trick.

Both are wrong. Distillation is one of the most boring, well-established techniques in deep learning, and the labs raising the alarms use it on their own models constantly. The actual controversy is narrower and more interesting than the headlines. Let's separate the engineering from the geopolitics.

What distillation actually is

Knowledge distillation trains a small student model to imitate a large teacher model. The classic framing comes from Hinton et al. (2015): instead of training the student only on ground-truth labels, you also train it to match the teacher's output distribution.

Why does that help? Because the teacher's full probability distribution carries far more information than the single correct answer. If a teacher classifies an image of a dog, it might output dog: 0.9, wolf: 0.08, cat: 0.001. That "dog and wolf are similar, cat is not" signal — Hinton called it dark knowledge — is exactly what a small model struggles to learn from hard labels alone.

There are two kinds of training signal:

Hard labels — the final answer (the token the teacher actually produced, or the ground-truth label).
Soft labels — the teacher's full probability distribution over outputs, usually its logits passed through a softmax.

The trick is temperature. You divide the logits by a temperature T > 1 before the softmax, which flattens the distribution and exposes those small-but-meaningful probabilities the student should learn from.

The loss is a blend of two terms: a standard cross-entropy against the real labels, and a KL-divergence pulling the student's softened distribution toward the teacher's.

import torch.nn.functional as F

def distillation_loss(student_logits, teacher_logits, labels, T=2.0, alpha=0.5):
    # 1. Standard loss: student vs ground truth (hard labels)
    hard_loss = F.cross_entropy(student_logits, labels)

    # 2. Distillation loss: student vs teacher's softened distribution (soft labels)
    soft_targets = F.softmax(teacher_logits / T, dim=-1)
    student_log_probs = F.log_softmax(student_logits / T, dim=-1)
    # T**2 keeps gradient magnitudes balanced when T > 1
    soft_loss = F.kl_div(student_log_probs, soft_targets, reduction="batchmean") * (T ** 2)

    return alpha * hard_loss + (1 - alpha) * soft_loss

For LLMs the same idea applies per token: the teacher's next-token distribution is the soft target. In practice teams mix hard and soft labels — recent work argues the gain from mixing comes less from "matching the teacher better" and more from reducing exposure bias (the train/inference distribution mismatch). The point: this is normal, published, peer-reviewed engineering.

And labs distill their own models all the time. The cheap, fast variant of a flagship model that you actually get to call in production? Very often a distilled student. Anthropic itself, in the middle of its own complaint about Chinese firms, acknowledged that AI companies routinely distill their own models to make smaller, cheaper versions.

Why distilling from a closed API is a different beast

Here's the part the headlines skip. Everything above assumes you have the teacher's logits — the raw output distribution. That's white-box distillation, and it requires access to the model's internals or at least its full probability outputs.

You do not get logits from a closed commercial API like Claude or GPT. You get text. That forces black-box (a.k.a. sequence-level) distillation:

Prompt the teacher with lots of inputs.
Collect its generated text outputs.
Build a synthetic dataset of (prompt → teacher answer) pairs.
Fine-tune your student on that dataset with supervised fine-tuning, often followed by RL.

You lose the dark knowledge in the soft labels, but it turns out you can get remarkably far just by training on a large, high-quality synthetic dataset generated by a strong teacher. This is exactly why "did model X learn from model Y's outputs?" is such a live and hard-to-prove question — the evidence isn't a stolen weights file, it's statistical fingerprints in behavior (a model that randomly claims to be ChatGPT, mirrors another model's quirks, etc.).

	White-box	Black-box (closed API)
Needs	Logits / weights	Just text outputs
Signal richness	High (full distribution)	Lower (final answers)
Feasible against a closed model?	No	Yes
What the China allegations are about	—	This one

So what are the actual allegations?

Strip the drama and here's the documented timeline:

Jan 2025 — After DeepSeek's R1 launch, OpenAI and Microsoft open an investigation into whether DeepSeek used ChatGPT outputs to train it. Users noticed R1 behaving suspiciously ChatGPT-like.
Feb 2026 — OpenAI sends a memo to the U.S. House Select Committee on China alleging DeepSeek used obfuscated third-party routers to access OpenAI models and programmatically extract outputs for distillation, in violation of its terms of service.
Feb 24, 2026 — Anthropic publicly accuses three Chinese firms — DeepSeek, Moonshot AI, and MiniMax — of coordinated "distillation attack" campaigns: flooding Claude with crafted prompts, allegedly via commercial proxy services running tens of thousands of accounts to sidestep Anthropic's China access restrictions.

Two things matter here, and most coverage gets them backwards:

These are allegations. The labs have not, as of writing, published the full underlying evidence, and the accused firms dispute or haven't confirmed them. Behavioral similarity is suggestive, not proof.
The dispute is not "distillation = bad." As one ethics researcher put it after Anthropic's statement, if Anthropic itself calls distillation legitimate and widespread, the controversy can't be the technique. It's two narrower things: unauthorized access (using proxies to evade geographic and account restrictions) and terms-of-service violations (most frontier APIs explicitly forbid using outputs to train a competing model). It's closer to a contract-and-access fight than an IP-theft slam dunk — and the legal status of "training on another model's outputs" is genuinely unsettled.

"How long does it take / how much does it cost?"

This is the question everyone asks, and the honest answer is: dramatically less than training from scratch — which is the entire economic motive — but precise figures for any specific alleged case are not public. Anyone quoting you an exact "they did it in N days for $M" is guessing.

What we can say structurally:

Pretraining a frontier model from scratch means a massive run on tens of thousands of high-end accelerators, plus the data pipeline and research iteration behind it.
Distillation collapses that timeline. The expensive part — discovering the capability — was already paid for by the teacher. The student's cost is roughly: generating a synthetic dataset (API calls + time) plus a comparatively cheap fine-tuning run. That's the asymmetry the U.S. labs are upset about: they spend billions to push the frontier, and a "free-rider" can chase it for a fraction.
This is also why DeepSeek's headline numbers were so contested. Its self-reported low training cost and modest hardware footprint were precisely what made rivals suspect a shortcut: it's much easier to hit those numbers if you bootstrapped from an already-trained Western teacher rather than doing all the discovery yourself.

So: distillation makes a strong-ish student fast and cheap. It does not let you leapfrog past the teacher — a student is generally capped by the teacher it learned from. You don't distill your way to the frontier; you distill your way to a cheap copy of someone else's.

Takeaways

Distillation is standard, published deep-learning practice. The labs complaining about it use it themselves.
White-box distillation needs logits; closed APIs only expose text, so distilling from Claude/GPT means black-box training on generated outputs.
The OpenAI and Anthropic allegations against DeepSeek, Moonshot, and MiniMax are about unauthorized access and ToS violations, not about distillation being inherently illegitimate — and they remain allegations.
The economic point is real: distillation is far cheaper than frontier pretraining, which is why it's a business and policy flashpoint. But a student is bounded by its teacher.

If you want the deep technical version of any of these — the math of temperature scaling, why mixing hard and soft labels beats either alone, or how behavioral fingerprinting tries to detect distillation — let me know in the comments.

Sources & further reading

OpenAI memo to the U.S. House Select Committee on China (Feb 2026) — reporting via Reuters and Rest of World.
"Anthropic joins OpenAI in flagging distillation campaigns by Chinese AI firms," CNBC, Feb 24, 2026.
Hinton, Vinyals, Dean, "Distilling the Knowledge in a Neural Network" (2015).
"Understanding LLM Distillation Techniques," MarkTechPost, 2026.
"The Bridge-Garden Dilemma in LLM Distillation," arXiv:2605.26246.
Winston & Strawn, "Is AI Distillation by DeepSeek IP Theft?" (analysis of the legal gray zone).

DEV Community: Sergei Parfenov

The Most Powerful Model on the Market Got Pulled by the Government in 3 Days. Is It Real, or a Hype Bubble?

What's genuinely new here

Where the bubble is

So what's actually true?

What to do if you build on this

You Fixed the Rate Limits. Now Your Agent Fails Quietly.

The trade you didn’t know you made

Two gates, not one

Per-call correctness: the three tags

The part single calls don’t prepare you for: trust must propagate

Making it observable (or it didn’t happen)

The takeaway

Sources & further reading

The Comments Got Good. That's How I Knew.

"Great post!" is dead. Meet the eco-comment.

My own thread, by the numbers

Is it just me? I swept 38 other posts.

I'd been calling this the wrong thing

Why this actually works (and why I couldn't just tell)

I modeled what it does to a thread

The line I actually care about isn't "bot vs. human"

What I'm going to do about my own blog

Limitations (read this before you @ me — if you're real)

The one-line version

Run it yourself

Your AI Coding Speedup Is a Loan, Not a Gift — and the Interest Is Coming Due

The number

Why "loan" is the right metaphor (and "gift" is the dangerous one)

The productivity-perception trap

So when is the loan worth taking?

The one habit that changes the math

Sources

I distilled a 7B vision model into a 2B one for screenshots — and the 7B teacher scored worse

Why distill a VLM for a narrow domain?

The setup: task, data, metrics

Method: three signals, one MVP

Teacher labelling

Training the student (and the part where MLX said no)

Results: the honest version

The trade‑off (the actual point)

Ablations: what actually moved quality

Inference engineering (the half that bites)

What the metrics miss (and what I'd do next)

Reproduce it yourself

Links

Your AI Agent Isn't Failing Because It Hallucinates — It's Failing Because of Rate Limits

The data nobody puts in the pitch deck

Why agents hit this wall harder than chatbots

The capacity-engineering toolkit

1. Budget and backpressure, don't just retry

2. Retry with exponential backoff and jitter

3. Fallback model, not just failure

4. Cache aggressively

5. Make capacity observable

The mental model shift

Sources & further reading

How Model Distillation Actually Works (and What the 'China Distilled Our Model' Headlines Really Mean)

What distillation actually is

Why distilling from a closed API is a different beast

So what are the actual allegations?

"How long does it take / how much does it cost?"

Takeaways

Sources & further reading