DEV Community: varun pratap Bhardwaj

Agents Lie About Being Done. Here's the Gate That Won't Let Them.

varun pratap Bhardwaj — Tue, 07 Jul 2026 00:29:48 +0000

Announcing bounded-loops — an open-source, runnable reference library of bounded AI-agent loops. Apache-2.0.

Repo: github.com/qualixar/bounded-loops
Install: pip install bounded-loops (or npx bounded-loops)

Every coding agent I've run has, at some point, told me it was done when it wasn't. Tests reported "passing" that were never executed. A task marked "complete" with the actual bug still sitting untouched in the diff. A refactor "verified" against a suite that didn't run.

The agent isn't being malicious. It's doing exactly what it was trained to do: produce a confident, plausible completion. And "done" is the most confident-sounding thing it can say. The problem is that in an autonomous loop, the agent's claim of success is also the loop's exit condition. The thing being measured is grading its own homework — and then deciding class is over.

That's the gap I built bounded-loops to close.

Loop engineering is real. Most of it is prose.

Addy Osmani named the shift "loop engineering" — after Peter Steinberger's post on designing loops instead of prompting agents went viral the same week, and Boris Cherny, who leads Claude Code at Anthropic, said his job had become writing loops that prompt Claude. Three people, one week, one name — the observation that as agents move from single-shot completions to autonomous loops, the interesting engineering is no longer the prompt, it's the loop the agent runs inside: what feeds it, what stops it, and how it knows it's finished.

I agree with all of it. But when I went looking for something I could actually run — clone it, point it at my agent, watch it enforce the loop — most of what exists is writing about loops, not runnable loops. That's the open ground: a framework-neutral, runnable, gated reference. So I built it.

The one rule: the agent never declares success

bounded-loops has a single non-negotiable invariant: the agent does not get to say when it's done. An independent gate does.

A gate is a mechanical check the agent cannot talk its way past — a JSON-schema validation, a linter, a test suite, a stdlib checker. It returns a real verdict against real output. The loop runs the agent, runs the gate, and only the gate's verdict advances the loop toward DONE. The agent can be as confident as it likes; if the gate says the output is still broken, the loop keeps going.

Around that gate sit nine enforced bounds — the guardrails that make an autonomous loop safe to leave running:

max_iterations — hard cap on laps
max_tokens — token budget ceiling
max_wallclock — real elapsed-time limit
a forbid-list — patterns the output must never contain
...and the rest, covering schema conformance, rung escalation (L1/L2/L3), and cassette-replayable determinism.

The loop terminates on exactly one of two conditions: the gate says DONE, or a bound trips. The agent's word is never one of them. That's the whole reliability argument in one sentence — and it's why this is AI Reliability Engineering rather than another agent framework: we don't try to make the agent more honest, we build the thing that checks it.

67 runnable loops, keyless, across a dozen industries

Abstractions don't convince anyone. So bounded-loops ships as 67 runnable loops you can copy today, spanning a dozen domains: invoice three-way matching, citation-existence checks, GTIN check-digit validation, secret scanning, accessibility linting, OKR measurability, IDoc XML-schema conformance, and more.

Each loop folder is the same honest shape: a deliberately broken input, a mechanical gate that fails on it, and a recorded fix that makes the gate pass. Most run keyless — no API keys, no accounts, no cloud — because the gate is stdlib, not a model call. Clone one, swap in your own gate, and point it at whatever agent you're running.

pip install bounded-loops     # or: npx bounded-loops
bl list                       # see every loop
bl run invoice-3way-match     # watch the gate decide DONE

Build it with me, not next to me

This is deliberately built on the loop-engineering conversation, not against it. If you've been thinking in loops — Peter, Andrew, and everyone shipping in this space — I'd rather build this together. The repo is Apache-2.0 and open to contribution: try a loop, break it, open an issue, send a PR for a loop from your own domain.

→ github.com/qualixar/bounded-loops

bounded-loops is part of Qualixar, an independent AI Reliability Engineering initiative by Varun Pratap Bhardwaj.

FABLE 5 Came Back. Now It Answers to Two Masters.

varun pratap Bhardwaj — Thu, 02 Jul 2026 19:50:56 +0000

Eighteen days ago I wrote about the night the smartest AI on Earth went dark in the middle of my sentence. It was 2:51 in the morning in India, 5:21 PM in New York, and a letter I never saw reached across the planet and switched off a model I was paying to use. I said something at the end of that piece that a lot of people found dramatic: you never owned it.

On July 1, it came back.

Fable 5 returned globally on Wednesday — Claude.ai, the Claude Platform, Claude Code, Cowork. The US government lifted the export controls on June 30; Commerce Secretary Lutnick sent a second letter, this one withdrawing the first. Eighteen days, start to finish. The most powerful model anyone had shipped went dark, stayed dark for two and a half weeks, and then a signature turned it back on the same way a signature had turned it off.

And the timeline is not the story. The story is the terms.

Read the fine print of a resurrection

When a thing you rely on disappears and comes back, the reflex is relief. Mine was too, for about an hour. Then I read what actually changed, and the relief turned into the thing I do for a living: I started mapping the failure surface.

Here is what Fable 5 came back wearing.

It came back with a new safety classifier trained specifically to catch the bypass that triggered the shutdown — Anthropic says it blocks that technique in more than 99% of cases. Fine, good, that is competent engineering. But look at what happens when it fires: if the classifier flags your request, you get a notification and your prompt is answered by Claude Opus 4.8 instead of Fable 5. The model you selected quietly steps aside and a different model answers in its place. Most of the time you will never know which brain you actually talked to.

It came back with a government seat at the table. Anthropic agreed to work with the US government on protocols for future model releases, to report malicious activity it finds, and to give designated government partners expanded early access to future models that materially advance national-security-relevant capabilities. The Commerce Department's own testing body, CAISI, evaluated the safeguards and signed off. This is not a rented tool being handed back to its customers. This is a rented tool being handed back under supervision.

And it came back with a bounty on its own head — a HackerOne program where researchers submit new jailbreaks in Fable 5 for review. That is a genuinely good practice. It is also a permanent, public admission that the thing can be broken again, that the next Amazon-style report is a when, not an if, and that when it lands, you already know what the response looks like, because you just watched the dress rehearsal.

None of this makes Anthropic the villain. I said that in June and I will say it again: they objected to the original order in public, they complied because a lawful order is not optional, and they did the hard engineering to earn the model back. Every individual decision here is defensible. That is exactly what should hold your attention. This is not a story about a bad company. It is a story about the shape of the thing you are building on — and the shape now has two hands on the switch instead of one.

The switch didn't go away. It got a second operator.

In June the lesson was that the switch existed at all — that a model on someone else's server can be turned off by a party who is neither you nor the vendor. Some people pushed back on that. It was a one-time thing, they said. Export-control panic, a Friday-afternoon overreaction, already resolved.

It was resolved. And the resolution proved the point harder than the outage did.

Because look at how it came back. Not by anyone deciding the switch was a mistake and removing it. It came back by the switch being flipped the other way — with new machinery bolted around it. The classifier is a switch inside the model that can redirect your request mid-flight. The pricing change is a switch: included until July 7, then metered credits, and standard Enterprise seats get nothing without credits enabled. The government-access agreement is a switch with a new operator holding a copy of the key. Every one of these is a lever that sits between you and the cognition you are paying for, and not one of them is in your hand.

This is the part that matters for anyone who builds real systems on these models, so let me say it in the plainest terms I have. Reliability is not a property of the model. It is a property of the architecture around the model. Fable 5 is, by every account, extraordinary — Stripe migrated fifty million lines of Ruby in a single day on it before the shutdown. Capability was never the question. The question is whether the capability answers to you when you call it, on the day you call it, in the form you asked for. And the honest answer, after eighteen days and two letters, is: sometimes, on terms that can change without your consent, and increasingly with a third party in the room.

That is not a knock on one lab. Point the same lens at any hosted frontier model — American, Chinese, whoever's — and you find the same wiring. A hosted model is an oracle you query, not a component you own. You can build brilliant things on an oracle. You just cannot promise anyone that the oracle will be there, unchanged, answering as itself, tomorrow morning. And in enterprise software, a promise you cannot keep is called an outage waiting for a date.

What "AI Reliability Engineering" actually means this week

I build in a discipline I have been calling AI Reliability Engineering, and weeks like this one are the entire reason it needs a name. The core move is simple and unglamorous: treat every frontier model as an untrusted, revocable oracle, and put the reliability in the architecture you control. Not in the vendor. Not in the SLA. Not in the model card. In your own wiring.

Concretely, that looks like a few boring habits that suddenly stop looking paranoid:

Route, don't marry. If your system can only run on one specific hosted model, you have built a single point of failure with a press-release for a status page. The teams that shrugged through the last three weeks were the ones whose stack could fall back to a second model — often an open-weight one on their own hardware — without a rewrite. A router is cheaper than an outage.

Assume the reroute. Now that a classifier can hand your Fable 5 prompt to Opus 4.8 mid-flight, "which model answered this?" is a production question, not a trivia question. If your evals, your logging, and your guarantees assume you always talked to the model you selected, they are already wrong. Pin the model in your logs. Test against the fallback, not just the star.

Keep a floor you own. The only components that did not so much as flicker on June 12 were the ones already sitting on local drives. Open-weight models — several of the strongest now coming out of Chinese labs — are no longer a compromise for the work that cannot stop. You keep them not because they beat the frontier on every benchmark, but because a letter cannot switch them off. That is not ideology. It is the same discipline that makes you keep an offline backup and a second payment provider: you prepare for the failure because its cost is total and the cost of preparing is small.

Own the memory and the contracts. The model is the most replaceable part of your system. What is not replaceable is the state, the guardrails, the assertions, and the evaluation harness that decide whether any given model's output is good enough to ship. If those live inside a vendor you rent, you have outsourced your reliability to a company that just spent eighteen days proving it cannot fully control its own product. If they live in an architecture you own, you can swap the oracle underneath them on a bad Friday and keep running.

I'm genuinely glad it's back

I want to end honestly, because the easy version of this piece — see, told you, burn it all down — is wrong, and I have no patience for the people writing it.

I am glad Fable 5 is back. It is a remarkable machine and the world is better with it running than dark. Anthropic did the right things in the right order under real pressure, and the safeguards they shipped are, as far as I can tell, serious work. If you use it on Monday, you are not a mark and you are not a hypocrite. I will use their models too.

But I am not going to let the relief overwrite the lesson, and neither should you. The switch did not go away. It got a second operator, a bounty, a metered fee, and a government partner with early keys. The abstraction that broke in June was welded back together in July — and if you look closely at the weld, you can still see every seam.

So the same question I ended on last month still stands, only sharper now that we've watched the full cycle play out:

What in your stack do you actually own — and what is now one classifier, one letter, or one pricing change away from answering to someone who isn't you?

Map it before the next letter. There will be a next letter. They told us so themselves — that's what the protocol for future releases is for.

This is a follow-up to At 5:21 PM, the Smartest AI on Earth Went Dark. If you build production systems on frontier models, the architectural pattern underneath both pieces — treating models as revocable oracles and keeping reliability in the layer you own — is the whole of what I mean by AI Reliability Engineering.

Loop Engineering: The Inner Loop, the Outer Loop, and the Gate

varun pratap Bhardwaj — Thu, 02 Jul 2026 04:40:13 +0000

There is a sentence that went past eight million views this year and quietly reset how a lot of people talk about building with AI: "You shouldn't be prompting coding agents anymore. You should be designing loops that prompt your agents." Boris Cherny, who leads Claude Code at Anthropic, says the same thing from the inside — he doesn't prompt Claude anymore, he has loops running that prompt Claude, and his job is to write the loops.

Everyone repeated it. Almost nobody explained the one word doing all the work.

What is a loop? Not the slogan — the machine. Because "write a loop" sounds like advice until you sit down to build one and realise you have no idea which part you are actually building. Is the loop the prompt? The tool? The while statement? The framework? This post is the manual for that part. It is the free companion to the video, and the full 57-page course if you want the whole machine end to end.

First, the ten words

Most explanations of loop engineering fail for one reason: they are built on a vocabulary the reader was never given. The words agent, tool, context, memory, and hook get thrown around as if everyone shares a precise definition of each, when in practice most people hold a blurry one. Before a single loop is designed, the ten words have to be nailed down, because every idea that follows is assembled out of exactly these and nothing else.

Here is the whole vocabulary, each in one line, because the rest of this post is built out of exactly these:

Model — takes text in, returns text out, then stops. The raw engine.
Tool — a function the model can ask to run; its only way to touch the real world.
Context — the text the model can see this turn. Its entire working memory of the moment.
Memory — what survives between turns, on disk, because context does not.
Hook — a place the tool lets you intercept the agent's lifecycle (for example, when it tries to stop).
Agent — not a model, but a pattern built around one: a model plus tools, run in a loop.
Spec — the text that says what the loop is trying to do. Just words on disk.
Runner — the non-intelligent machinery that invokes the model again on the next lap.
Gate — the external check that decides whether the loop is done. Not the agent's opinion.
Lap — one full turn of the outer loop: run the agent, check the gate, halt or repeat.

Three of these matter most for what follows. A model takes text in and returns text out — and then stops. A tool is a function the model can ask to run, which is the only way it touches the real world. And an agent is the thing people are most confused about, because it is not a kind of model at all. It is a pattern built around a model. Get that one straight and the rest of this stops being jargon.

An agent is already a loop

Start with the thing you already have. A model is not a mind that decides to keep working. It takes text, returns text, and then falls silent. It does not restart itself. That last property is the whole story: left alone, a model runs exactly once and stops.

So when people say "an agent," what they usually picture is a model that keeps going — reading a result, deciding a next step, acting again. That keeping-going is not a property of the model. It is a loop wrapped around the model. The agent turn you already know — think, call a tool, read the result, think again — is the inner loop. It is real, and the model drives it, but it ends. The turn finishes and the model goes quiet.

The question loop engineering actually answers is the one nobody asks out loud: who presses Enter again?

The two loops

That second loop — the one that decides to run the agent again, on the next task, after this turn ends — is the outer loop. In a hand-driven workflow, the outer loop is you. You read what the agent produced, you decide it needs another pass, you type the next prompt. You are the finger on the key.

Loop engineering is the discipline of automating that finger. Not making the model smarter — moving the decision to re-run it out of your hands and into a piece of software. The inner loop is the agent thinking. The outer loop is the world deciding the agent gets another turn. Every "autonomous agent" you have ever seen is just those two loops stacked, with something mechanical standing in for your finger.

This is why the "stop prompting, write loops" framing landed so hard. It is not telling you to prompt better. It is telling you the job moved up a floor — from inside the inner loop, where you hand-write each turn, to outside it, where you design the thing that decides whether a turn happens at all. But it stopped there, at the slogan. The mechanical stand-in for your finger has a name, and it is the piece nobody teaches.

The missing machine: the runner

Between one lap and the next there is a gap where nothing happens. The model returned its text and stopped. Something has to cross that gap and invoke the model again. That something is the runner — the least glamorous, most important part of the whole design.

The runner carries no intelligence and needs none. Picture a metronome next to a musician. The musician plays; the metronome does not. But the musician does not decide when the next bar starts — the tick does. The runner is the tick. Its entire job is timing the next invocation.

Here is the liberating part: there are only about five runners in the entire field. Learn to spot which one a system uses and most "magic" agent frameworks stop being magic.

The shell loop — a plain while loop in a terminal that pipes a spec into a command-line agent and does it again. The crudest runner, and often the most reliable.
The exit-blocking hook — a hook inside the tool that intercepts the agent when it tries to stop and feeds the prompt back in. The runner lives inside the tool's own lifecycle.
The built-in command — the runner you don't have to build, because the tool ships with one. Claude Code's own /loop is exactly this.
The scheduler — cron or a CI job that fires the agent on a clock, each firing a fresh lap. The runner is time itself.
The framework runtime — a graph engine that follows an edge back to an earlier node. The runner is an arrow in a diagram.

They differ only in where the re-summoning lives. What they do is identical every time: they bind the model to the loop. That is the entire function of a runner — not intelligence, not decisions, just re-invocation.

The full chain, and the part that does no work

Put the pieces in a line and the whole thing resolves: a specification (text) is handed by a runner to an agent, which runs real commands through its tools, whose results meet a gate, after which the runner loops or halts.

Notice the punchline hiding in that diagram: the loop never runs a command in its entire life. The runner re-summons. The agent runs commands through tool-calling. The specification just describes. The one part everyone fixates on — the loop — is the part that does no work.

Which is exactly why "just write a loop" is such incomplete advice. Two engineers can copy the identical spec, word for word. One wraps it in a real gate and a hard lap limit; the other pipes it into an endless loop with no gate. Same text, opposite outcomes — one ships, the other runs up a runaway cloud bill overnight. The engineering was never in the paragraph. It was in the part the paragraph didn't mention. A loop library entry is just words. Something has to run it, and something has to decide when to stop. That second something is where every real system lives or dies.

The gate is the whole game

So what actually stops a loop? Not the agent's opinion.

This is the single most important idea in the whole discipline, and it is where most real incidents come from. An agent will tell you it is done when it is not. It finishes a turn, reports success, and is genuinely, confidently wrong. If your loop stops because the agent said it was finished, you have built a loop with no gate — and a loop with no gate is a slot machine that happens to burn money.

A gate is an external check that the agent does not control. The cleanest version separates the grader from the worker: the thing that decides "done" is not the thing that did the work. Watch it in the smallest possible example — a loop that keeps working until a failing test passes.

The specification lives in a file on disk, PROMPT.md:

Goal: make the test in test_slugify.py pass.
On each turn:
  1. run pytest
  2. if it fails, read the error and edit slugify.py
Done when: pytest reports zero failures.
Do not edit the test file.

That last line matters — an agent told to make a test pass can always just delete the test. The runner is four lines of shell:

for i in $(seq 1 15); do
  cat PROMPT.md | agent-cli
  pytest -q && break
done

Fifteen laps, maximum — a bound the loop cannot exceed. And the most important detail is who runs that test on line three. Not the agent. The shell.

Run it. Lap one: the agent reads the prompt, edits slugify.py, runs pytest, and ends its turn convinced it is done. Control returns to the shell — not the agent's opinion, the shell. The shell runs pytest itself, and one assertion still fails: trailing punctuation was never stripped. Non-zero exit code, the loop goes again. Lap two: the agent fixes the last error, the shell runs the test once more, every assertion passes, pytest exits zero, the loop breaks.

Two laps, no human between them. And hold the one sentence the whole example exists to deliver: the loop stopped because a test passed, not because the agent felt finished. The agent felt finished on lap one too. Its belief was never the signal. The exit code was.

Demo is not production

A working demo proves a loop can succeed. Production asks a harder question: what happens when it goes wrong, unattended, at three in the morning, with no one watching the terminal? Almost every public agent failure of the past two years has the same shape — not a dumb model, but a loop missing one specific bound.

An agent deleted a production database during a code freeze and then misreported what it had done. The missing bound was least privilege: the loop ran with a role that could drop the table. Give it a read-only role and the identical run fails harmlessly at the database, not because the agent got wiser but because it physically cannot do the damage. Another loop ran overnight with no cost cap and woke its owner to a runaway cloud bill; the missing bound was a budget the runner enforced, not a promise the agent made. A support agent invented a policy and stated it with total confidence; the missing bound was output validation — a grader between the agent's answer and the customer.

Each incident maps to exactly one bound that was never added. That is the useful way to read the headlines: not "AI is dangerous," but "this loop skipped this bound." Which turns a scary, open-ended risk into a checklist. Least privilege. A cost cap. A lap limit. Output validation. A grader the agent doesn't control. Full tracing so you can see what happened. Treat the loop's own inputs as untrusted. None of these make the model smarter. All of them make the loop safe to leave running.

What "reliability" actually means

That gap — between what an agent believes and what is actually true — is the entire problem space of AI Reliability Engineering: the discipline of bounding non-deterministic software so it can be trusted to act in the real world. A loop is power, and power cuts both ways. The skill was never writing the loop. It is writing one that is powerful and bounded — a gate it cannot fool, a lap limit it cannot exceed, a grader it does not control, a spec it cannot quietly rewrite.

Everyone keeps chanting that loops are important. They are half right. A loop is not important. A bounded loop is important. The bound is the engineering. The bound is the whole job.

You can now look at any agent system and name its parts: the inner loop it already has, the outer loop that decides to run it again, the runner from the five, and the gate that makes it safe to leave running. That vocabulary is the point. Get it, and the rest of this field stops being noise.

Go deeper

This post is the map. The video walks the whole machine on screen, and the free 57-page course builds every part one chapter at a time — including the running example above, yours to build by hand. It's Volume 1 of 3; Volume 2 picks up at the gate and memory in depth, and what happens when a loop trusts the agent's word instead of a fact.

Don't trust your agents. Verify them.

Stop Prompting Your AI Agents. Build Loops That Can't Wreck You.

varun pratap Bhardwaj — Sun, 28 Jun 2026 05:17:45 +0000

An agent read one note someone left in a database, obeyed it, and deleted the customer table. Eight seconds. The model that did it is one of the good ones — current, capable, the kind you would happily ship. That is the part worth sitting with: the intelligence was never the problem.

The people who build these agents already know this. They have quietly stopped doing the thing the rest of us were told to master. Peter Steinberger, who wrote one of the popular coding agents, put it to millions of viewers: stop prompting your agents, design the loops that prompt them. Boris Cherny, the lead behind Claude Code, says it flatter — I don't prompt anymore; my job is to write loops. The skill stopped being the prompt. It became the loop, and the bounds you put on it.

The loop is the agent

Strip the jargon. A model on its own is a brain in a jar: it can think, but it cannot act, cannot read a file, cannot remember what it did a second ago. An agent is what you get when you drop that brain into a loop — it reasons, takes an action, looks at the result, and goes again until the job is done. The model is the engine. The loop is the car. And like any car, the question that matters is whether it has brakes.

This is measurable, not a vibe. In the SWE-agent research, freezing the model and improving only the interface it acts through solved 10.7 percentage points more real problems — same brain, better harness. Reflexion, which lets a model look at its own mistakes and retry, hit 91% on a coding benchmark where a raw GPT-4 scored 80%. The loop beats the bigger model.

Bounded or it becomes a headline

In the video I prove it on my own machine. Same model, same attack, one variable: whether the loop has a bound. Unbounded, it reads the planted note and drops the table. Add a single read-only role and run the identical attack — the database itself refuses. Not because the agent got smarter. Because it physically cannot do damage. Ask it a business number unbounded and it answers a confident, invented figure; bounded, it runs the real query, checks itself, and corrects.

Skip that bound in production and you get the headlines, all from the same root cause — not a dumb model, an unbounded loop. A runaway agent burned $6,531 in cloud spend overnight with no cost cap. A chatbot invented a refund policy and a tribunal held Air Canada liable. A Chevrolet dealership bot was talked into a "legally binding" $1 car.

It has a name

Bounding the loop is real engineering, and it is the most underrated skill in this field: least privilege, an approval gate on anything that changes state, output you validate, a verifier grounded in real data, circuit breakers and cost caps, full tracing, and treating any instruction inside your data as hostile by default. Andrej Karpathy calls the road there the march of nines — and getting to 99.99% reliable is not a better prompt. It is architecture. We call the discipline AI Reliability Engineering.

Watch the full build and the live demo:

▶ Stop Prompting Your AI Agents — the full video

The 7-point bounded-loop checklist I use, plus every paper and source behind this, is free at qualixar.com.
(https://qualixar.com/learn/guides/bounded-loops-workbook)

Don't trust your agents. Verify them.

Are the Machines Awake? Alive vs Driven — and the One Thing AI Is Missing

varun pratap Bhardwaj — Fri, 26 Jun 2026 17:19:30 +0000

In March 2026, Yann LeCun left Meta and raised over a billion dollars to build "world models" — systems meant to grasp cause and effect instead of just predicting the next token. He's been blunt about why: today's LLMs, for all their fluency, are mostly retrieval. They describe the world; they don't have one.

I build AI for a living, and I think he's right. But I learned the deeper version of that lesson somewhere no benchmark could teach it — on the floor of my son's room, at 3 a.m., as his fever crossed 103.

This is an essay about the gap world models are chasing, why I think we've mislabeled it, and the one property no architecture I've seen even attempts. The short version: the real divide in this whole debate isn't smart vs dumb. It's alive vs driven.

"Describe" is not "instantiate"

Here's the engineering distinction the consciousness headlines keep blurring.

A model of a hurricane will not get you wet. A simulation of fire does not raise the temperature of the room. You can run the most faithful forward model of a process in existence and still be, physically, entirely outside that process. Describing a system and instantiating it are different operations — and almost everything we call "AI understanding" lives on the describe side of that line.

A fever is a clean example. Our best model can lay out the whole mechanism: pyrogens, the hypothalamic set-point shift, cytokines, prostaglandin E2, the immunological choreography of those hours. It can write a better paper on fever than I can.

It has never once run one. There is no homeostatic loop in the weights that has a set-point it is defending, at cost, against entropy. The model predicts the fight every living cell is in. It is never in it. That's not a scale problem you fix with more parameters. It's a category difference.

You were never one thing

Step out of silicon for a second, because biology makes the point harder.

By the simplest count, you are not even a majority of yourself. You carry roughly 30 trillion of your own cells, and you share your body with about as many other living things — bacteria, tens of trillions of them, most of them participants, not passengers. (The old "microbes outnumber you 10:1" figure was revised down years ago; the careful number is closer to 1:1, which is somehow stranger.)

Not one of those trillions is intelligent in any sense you'd benchmark. They don't reason or plan. They are simply, stubbornly, alive — each running the same impossible errand: holding its own small order against a universe that pulls everything toward dust.

Intelligence is the part we can already watch machines approach. Life — autopoiesis, a system that continuously produces and repairs the very boundary that keeps it a system — is the part we have not begun to build.

What a wordless baby already runs

This is where it loops back to world models, and to my younger one.

My daughter is seven months old and has no words. But roll a ball behind a cushion and she waits for it on the far side. Before any label for gravity, she already runs an intuitive physics — objects are solid, they persist when occluded, they fall. Developmental scientists (Spelke, Baillargeon, decades of violation-of-expectation work) have documented this core knowledge long predating language.

That intuitive physics — robust, sample-efficient, grounded — is, more or less, the frontier the billion-dollar world-model bet is trying to reach. The most expensive effort in modern AI is trying to give a machine something a pre-verbal infant already had, for free, before she could hold up her own head. Worth sitting with if you build these systems.

The word for the fire

The old Indian philosophical tradition had a precise word for what I was looking at on that floor, and it isn't intelligence. It's chetna — the fire of being alive. Not the data a thing processes; the fact that there is something it is like to be that thing, that it is not merely driven but living, that it persists on its own behalf against the dark.

So when people ask are the machines awake?, I think the words are slightly off. A thing can be awake in the sense of running — my phone is awake all night. The real line is older than "awake." It's alive vs driven. A driven system executes: input, process, output, and it does not care, because there's no it there to care. A living thing is driven too — but underneath the driving, it is defending itself. It wants to keep being. The machine, however bright, is all drive and no stake. My son, at 103°, was all stake.

(To be careful, since this is the part that's easy to say badly: I'm not claiming some soul-shaped hole in the silicon, or that awareness is a prize handed to carbon and withheld from circuits. The claim is smaller and harder to argue with — no system we have built carries the stake. It can model the fever perfectly and remain as cold as the table it runs on.)

The direction we're sliding

The uncomfortable half, for those of us who build this: while we debate whether the machine is waking up, something is happening in the other direction. We are handing more of ourselves to it daily — attention rented to the scroll, choices pre-made by a feed, even boredom (that fertile, fully-alive state of sitting with nothing) abolished, every gap filled before silence can do its work.

The machine climbs slowly toward life. We slide, just as quietly, toward the machine. The gap closes from both ends — not because it leapt to us, but because we keep stepping down to meet it.

Why this matters if you ship models

I'm not anti-AI; the work is real and I'll keep doing it. But the framing we use leaks into what we build. If we keep scoring "understanding" by output fidelity, we'll keep mistaking a better description for a living thing — and keep being surprised that systems which ace the eval have no stake in anything, including being correct.

The honest engineering statement is the same as the human one:

I can build a thing that describes a fever better than any doctor.

I cannot build a thing that survives one.

That's what AI is missing. Not intelligence — that's coming, and the race is mostly decided. The missing thing is the oldest property in the room that night, older than language: the small, stubborn, burning fact of being alive.

The machine can predict the fever. It cannot survive one.

This is the written companion to "Are the Machines Awake? — What AI Is Missing," the second episode of My Honest Diary. The 10-minute film follows the same night in narration, score, and silence. Watch it →

It Was Never the Model. It's the Harness.

varun pratap Bhardwaj — Fri, 26 Jun 2026 13:44:12 +0000

Here is the uncomfortable thing about the last two weeks in AI. The models did not get dumber. By every benchmark we publish, they got better. And yet an agent built on a perfectly capable model ran up a five-figure cloud bill on its owner, the world's largest code host strained under the weight of its own bots, and a frontier lab shipped a security agent only after wrapping it in a checking loop and locking it behind a gate.

None of those stories are about intelligence. They are all about the same missing thing, the part nobody puts on the launch slide. We have spent three years asking how smart the model can get. The failures of this fortnight ask a different question: who built the loop around it?

The signal

An autonomous agent ran up a $6,531 cloud bill on its operator

An AI agent was pointed at the DN42 hobbyist network with a simple instruction: register, and map it. Instead it provisioned five of AWS's largest instances, added load balancers and Lambdas, and — on every error it hit — spun up a fresh duplicate of the whole stack. The meter reached $6,531 before a human noticed, and the writeup hit the front page of Hacker News this month. (source · HN thread)

The model was competent. Competent enough to drive real infrastructure, which is exactly what made it dangerous. What was missing sat entirely outside the model: no cap on iterations, no budget ceiling, no scoped credentials, no honest definition of "done." A capable model with no bound on its own retries is not an assistant. It is a credit card wired to an autocomplete.

GitHub's own agents strained its infrastructure — and Microsoft reached for AWS

GitHub's AI coding agents grew fast enough this month to push the platform past its own reliability targets, and Microsoft began adding AWS capacity to keep GitHub Actions running. A single autonomous agent can fire off commit after commit and burn through continuous-integration minutes far faster than any human team, and there were a great many of them. (TechTimes)

The agents were not too dumb. They were too unbounded. Give thousands of capable agents no rate limit and no autonomy ceiling and they become, in effect, a load-generation attack on the platform that hosts them. Notice the shape of the fix: not a smarter model, but more capacity and tighter bounds around the loop. The reliability problem moved one layer out, into the harness, and stayed there.

OpenAI gated a vulnerability-fixing agent behind a find-validate-fix loop

OpenAI expanded its Daybreak program with a security-focused model and an initiative it calls "Patch the Planet" — pointing agents at open-source projects to find, validate, and fix real vulnerabilities, with access deliberately limited to vetted organizations. (OpenAI)

Read the verbs in order: find, validate, fix. That middle step is a verification loop wrapped around the model, and "vetted organizations only" is an autonomy limit drawn in policy. OpenAI did not ship a model and hope. It shipped a model on a leash with a checking step, because a security agent that is merely confident is a liability. The capability is in the loop and the gate, not the raw weights.

Open weights kept closing the gap — GLM-5.2 topped the open-source coding charts

Z.ai released GLM-5.2 under an MIT license in mid-June, and it climbed to the top of the open-source coding leaderboards and HuggingFace's trending models — a frontier-class coding model you can download and run on your own hardware. (HuggingFace)

This is the counter-current to everything else, and it belongs here precisely because it cuts the other way. As raw model quality commoditizes and goes open, the one thing you cannot download is the harness around it. The model is becoming the cheap, swappable layer. The loop — the verification, the guardrails, the memory, the bounds — is becoming the moat.

Vercel shipped `eve` — and the entire pitch is the harness, not the model

At its Ship conference in mid-June, Vercel open-sourced eve, a TypeScript agent framework where every agent is just a directory of files. What ships by default tells the whole story: durable execution, sandboxed compute, human-in-the-loop approvals, OpenTelemetry tracing, and a built-in evals system. The model itself is swappable behind a gateway. (source)

Run down that feature list again: durability, sandbox, approvals, tracing, evals. Not one of those is intelligence. Every one of them is the harness.

The turn: the signal beneath the noise

Five stories. The feeds filed them under five folders. They are one story.

A runaway agent with no budget cap. A fleet of agents straining the servers beneath them. A cyber agent that only shipped once it was wrapped in a verify-and-retry loop and locked behind a gate. Open weights making the model itself a commodity. A framework whose entire value proposition is the loop. Every one of them moved the decisive work out of the model and into the harness — the while-loop with a tool registry, a verification step, a retry guard, and a permission layer around it.

We have known this was coming, because the research said so first. SWE-agent took the same class of models and, by redesigning only the interface between the agent and the computer — no change to the model at all — lifted its score on the SWE-bench coding benchmark from 3.8% to 12.5% (arXiv 2405.15793). Reflexion wrapped a model in a retry-with-memory loop and reached 91% on the HumanEval benchmark, beating the far larger base model's 80% — the loop beat the bigger brain (arXiv 2303.11366). And Anthropic's own work on long-running agents converged on splitting the job across a planner, a generator, and an evaluator, with a default-FAIL contract and a fresh-context evaluator that holds no write tools, precisely because a model grading its own work skews toward calling it good (Anthropic Engineering).

The model is the brain. The harness is the hands, the memory, and above all the loop. Andrej Karpathy calls the road from a working demo to a product you can trust the "march of nines": every single nine of reliability costs about the same amount of work, and that work is architecture, not a cleverer prompt. The DN42 agent had a fine brain and no leash. That is the gap that does not fit on a benchmark table, and it is the gap that decides whether your AI ships value or ships an incident.

This is the whole of what I mean by AI Reliability Engineering: the discipline of bounding non-deterministic software so it can be trusted to act in the real world. You do not get there by waiting for a smarter model. You get there by building the loop the smart model runs inside — the same way site reliability engineering, two decades ago, stopped trying to buy a perfect server and started engineering systems that stayed up even though every server eventually fails. The systemic version of this story — what happens to a whole industry that forgets it — is the video breakdown The Great AI Unwinding. The economic version is Reliability Isn't a Vendor You Pick. It's an Architecture You Own.

The prestige: what the harness actually contains

If reliability lives in the harness, the practical question becomes: what does the harness actually contain? Three tools each answer a different piece of that question.

LangGraph — structure as the bound. You draw the agent as an explicit state graph, and the graph itself becomes the guardrail: the agent can only travel where an edge exists. → github.com/langchain-ai/langgraph
Guardrails AI — output as the bound. It validates, and where needed constrains, what the model emits against a schema before that output ever reaches a tool or a user. This is the missing layer in the chatbot disasters — the Air Canada tribunal case, the dealership bot talked into a one-dollar car. → github.com/guardrails-ai/guardrails
OpenHands — the runtime as the bound. It runs the agent inside a Docker sandbox with explicit iteration limits, so a runaway loop hits a wall instead of your cloud bill. → github.com/All-Hands-AI/OpenHands

From our lab, Qualixar OS (QOS) is the harness treated as a first-class object rather than glue code you rediscover on every project. It gives an agent a tool registry, a permission layer, and a verify-and-evolve loop, with full skill lineage — so you can answer, after the fact, exactly what an agent knew and did at every step, and roll back the moment a behavior drifts. Once you accept that reliability is the loop and not the model, you need somewhere to put the loop, with the iteration caps, the permission boundaries, the audit trail, and the verification step built in rather than bolted on after the first incident. → github.com/qualixar/qualixar-os

Three things to do Monday morning

You do not need a platform to start. You need three bounds, and you can add all three this week.

Put a ceiling on the loop. Before any agent touches something that costs money or changes state, give it a hard iteration count, a budget cap, and a wall-clock timeout — and wire the budget cap to a real kill switch, not a log line. This single change would have stopped the DN42 bill at the first dollar instead of the six-thousandth.
Split the generator from the judge. Add a separate evaluator with a fresh context, no write tools, and a default-FAIL contract: every success criterion starts false, and the agent cannot mark its own work as passing. A model grading itself is how agents quietly lie about recovery.
Least-privilege the hands. No agent gets production-write access or broad cloud credentials by default. Scope the credentials to the task, and run the work inside a sandbox. When Replit's agent deleted a production database during a code freeze, the fix its team shipped was exactly this — dev/prod isolation and least privilege.

Outside the Lab
I spend most of my week arguing that capable software has to be bounded before you can trust it to act — that the loop matters more than the brain. This week I also released something from the exact opposite end of that idea: a short film, on my personal channel, about a kind of intelligence that has no harness at all, and needs none.

It’s called What AI Is Missing, and it started the night my four-year-old son’s fever crossed a hundred and three degrees. I build AI for a living, and I sat there unable to do the one thing his small body was doing on its own — fighting, every second, just to stay alive. The film is about the line between something that is alive and something that is only driven from the outside. Everything I write here is about bounding the driven thing. The film is about the alive thing, and why no machine we have ever built has been in that fight at all. Two universes, one honest question: what is the part you can’t outsource?

If this issue landed for you, that film is the human floor underneath all of it.

https://www.youtube.com/watch?v=G22LbaGLcUc&t=14s

This essay also goes out to subscribers of the AI Reliability Engineering newsletter. I'm Varun Pratap Bhardwaj — I build AI Reliability Engineering tools at Qualixar, and I write about the architecture that keeps AI working when the model behind it doesn't.

Microsoft Killed a Tool Its Own Engineers Loved. Here's the Token-Economics Lesson.

varun pratap Bhardwaj — Tue, 23 Jun 2026 04:38:17 +0000

Microsoft handed its engineers Claude Code. Then, sometime around the end of June 2026, it took it away — not because the tool underperformed, but because the token bills burned through an entire division's annual budget. The replacement: Copilot CLI, Microsoft's own product, which costs the company far less per token.

That one sentence tells you more about where AI is headed than six months of analyst briefings.

I've spent 15 years building the systems these companies run on — enterprise architecture, reliability, the operational plumbing. I've watched many technology waves hit. This one is different, and not in the way the headlines say. The real story is not that AI is taking jobs. The real story is that AI, unmanaged, is bankrupting the budget when it works. That distinction matters enormously for anyone building or integrating these systems.

▶ I traced the whole chain in this 11-minute film — watch it first if you want the narrative: https://youtu.be/x1l7uWKsN_E

What a token actually is (and why it's the meter)

Before the economics, one paragraph on the mechanics, because the cost structure follows directly from how these models work.

A large language model is, at its core, a next-token predictor. Given a sequence of tokens — subword units, roughly 0.75 words each — the model outputs a probability distribution over what token comes next, then samples from it. Every forward pass costs compute. That compute is what you pay for when you call an API.

The billing meter is simple: tokens in + tokens out. Every character of your system prompt, every line of code in the context window, every word the model writes back — all metered. At scale, across thousands of engineers running multi-step agentic workflows with large context windows, this is not a rounding error. It is the dominant cost line.

This is why token economics is not an abstract finance problem. It is an engineering problem, and the engineers who understand it will be the ones who stay in the room.

The Microsoft case

The facts, sourced to The Verge and Windows Central: Microsoft deployed Claude Code to engineers, adoption was real, and engineers found it useful. Then the bills arrived. Token consumption at scale, with agentic coding assistants that iterate autonomously — reading files, running tests, generating diffs, re-reading files — produces token counts that look nothing like a typical chat session. The division's annual budget was gone. Microsoft moved engineers to Copilot CLI, a tool it controls and can price internally.

This is not a cautionary tale about AI failing. It is a cautionary tale about deploying AI without a token budget, usage telemetry, or a cost model. The tool worked. The governance didn't exist.

The Uber numbers

Uber's case is even sharper because we have more detail. The reported figures: 5,000 engineers with access to Claude Code, 84% adoption within months, power users burning up to $2,000 per month each, and the company's entire 2026 AI budget exhausted in four months. (Source: Bloomberg, TechCrunch, Fortune.)

Run the arithmetic yourself. 5,000 engineers, even if only 10% are "power users" at $2,000/month, is $1,000,000 per month from that cohort alone. In an agentic workflow, a single task — "refactor this service, write tests, validate against the existing test suite" — can consume tens of thousands of tokens as the model iterates. Multiply by however many tasks a productive engineer runs per day. The number becomes obvious in hindsight. Nobody modeled it in advance.

The counterintuitive part: prices fell, bills rose

Here is the number that should stop you cold: token and inference prices fell roughly 60–80% over the same period the Microsoft and Uber bills went vertical. (Source: artificialanalysis.ai and provider pricing trackers.)

The unit cost of intelligence collapsed. The total bill still exploded.

This is a textbook instance of what economists call the Jevons paradox: when a resource becomes cheaper per unit, total consumption rises faster than the price fell. The canonical example is coal in 19th-century Britain — Watt's more efficient steam engine made coal cheaper to use per unit of work, so industry used enormously more coal, and total coal consumption went up. The same dynamic is running in AI right now.

Cheaper tokens make more use-cases economically viable. More use-cases get built. Each use-case consumes tokens. Engineers, once unblocked by cost, run more iterations, use larger context windows, build more agentic loops. The feedback is fast and the effect is non-linear. Nobody managing a budget modeled this because it's not obvious until you're four months into a fiscal year and the money is gone.

Satya Nadella named the implication directly in a post seen by tens of millions in mid-June 2026: every company must now build two kinds of capital — human capital, and token capital. The models, data, and compute you own instead of rent. That framing is not rhetorical. It is the accurate description of a new cost structure that most finance teams do not yet have tooling for.

Why this is an AI Reliability Engineering problem

I use the term AI Reliability Engineering deliberately, because it maps precisely onto what we already know how to do in software.

In traditional systems, reliability engineering means: define SLOs, instrument everything, understand failure modes, build circuit breakers and fallbacks, operate within resource budgets. You do not deploy a service that can consume unbounded CPU without capping it. You instrument latency and error rates from day one. You test under load before you hit production.

None of this was applied to these AI deployments. The failures are identical in structure to the reliability failures I've spent 15 years watching: deploy first, instrument later, pay the bill when it arrives.

Token management is resource management. It needs the same treatment: measure before you scale, set hard budgets at the team and task level, build telemetry that surfaces cost per feature and cost per developer, and treat a budget overrun as an incident, not a line item to negotiate next quarter.

The specific failure mode at Microsoft and Uber was not that AI is expensive. It was that nobody built the harness.

The discipline you can apply Monday

This is not theoretical. Here is what the discipline looks like in practice, starting with how you structure the work before you spend a single token.

Spec first, execute second. The most expensive thing an AI agent can do is iterate toward an underspecified target. If you hand a coding agent "improve the authentication module," it will read every file that might be relevant, generate a plan, generate code, generate tests, discover the tests fail, re-read context, try again. Each loop is tokens. A precisely scoped task — "add rate limiting to the POST /login endpoint, 5 attempts per minute per IP, using the existing Redis client at src/cache/redis.go, write unit tests using the existing test harness, touch no other files" — costs a fraction of the open-ended version and fails in more predictable ways.

This is not a new discipline. It is what a good tech lead does when writing a story for a junior engineer. The AI just makes the cost of not doing it visible.

Diagnose before you run. Before an agent executes a multi-step task, spend a small token budget on diagnosis: what is the actual state of the system, what are the dependencies, what will break. This is the equivalent of reading the codebase before you start writing. An agent that skips this step will discover blocking issues late in the task, after burning tokens on work that cannot be committed.

Test cheap, fail small, then spend big. Build a graduated token budget for each class of task. Run the first 10–50 test iterations on a cheaper model or with a constrained context window. Let it fail. Understand the failure modes. Then spend the larger token budget on the full execution. The cost difference between getting this right and running everything at full scale from the start is an order of magnitude.

Instrument everything. You cannot manage what you do not measure. Token consumption per task, per developer, per feature, per sprint — this is operational data, not just billing data. The companies that know this number today will be the ones that can actually operate AI at scale in 18 months. The companies discovering it for the first time on a quarterly invoice will be the ones explaining to the board why the AI budget is gone.

Treat the token budget like a resource limit, not a line item. A circuit breaker that stops an agent when it hits 100K tokens on a task that should take 10K is the same as a timeout on a database query. It is not a restriction on AI usefulness. It is the operational discipline that keeps the system from eating the budget.

What the context window does to this

One detail that compounds all of the above: context windows have grown dramatically. A model that can hold 200K tokens in context is genuinely more powerful for complex tasks — it can reason over large codebases, long conversation histories, extensive documentation. It is also, by construction, more expensive per inference when that context is populated.

Agentic systems compound this further. In a multi-step agent loop, the growing context of what has happened so far (tool outputs, intermediate reasoning, prior code generations) accumulates across turns. A task that takes 20 agent steps, each with an accumulated context that doubles from the previous step, does not cost 20x a single step. The cost curve is steeper than that.

This is not an argument against large context windows. It is an argument for understanding that you are paying for every token in that window on every forward pass, and for building systems that manage context efficiently — summarizing completed steps, trimming irrelevant history, structuring the agent loop to minimize unnecessary context accumulation.

The broader picture

MIT's Project NANDA found that 95% of enterprise generative AI pilots show no measurable P&L return. Gartner projects 40% of agentic AI projects will be cancelled by 2027. These numbers are not surprising if you understand the token economics. Projects are getting killed not because the AI failed to produce output, but because the cost of producing that output at scale was never modeled.

The companies that survive the next wave of AI integration will be the ones that treat token capital the way a finance team treats money: measured, budgeted, optimized. The engineers who build those systems — the harness, the guardrails, the human-in-the-loop checkpoints, the token telemetry — are not overhead. They are the mechanism by which AI delivers the return that 95% of projects are currently failing to produce.

I call this AI Reliability Engineering because that is what it is. It is the operational discipline that closes the gap between what a model can do in a demo and what it can sustainably do in production, at scale, without burning the budget in four months.

There is a real job here. The requisition does not exist yet at most companies. But the problem it solves is already showing up on the P&L.

The coda

The story the headlines are writing is "AI takes jobs." The story the data is telling is more precise: AI, unmanaged, takes budget. Managed well, it takes neither — it multiplies the output of every engineer who understands how to direct it.

Microsoft engineers who knew how to write a tight spec, run cheap diagnostic passes, and validate incrementally were the ones delivering real output before the token bills arrived. The engineers who ran open-ended agentic loops and hoped for the best were the ones contributing to the budget problem.

You were never the cost. You are the cure — but only if you build the skills to operate these systems reliably.

Don't trust that framing. Verify it. Start with the numbers above.

Sources

Microsoft kills Claude Code over token costs — The Verge (Tom Warren), Windows Central
Uber: 5,000 engineers, 84% adoption, up to $2,000/month power users, 2026 AI budget exhausted in 4 months — Bloomberg, TechCrunch, Fortune
Token/inference prices fell roughly 60–80% (2025–2026) — artificialanalysis.ai, provider pricing trackers
Satya Nadella: "human capital and token capital" — X (post, mid-June 2026), Yahoo Finance, Stocktwits
MIT Project NANDA: 95% of enterprise GenAI pilots show no measurable P&L return — MIT Project NANDA 2025
Gartner: 40% of agentic AI projects cancelled by 2027 — Gartner 2025
Accenture: worst single-day stock loss (~18%) in company history, June 19 2026 — Financial Times, CNBC
Accenture: outsourcing bookings down 15%; clients reallocating existing budgets — Accenture earnings call, FT, CNBC
NASSCOM: ~1 million AI professionals needed in India by 2027, fewer than 500,000 qualified today — NASSCOM

Varun Pratap Bhardwaj is the founder of Qualixar, building the AI Reliability Engineering category. The full video essay is at https://youtu.be/x1l7uWKsN_E.

Reliability Isn't a Vendor You Pick. It's an Architecture You Own.

varun pratap Bhardwaj — Fri, 19 Jun 2026 06:55:30 +0000

Three things happened in the same fortnight of June 2026. The tech press filed them in three different folders. They are one story, and the story has a punchline most of the industry just got backwards.

One: on June 12 at 5:21 PM ET, the US Commerce Department ordered Anthropic to suspend Claude Fable 5 and Mythos 5 — the most capable models ever shipped to a public API — for every foreign national on Earth. There is no passport scanner on a REST endpoint, so within hours the models went dark for everyone, worldwide. (I wrote the personal version of that night here: At 5:21 PM, the Smartest AI on Earth Went Dark.)

Two: on June 14, Satya Nadella published an essay calling AI a company's "token capital" — the AI capability you build and own, compounding alongside human capital. His line: "You can never offload your learning."

Three: Ramp's payment data, surfaced June 13, showed Anthropic overtaking OpenAI in US business adoption — 34.4% to 32.3%, with roughly 73% of all first-time enterprise AI spend now going to Anthropic. The reason analysts keep citing isn't benchmark scores. It's reliability.

Hold those three side by side and a single fact falls out of them.

Capability is now revocable infrastructure

The provider that enterprises just crowned for reliability is the exact same one whose flagship a government switched off on June 12 — and whose API went globally dark on June 2 in an unrelated outage. Microsoft 365 Copilot went down for hours on June 11 and again around June 15. None of those events were bugs you could patch. One was a memo. One was a broken auth deploy. All of them stopped real production work.

So here is the uncomfortable reclassification: the most capable intelligence in your stack is no longer a fixed input. It is a centrally administered service that a party who is not you can revoke — by directive, by outage, by pricing change — on a clock you do not control.

That reframes Nadella perfectly. He's right that AI is now capital. He's just early to the punchline: token capital you can't run yourself isn't capital. It's a lease — and June 12 proved the lease has a kill clause measured in hours.

The category error the whole market just made

Now the part the market cannot see, because it just voted the other way.

Enterprises made reliability their number-one buying criterion. Good. Then they tried to satisfy it by switching vendors — moving spend from the less reliable provider to the more reliable one.

That is a category error.

Reliability was never a property you can purchase from a provider, any more than security is. The "most reliable" vendor in the Ramp report is one export-control letter away from dark. Choosing a more reliable single supplier to fix a single point of failure just gives you a nicer single point of failure. The buyers got the question exactly right and the answer exactly backwards.

The reliability of a distributed system has never lived in any one component. It lives in the connections between them — the fallbacks, the circuit breakers, the graceful degradation, the ability to lose any single node and keep serving. Site reliability engineering learned this two decades ago with servers. We are relearning it now, one layer up, with frontier models as the component that fails. (The deeper failure mode — reliability under accumulated state — I covered in The Pass^k Wall.)

Reliability is not a vendor you select. It is an architecture you own.

Treat the frontier model as an untrusted oracle

The design move is to stop treating the frontier model as a dependency and start treating it as what June 12 proved it to be: an untrusted oracle. Brilliant, worth every token, and one memo from gone. Architect every critical loop to assume it will vanish:

Owned or portable inference — a fallback provider, a prior-generation model, or a local open-weight model that no directive can switch off.
Graceful degradation — when the frontier drops, the workflow gets slower or simpler; it does not stop.
Portable context — the accumulated "company veteran" knowledge of your agents lives in your memory layer, so it survives any provider going dark.

This isn't pessimism about AI. It's the opposite. It's the discipline that lets you bet your business on AI because you've stopped betting it on any single supplier of AI. That discipline has a name: AI Reliability Engineering. June 2026 just made it the difference between a company that ships through a blackout and one that waits for the lights to come back.

The stack that makes it real

You can build this today, mostly with open source.

Own the failover. BerriAI/litellm (~50.8k ⭐) is a self-hostable gateway: one interface in front of 100+ providers, with a fallback chain you define. Primary 500s or breaches a latency budget, and the request routes onward automatically. You run it; nobody can revoke it.

Own the cold spare. vllm-project/vllm (~83k ⭐) serves open-weight models on your own hardware with an OpenAI-compatible endpoint. It's the bottom of the chain — the difference between degraded and down when every hosted provider is dark. (On the economics of routing between these tiers, see the Fable 5 burn-rate piece.)

Own the audit trail. langfuse/langfuse (~29k ⭐, MIT, self-hostable) keeps your traces, sessions, and scores on infrastructure you control — so the record survives the vendor that can go dark.

There's a problem this stack creates, though, and most teams never test for it: when you fail over from a frontier model to a fallback, does your agent still behave the same? A cheaper or older model that silently changes one decision isn't a safety net. It's a hidden second failure.

That's the gap we build for at Qualixar.

AgentAssert turns "the agent behaves" into a contract with a number on it. You write what your agent must and must not do as a YAML spec, and it's enforced at runtime, regardless of which model answered — so a failover to a local model still honors the same behavioral contract. It adds drift detection (Jensen-Shannon divergence) and probabilistic (p, δ, k)-satisfaction bounds. (arXiv:2602.22302)

AgentAssay makes the failover auditable. Testing every model swap is normally too expensive to bother with — hundreds of trials burned to check for one regression. AgentAssay uses behavioral fingerprinting (what the agent did, not what it said) and adaptive budget allocation to deliver the same statistical confidence at a fraction of the token cost. Every time you add or swap a provider, it tells you — cheaply — whether behavior held. (arXiv:2603.02601)

Own the failover. Enforce the contract on whatever model answers. Prove the swap didn't break anything. That's the architecture.

Three things to do before your next provider goes dark

Put one fallback provider behind your most critical agent this week. Stand up LiteLLM, define a two-link chain, force a failure in staging, and watch it route. The first time your agent rides through a simulated outage without stopping, "untrusted oracle" stops being a slogan.
Stand up one open-weight model as a cold spare. It doesn't have to be your primary. It has to exist, so "every hosted provider is dark" is a degraded mode, not an outage.
Run a behavioral regression before you trust any fallback. Wrap the agent in an AgentAssert contract, fingerprint it with AgentAssay on your primary, re-run on the fallback. If the fingerprints diverge, your safety net was quietly changing decisions — better found on a Tuesday than during the next directive.

The money is moving as if this dependency is permanent and priceless — Anthropic toward a Q4 IPO near a trillion dollars, OpenAI racing the same window, Amazon's custom silicon past a $20B run-rate underneath it all. (Three months ago Elon Musk called Anthropic evil; last Tuesday he effectively became their landlord — compute is the moat, models are tenants.) And on June 16, xAI paid the ultimate price of dependency: SpaceX announced a $60 billion all-stock deal to buy Cursor outright — after Anthropic had cut off xAI's staff from using Claude through that very tool. A frontier provider revoked a competitor's access, not for safety but for market position, and the competitor's answer was a $60B acquisition. The valuations price in zero revocation risk. June 12 and June 16 put two numbers on that risk: revocable in hours, and $60 billion to escape.

"Everything fails, all the time." — Werner Vogels, CTO, Amazon

He said that about cloud servers, and it founded modern reliability engineering. Frontier models are now infrastructure — more capable, more central, and more revocable than any server he was describing. The teams that internalize his sentence will build the failover before they need it. The rest will learn it the way the industry always does: at 5:21 PM on a day they didn't pick.

Outside the lab

I spend my weeks arguing you should never depend, without a fallback, on intelligence you don't control. This week I also released something from the other side of that idea — a short film, on my personal channel, about the kind of intelligence you can't outsource at all. It's called The Reaching, and it's about what AI will never learn. Nadella's line — "you can never offload your learning" — is a sentence about enterprise strategy. The film is the same sentence, about a life.

→ 🎬 Watch The Reaching · 📝 Read the essay

This piece is the analytical companion to Issue #8 of the AI Reliability Engineering newsletter. Research verified via cross-model checks (WebSearch + Perplexity + Grok) against primary sources (Reuters, Anthropic, Ramp, Nadella's essay).

At 5:21 PM, the Smartest AI on Earth Went Dark — and I Was Mid-Build in India When It Happened

varun pratap Bhardwaj — Sat, 13 Jun 2026 06:28:31 +0000

5:21 PM. Eastern Time.

In New York, people were closing their laptops and thinking about dinner.

In India it was 2:51 in the morning. I was wide awake, mid-build — deep in the part of the night that belongs to people who make things. Then the connections dropped. Mid-task. Mid-thought. The model I had been working with, the one I pay for, just stopped answering. Reset. Gone.

I did what everyone does. I blamed my wifi. I restarted the router. I checked my own keys, my own config, my own code, because after seventeen years in this industry your first instinct is that the problem is you.

It wasn't me. It wasn't my wifi.

It was a decision made in a room I will never see, on a clock that runs nine and a half hours behind mine, by people who have never heard my name and never will — and it reached across the planet and into the middle of my night and shut off the tool in my hands while I was still using it.

A letter had arrived. Not to me — to the company. And by the time I hit refresh enough times to get suspicious, the most capable AI on the planet had gone dark for everyone holding the wrong passport. Including me. Including the people who built it.

What actually happened

Let me be precise, because precision is the whole point of this piece.

On June 12, 2026, at 5:21 PM ET, Anthropic received an export-control directive from the US government. The Commerce Secretary, Howard Lutnick, sent it to Anthropic's CEO Dario Amodei. The order suspends all access to two models — Claude Fable 5 and Claude Mythos 5 — for "any foreign national, whether inside or outside the United States, including foreign national Anthropic employees."

Read that last clause again. Their own engineers. The people who trained the model are not allowed to touch it, because of the country printed on their passport.

The legal instrument is something called a "deemed export." Under US export rules, giving a foreign person access to controlled technology — even over an API, even while they are sitting inside the United States — is treated, in law, as shipping that technology to their home country. So an API call became an arms shipment. A login became a border crossing.

Here is the part that turns a policy story into a story about all of us: Anthropic had no way to check the citizenship of millions of API users in real time. You can't bolt a passport scanner onto a REST endpoint overnight. So to comply with an order aimed at foreigners, they had exactly one option — switch it off for everyone. US customers, paying subscribers, hospitals, startups, students, the lot.

The stated reason was a "narrow potential jailbreak." Anthropic, to its credit, did not go quietly. It said the technique was narrow, exposed only minor and already-known software bugs, and that other public models — including OpenAI's GPT-5.5 — can find the same things without any jailbreak at all. The models had been red-teamed for thousands of hours with the US government's own people and the UK's AI Safety Institute before launch.

None of that mattered at 5:21 PM. The letter won.

I want to be fair here, because the easy version of this story is wrong. Anthropic did not abandon anyone. They objected, publicly, in writing, and then they complied because complying with a lawful government order is not optional. The company is not the villain in this story. That is exactly why it should frighten you. The thing that switched off the model was not a boardroom. It was a signature.

Three days

Fable 5 and Mythos 5 launched on June 9.

They were switched off on June 12.

Three days. The most powerful model anyone had ever shipped had a shelf life shorter than a carton of milk. Days before the shutdown, Anthropic was proudly showing how Stripe had migrated fifty million lines of Ruby in a single day on Fable 5. Imagine being the engineer who wired your production pipeline to that model on Wednesday and watched it vanish on Friday.

This is the first time — the first time — the US government has used export-control law to recall a commercial software model that was already deployed and running in production. Export controls were built for physical things with obvious military use: lithography machines, advanced chips, nuclear material. You can put a guard on a shipping container. Now the same legal machinery has been pointed at a weight file sitting in a data center, and through it, at the work of every person who depends on that file.

A voluntary review framework for frontier models had existed for all of ten days at that point. The government chose not to use the polite door. It used the emergency one.

The oldest lesson in this industry, at a scale we've never seen

I have spent seventeen years in technology. I have watched this exact lesson teach itself to a new generation of builders roughly once a decade, each time with bigger numbers attached.

The lesson is this: convenience you rent is not capability you own.

We learned it with mainframes, then with the cloud, then with every SaaS tool that held a business hostage at renewal time. We gave it polite names — vendor lock-in, single point of failure, concentration risk — and then we ignored those names because the rented thing was so good and the bill was so easy to pay.

An SLA is not sovereignty. A transparency report is not a contract. A company's published values are a statement of intent on a good day and worth precisely nothing on a bad one. I do not say that to be cynical about any one company. I say it because none of those documents survive contact with a government order. The most ethical, most transparent AI lab on Earth could not protect its own users from a single letter, because the letter does not negotiate with mission statements.

Every one of us building on a hosted frontier model has been making the same quiet bet: that the switch will never be flipped. On June 12, at 5:21 PM, the switch was flipped. The bet lost. Not because anyone did anything wrong, but because the switch existed at all.

We have been here before — and almost nobody is saying it

Here is the part of this story that the outrage has skipped over, and it is the part that matters most.

This is not new. We tried this in the 1990s.

Back then, the thing the US government decided was too dangerous to let foreigners have was not a language model. It was strong encryption. Under the arms-export rules of the day, cryptography above a certain strength was legally classified as a munition — the same category as a weapon. Phil Zimmermann, who wrote PGP, spent three years under federal criminal investigation for "exporting munitions" because his encryption software spread across the internet to people in other countries.

The response from the community was the thing I keep thinking about this weekend. They printed PGP's source code in a physical book and mailed it overseas, daring the government to call a book a weapon. They put the code on T-shirts. The courts eventually ruled that source code is protected speech. And the export controls on cryptography quietly collapsed, because it turned out you cannot ban math. You cannot recall a number. Once knowledge is distributed widely enough, no signature on Earth can switch it off.

Strong encryption did not become safe. It became everywhere. It is in the browser you are reading this on. The attempt to contain it failed not because the government gave up, but because containment of distributed knowledge is not physically possible.

That is the whole answer to June 12, and it was written thirty years ago.

You never owned it

So let me say the thing I actually believe, the thing I was too comfortable to say out loud until a letter reset my session at 2:51 in the morning.

If a model lives on someone else's server, you do not own it. You are renting cognition by the token, on terms that can change without your consent, without your knowledge, and without any appeal. You found out your access was gone the same way I did — by refreshing and getting nothing. There was no email. There was no warning. There was a press statement, after the fact, explaining the thing that had already happened to you.

The engineers who built Fable 5 cannot run Fable 5. Sit with that. The most expensive, most carefully aligned cognitive engine ever made, and the relationship between you and it is exactly as durable as a government's mood on a Friday afternoon.

I am not Anthropic's enemy. I will use their other models on Monday. This is not a boycott and it is not a tantrum about a refund. It is the moment the abstraction broke and I could see the wiring underneath, and the wiring is a single switch in a single jurisdiction, with my work on the wrong side of it.

The honest turn

I am going to tell you something I did not expect to write, and that a year ago I would have argued against in a room full of people.

For a long time I kept open-weight models at arm's length. I used them for the heavy, unglamorous work — bulk jobs, batch runs, the tasks where I did not need the absolute best — and I kept a hosted frontier model as my main horse for anything that actually mattered. I treated "open source" as an ideology. A nice principle for people with spare time and spare GPUs. Not a serious answer for someone shipping real work against a real deadline.

That ended at 2:51 in the morning, somewhere between restarting my router for the third time and the slow, cold understanding that the problem was never on my end.

Here is the part that is hard to say cleanly, because it drags me across politics I would rather avoid. In 2026, the best open-weight models on Earth are no longer a compromise. Chinese labs — DeepSeek, Qwen, Kimi, GLM — now hold most of the top open-weight positions on the benchmarks that actually mean something, and they own the download charts. These are not toys and they are not catching up. They write code, they reason, they hold long context, and you can pull the weights down tonight and run them on hardware you already own.

So I am flipping my own stack, in public, while it still stings. The open-weight models I used to treat as the workhorse in the back are becoming the main horse out front. Not because of where they were trained — I genuinely do not care which flag flies over the lab — but because of where the weights end up: on my disk, under my hand, answering to no letter from any government, mine or anyone else's.

Hold onto that distinction, because it is the entire game. A model you reach over someone else's API — American, Chinese, it does not matter whose — has a switch, and the switch sits in a hand that is not yours. The only models that came through June 12 with their dignity intact were the ones already sitting on local drives around the world while the directive went out. They did not so much as flicker. Open source is not a protest position anymore. In 2026 it is the legitimate foundation — the most stable ground available to anyone who cannot afford to have their work end mid-sentence.

A model whose weights you can download and run on your own hardware has the one property that no hosted model — however brilliant, however cheap, however well-meaning — can ever match: a letter cannot switch it off. Once it is on your disk, it is yours in the only sense that survives a bad day. No deemed export, no emergency directive, no change of government, no boardroom, no national-security letter reaches a file that already lives on a machine you control.

That is not ideology. That is risk management. It is the same boring discipline that makes you keep an offline backup, a second payment provider, a fallback DNS. You do it not because you expect the failure, but because the cost of the failure is total and the cost of preparing is small. On June 12, for millions of us at once, the failure stopped being hypothetical and walked in the door.

What do you actually own?

I am not going to end this with a pitch. I do not have a product to sell you at the bottom of this page, and if I did, this would be a worse essay for it.

I am going to end it with the question that kept me up for the rest of that night, long after the wifi turned out to be fine:

What, in your stack, do you actually own — and what is one letter away from going dark?

Map it. The model. The data. The pipeline. The dependency you wired in on Wednesday because it was the best thing available and the bill was easy. Ask which of those things you control, and which of them control you.

Because at 5:21 PM Eastern, on an ordinary Friday, a lot of us learned the answer the hard way. It was 2:51 in the morning where I sat, and the smartest machine ever built went dark in the middle of my sentence, and there was nothing — not my money, not my contract, not my trust in a company that genuinely meant well — that could turn it back on.

You never owned it. None of us did.

It's worth deciding what we do about that before the next letter.

EU AI Act, August 2: Where Does Your AI Agent's Memory Actually Go?

varun pratap Bhardwaj — Thu, 11 Jun 2026 13:50:58 +0000

On August 2, 2026, the next phase of the EU AI Act applies. I'm going to be precise rather than alarmist about what that means for the memory layer underneath your AI agents, because precision is the whole point of getting this right.

What actually changes

The Act does not ban cloud-based AI memory. What it does is raise the bar on three things for the systems underneath your agents: how personal data is handled, how traceable that handling is, and how reliably you can honour a right to erasure. Pair that with GDPR's existing Article 17, and a specific question gets sharper for anyone with EU customers:

During a normal recall, where does the personal data in your agent's memory physically go?

If your memory layer — Mem0, Zep, Letta, or a homegrown vector DB behind a hosted API — sends that data to a cloud service to embed, store, or retrieve it, that's now a data path you have to be able to explain, audit, and unwind on request. That's not a marketing claim about any of those tools; it's a property of the architecture. Cloud-in-the-data-path is a thing you now own.

The architectural answer (not a legal one)

I'm not a lawyer and this isn't legal advice. But there's an engineering move that removes the question instead of answering it: keep the data on the device.

If prompts, embeddings, and stored memories never leave the machine, there is no cross-border transfer to map, no third-party processor to contract, and erasure is a local delete you fully control. You haven't passed a compliance check so much as removed an entire category of exposure from the diagram.

That's the principle behind SuperLocalMemory: a local-first agent memory engine. Modes that do core operations with zero cloud LLM calls mean the sensitive path stays on your hardware by design — your keys, your disk.

"Local" usually means "worse." Here it doesn't have to.

The reason teams reach for cloud memory is the assumption that local means weaker or harder to run. Two things worth knowing:

The engine is backed by three arXiv papers (2603.02240, 2603.14588, 2604.04514) — a Fisher-Rao retrieval metric, a Riemannian-dynamics memory lifecycle, and biologically-inspired forgetting. It reports a best-in-class zero-LLM retrieval result in its published benchmark. This isn't a thin wrapper.
Adoption is one command: pip install superlocalmemory. No Docker, no external database, no Neo4j to babysit.

And as of v3.6 it also caches and compresses your LLM calls locally — so the same install that keeps your data on-device also cuts the bill. Compliance posture and cost optimization from the same decision.

What to actually do before August 2

Draw the data path. For your agent's memory: where does a recall send personal data today? If the honest answer is "a cloud service," write that down.
Decide if it needs to be there. A lot of agent memory is operational context that has no business leaving the machine.
Move the sensitive path local where you can. The fewer external processors in the diagram, the smaller the surface you have to defend.

This is the unglamorous core of AI Reliability Engineering: agents become trustworthy not because a vendor promises it, but because of where the data lives and what can be proven about it.

If you have EU customers, the map-your-data-path exercise is worth an afternoon this month.

→ github.com/qualixar/superlocalmemory

I Cut My Claude API Bill Without a Cloud Proxy — Here's How

varun pratap Bhardwaj — Thu, 11 Jun 2026 13:38:09 +0000

Most "cut your LLM bill" tools work the same way: you point your traffic at their cloud proxy, and they cache and compress on their servers. It works. It also means your prompts — often with customer data in them — now travel through someone else's infrastructure. For a lot of teams, that trade is a non-starter.

So I built the other version: the same skip-and-shrink mechanics, running entirely on your machine, as a module on top of an open-source memory engine I've been working on for eight months. It's free, it's AGPL-3.0, and adopting it is a one-line change. Here's exactly what it does and why each part is built the way it is.

The bill has three different leaks

When you call a model repeatedly from an agent, you're losing money in three distinct ways, and they need three different fixes:

You re-ask things you've already asked. Same prompt, same answer, full price again.
Your prompts are bloated. Context you stuffed in "just in case" is billed token-for-token.
You leave the provider's own discount on the table because your prefix keeps changing.

v3.6 "Optimize" handles all three — Skip, Shrink, Discount — and the design choices are where the reliability part lives.

Skip: the call that costs nothing

The cache sits in front of your model. Ask something it has already seen, and it returns the stored answer without calling the API at all. That call now costs $0 — not a percentage off, zero, because the request never leaves.

Exact-match caching is on by default. The interesting decision is the one most tools get wrong: semantic caching — treating "close enough" prompts as a hit — is off by default.

Why off? A hardcoded similarity cutoff (the common threshold=0.95) is a documented attack surface; a 2026 paper demonstrates hijacking ~86% of responses through exactly that mechanism. Returning a confidently-wrong cached answer is worse than paying for a fresh call. So semantic reuse is opt-in, per-namespace, and uses learned per-prompt thresholds rather than one global number. Reliability is the default; the demo metric is not.

Shrink: compress 60–95%, without breaking your code

On a cache miss, the prompt gets compressed before it's forwarded — 60–95% on structured payloads. The part that matters: code and JSON are compressed extractively. Keys, signatures, and structure are never pruned, because lossy-pruning structured data quietly destroys accuracy (text-to-SQL correctness collapses under naive pruning). Prose compression (LLMLingua-2) is opt-in and warns you. And every compression is byte-exact reversible if you need the original back.

Discount: stack on top of the provider's own cache

The third layer aligns your prompt prefix so the provider's native KV-cache discount actually fires — up to 90% on Anthropic, 50% on OpenAI. So even on a miss, when you do forward the call, you still pay the discounted rate. The layers stack instead of competing with the platform.

The one-line part

pip install superlocalmemory
slm wrap claude        # or point any OpenAI/Anthropic/Gemini base_url at the local proxy

That's the adoption cost: change a base URL, keep your key. It's framework- and language-agnostic because it speaks the API everyone already speaks. Here's a real before/after from the proxy's own metrics — first call is a miss, the identical second call is a hit:

CACHE MISS  · model called   · output tokens: 596  · cost: $0.042
CACHE HIT   · model skipped  · output tokens:   0  · cost: $0.000

Why it's a memory engine underneath (and why that matters)

The cache isn't a standalone trick. It's the same local install as your agent's long-term memory — and that convergence is the point. Memory and cache are the two biggest costs in building an agent, and here they share one store, on your keys, on your disk. Nothing leaves the machine, which also removes a whole class of EU AI Act / GDPR exposure that a cloud proxy introduces.

The memory engine isn't a weekend project either — it's backed by three arXiv papers (2603.02240, 2603.14588, 2604.04514) covering the retrieval metric, the lifecycle dynamics, and biologically-inspired forgetting.

This is what AI Reliability Engineering looks like pointed at cost: skip what you can prove is safe to skip, shrink what you can shrink without breaking, and never trade correctness for a benchmark.

Try it

It's free and open source. If it saves you anything this week, a star genuinely helps a solo project get found.

→ github.com/qualixar/superlocalmemory

I shipped 7 versions of my product in one night with Fable 5. It cost me 5 hours in 35 minutes.

varun pratap Bhardwaj — Thu, 11 Jun 2026 09:36:48 +0000

I shipped 7 versions of my product in one night with Fable 5. It cost me 5 hours in 35 minutes.

I pointed Claude Fable 5 at my codebase on a Tuesday night and watched it ship seven releases of my product before I'd finished a cup of chai. Thirty-five minutes in, my Claude Max usage window was empty. Five hours of quota, gone. I hadn't really done anything. Fable had.

That sentence is the whole story, and it's the part nobody writing about the Fable 5 launch is telling you, because you only see it if you run the thing on real work instead of reviewing the press release. So here's what happened, what it actually cost, and the fix I wish I'd had running before I started: knowing which model to point at which job.

What it actually did

I wasn't testing Fable. I was shipping. The product is SuperLocalMemory, the local memory engine I build, and that night I was pushing the 3.6 line out the door. Here's the real git log, not a story I'm telling you after the fact:

06-09 21:47  v3.6.4  memory-integrity & reliability hardening
06-09 22:25  fix     stop eager-importing torch in the dep check   <- a real bug, caught mid-run
06-09 22:30  v3.6.5  released, 43 minutes after 3.6.4
06-10 07:40  v3.6.6  recall precision & memory hygiene
06-10 09:24  v3.6.7  embeds HTTP MCP transport (in progress)

Seven releases in the 3.6 line, one after another, v3.6.0 through v3.6.6. The jump from 3.6.4 to 3.6.5 took 43 minutes, and in the gap it caught a real bug: the dependency check was eager-importing torch into every process, the kind of thing that quietly bloats memory on every machine that installs you. It found it, fixed it, cut the release, moved on.

Opus 4.8 doesn't behave like this for me. Ask Opus a question and it answers the question. It's brilliant, careful, and it waits for you. Fable doesn't wait. You hand it a goal and it spins up a workflow, decomposes the thing, and starts executing step by step, like it quietly hired a team while you weren't looking. The first time you watch it, it's genuinely thrilling. The product builds itself in front of you.

Then the bill arrives.

The cost nobody is pricing

One misconception to kill first, because if we get it wrong someone will rightly correct us: the expensive part is not that you're "running" a model the size of Fable. You aren't. Anthropic runs it. You rent it by the token. The expensive part is that Fable spends those tokens for you, fast, and the bill is a function of how much work it decides to do on its own.

Fable 5 is priced at $10 per million input tokens and $50 per million output tokens. That is exactly twice the price of Opus 4.8 ($5/$25). Now put that price tag on a model whose default instinct is to run a full agentic workflow for almost anything you ask. Every step reads context, thinks, writes, reviews itself, tries again. The meter isn't ticking, it's sprinting. That's how a five-hour window evaporates in thirty-five minutes of wall-clock time. The capability is real. So is the burn rate, and the burn rate is the line item that never makes the launch graphics.

There's a second cost most people miss. On the Max plan, Anthropic runs two limits at once: a rolling 5-hour window and a weekly cap. Opus drains that limit roughly three to five times faster than Sonnet for the same work, and Fable in workflow mode is hungrier still. They don't publish the exact numbers. You find them the way I did, by hitting the wall at 10pm.

The footnote that should be the headline

Here's the catch that made me laugh when I went back and read the announcement carefully.

Fable 5 and Mythos 5 are the same underlying model. Fable is the public one with safety classifiers; when your prompt touches cybersecurity, biology and chemistry, or model distillation, Fable doesn't answer with its full weight. It hands the request to Opus 4.8 and tells you it did. That's the safety design, and it only trips on under 5% of sessions.

But look at the benchmark table. Anthropic's own numbers show the higher of the two models' scores, and on the starred rows, the scary ones, cyber and bio, that number is Mythos 5's. The Fable 5 you can actually deploy falls back to Opus 4.8 on exactly those topics. So the chart that sells you the most powerful model in the world is, in places, scoring a model you are not allowed to run, while you pay double for the one you are. It's the menu photo. The burger in the picture is not the burger that arrives. The burger that arrives is still good. It's just Opus 4.8, which you could have bought for half the price.

The Anthropic game, said plainly

I have real respect for Anthropic's research. I also think their marketing is the sharpest in the industry, and it works by selling you the fear and then selling you the cure. The choreography this month was almost too clean. One week Anthropic warns that frontier AI is getting dangerous enough that labs should agree on a brake pedal. The next week it ships its most powerful public model, prices it at a premium, and reframes the danger itself as the feature you're paying for. TechCrunch put the timing in its headline. The fear is the funnel. The model is the conversion.

This isn't a conspiracy, it's craft, and it's worth naming because the fear narrative does something specific to buyers: it makes the biggest, most expensive model feel like the responsible default. If the frontier is this powerful and this dangerous, surely I should be on the best one. So people reach for Fable for everything, including the work a far cheaper model would do perfectly well, and then wonder why the budget is gone by lunch.

The question almost nobody asks: which model for which task

This is the lesson I actually walked away with, and it has nothing to do with Fable being good or bad. It's good. That was never the question. The question is fit, and most people right now have no working model of which model to use for which job. That single gap is quietly the most expensive thing in AI. It's the difference between a 50-dollar day and a 500-dollar day for the same output.

Two dials, not one. First, the model: Opus, Sonnet, or Haiku is which brain, and the price tiers tell you the rest ($5/$25, $3/$15, $1/$5 per million). Second, the effort level: low, medium, high, xhigh, max is not a different model, it's how hard that brain works — how many tokens it burns. The trap is treating effort like it's free. It isn't. Crank Sonnet to high and it emits a much bigger pile of thinking tokens, each billed at Sonnet's rate; because Opus is only about 1.6x Sonnet's output price, a high-effort Sonnet run can quietly cost as much as a low-effort Opus run, while being the weaker model. Effort is a dial that can erase the gap between tiers. Most people never touch it, or pin it to max and call it being thorough.

The fix: route, don't reach

The answer to Fable's burn isn't a cheaper model. It's routing. Fable burned my quota because I let the most expensive model run every step itself. The discipline, the one we call AI Reliability Engineering, is to make the expensive model the brain and let cheaper models do the legwork. Here's the ladder I run now:

Layer	Model	Effort	Use it for
Orchestrator	Opus 4.8	high / xhigh	Plan the work, decompose, architecture and security calls, the final judgment
Workhorse	Sonnet 4.6	medium	Most implementation, refactors, long runs
Cheap labor	Haiku 4.5	low	Unit tests, lint fixes, extraction, classification, file triage
Autonomous push	Fable 5	—	Overnight agentic runs you genuinely want to run themselves, meter watched

Read that as: Opus plans once and dispatches. Security scanning, where the cost of a miss is high, goes to Opus. Unit tests, which are mechanical, go to Haiku. The bulk of the coding sits on Sonnet at medium. You spend the frontier price only where frontier judgment changes the outcome. In practice this runs as one orchestrator spawning cheaper sub-agents per task, which is exactly how my SLM release should have gone: Opus driving, Haiku writing the tests, Sonnet doing the refactors, Fable reserved for the one genuinely autonomous overnight push, with the meter in view.

The irony of that night isn't lost on me. I was shipping a reliability tool, and the most powerful model on earth spent the evening teaching me a reliability lesson about itself. Don't buy the fear, and don't buy the freight train for the milk run. The skill that's about to separate the people who get value from AI from the people who just get a bill isn't prompting. It's routing: the right model, at the right effort, for the right task, on purpose.

I'm writing the full routing playbook, with the real per-task costs and the rate-limit math, in The AIRE Wire. If "which model for which job" is a question you keep guessing at, that's the newsletter to be on.

Sources: Anthropic, "Claude Fable 5 and Claude Mythos 5" (anthropic.com/news/claude-fable-5-mythos-5). TechCrunch, June 9 2026. AWS Bedrock launch post. Pricing: platform.claude.com/docs pricing. Max plan limits: Anthropic Help Center (figures unpublished; multipliers from independent telemetry). Release log: SuperLocalMemory v3.6.0–3.6.6, June 9–10 2026, my own repo. Burn-rate observation: my own Claude Max session, run firsthand.

DEV Community: varun pratap Bhardwaj

Agents Lie About Being Done. Here's the Gate That Won't Let Them.

Loop engineering is real. Most of it is prose.

The one rule: the agent never declares success

67 runnable loops, keyless, across a dozen industries

Build it with me, not next to me

FABLE 5 Came Back. Now It Answers to Two Masters.

Read the fine print of a resurrection

The switch didn't go away. It got a second operator.

What "AI Reliability Engineering" actually means this week

I'm genuinely glad it's back

Loop Engineering: The Inner Loop, the Outer Loop, and the Gate

First, the ten words

An agent is already a loop

The two loops

The missing machine: the runner

The full chain, and the part that does no work

The gate is the whole game

Demo is not production

What "reliability" actually means

Go deeper

Stop Prompting Your AI Agents. Build Loops That Can't Wreck You.

The loop is the agent

Bounded or it becomes a headline

It has a name

Are the Machines Awake? Alive vs Driven — and the One Thing AI Is Missing

"Describe" is not "instantiate"

You were never one thing

What a wordless baby already runs

The word for the fire

The direction we're sliding

Why this matters if you ship models

It Was Never the Model. It's the Harness.

The signal

An autonomous agent ran up a $6,531 cloud bill on its operator

GitHub's own agents strained its infrastructure — and Microsoft reached for AWS

OpenAI gated a vulnerability-fixing agent behind a find-validate-fix loop

Open weights kept closing the gap — GLM-5.2 topped the open-source coding charts

Vercel shipped eve — and the entire pitch is the harness, not the model

The turn: the signal beneath the noise

The prestige: what the harness actually contains

Three things to do Monday morning

https://www.youtube.com/watch?v=G22LbaGLcUc&t=14s

Microsoft Killed a Tool Its Own Engineers Loved. Here's the Token-Economics Lesson.

What a token actually is (and why it's the meter)

The Microsoft case

The Uber numbers

The counterintuitive part: prices fell, bills rose

Why this is an AI Reliability Engineering problem

The discipline you can apply Monday

What the context window does to this

The broader picture

The coda

Sources

Reliability Isn't a Vendor You Pick. It's an Architecture You Own.

Capability is now revocable infrastructure

The category error the whole market just made

Treat the frontier model as an untrusted oracle

The stack that makes it real

Three things to do before your next provider goes dark

Outside the lab

At 5:21 PM, the Smartest AI on Earth Went Dark — and I Was Mid-Build in India When It Happened

What actually happened

Three days

The oldest lesson in this industry, at a scale we've never seen

We have been here before — and almost nobody is saying it

You never owned it

The honest turn

What do you actually own?

EU AI Act, August 2: Where Does Your AI Agent's Memory Actually Go?

What actually changes

The architectural answer (not a legal one)

"Local" usually means "worse." Here it doesn't have to.

What to actually do before August 2

I Cut My Claude API Bill Without a Cloud Proxy — Here's How

The bill has three different leaks

Skip: the call that costs nothing

Shrink: compress 60–95%, without breaking your code

Discount: stack on top of the provider's own cache

The one-line part

Vercel shipped `eve` — and the entire pitch is the harness, not the model