DEV Community: Michael Tuszynski

Context Is Four Different Words: The Window, The Corpus, The File, and The Session

Michael Tuszynski — Sun, 16 Aug 2026 16:34:16 +0000

Someone on your team says the agent "needs better context." Four people nod. They are agreeing about a word and disagreeing about the work, and nobody finds out until two of them have built the wrong thing.

I run four systems that all get called context, and they have almost nothing in common. One is a hard capacity limit. One is a 35-million-token corpus. One is a 4.7KB file I hand-edited last week. One evaporates when the session ends. Same word, four owners, four failure modes, four different afternoons of work.

Here is how I keep them apart.

Job one: context as capacity

This is context as a number. The window. It has a ceiling, you can exceed it, and when you do, something gets dropped or summarized whether you like it or not.

Capacity is the only one of the four you can measure without judgment, which is why it dominates conversation. It is also the least interesting, because being under the limit tells you nothing about whether the right things are in there.

My own number is worth stating plainly. Before I type a single word, my agent has already loaded a global instruction file, a shared cross-tool rules file, a workspace guide, a memory index, and a lessons file. Together that is about 75,000 characters, call it 19,000 tokens, spent every session on standing orders. That is my floor, not my usage.

Job two: context as corpus

This is context as a pile of things you could retrieve. Vector store, RAG index, semantic search. It is not in the window; it is the universe you draw from to fill the window.

Mine is Postgres with pgvector, embeddings served by a local model:

select sum(token_count), count(*), round(avg(token_count))
  from chunks where status = 'active';
-- 35,035,979 | 159,669 | 219

Thirty-five million tokens of retrievable material against a window that holds a fraction of a percent of it. The binding constraint was never storage. It is selection.

The breakdown is the part that changed how I think about this. Of those 159,669 chunks, session transcripts account for 99,259. Archived history from two prior systems adds another 51,165. The hand-curated knowledge vault — the notes I actually wrote on purpose — is 2,515 chunks, about 1.6% of the corpus. The rest is exhaust.

That ratio is not a bug I am about to fix. It is what a retrieval corpus looks like after eighteen months: mostly a record of what happened, with a thin seam of what you decided.

Job three: context as instruction

This is context as a durable file you hand-author and version. CLAUDE.md, AGENTS.md, cursor rules, system prompts checked into the repo. It loads every session, it is read by a machine but written by a person, and it goes stale silently.

The artifact:

$ ls -la ~/.codex/AGENTS.md
~/.codex/AGENTS.md -> /Users/mpt/.claude/AGENTS.core.md

One 4.7KB file of rules — never commit secrets, never deploy without asking, batch privileged steps into one paste-able block — symlinked so that two different vendors' coding agents read the identical text. I edit one file; Claude Code and Codex both change behavior. That is context as configuration, and it has more in common with a dotfile than with anything in jobs one or two.

Its failure mode is drift, not overflow. A stale rule is worse than a missing one, because the agent follows it confidently.

Job four: context as session state

This is what is live right now: the conversation so far, what you already tried, the decision you made twenty minutes ago and have not written down. It is the only one of the four that dies on restart.

Everyone underestimates this one until a session compacts mid-task and the agent cheerfully re-litigates something it settled an hour ago. The fix is unglamorous. Write state to disk on purpose, or accept that it is gone.

The two axes nobody separates

Capacity and curation are not the same axis, and treating them as one is where the money goes.

	Small window	Large window
Curated input	Fast, cheap, works	Works, costs more than it needs to
Uncurated input	Overflows immediately	Fits, and quietly degrades

The bottom-right cell is the expensive one, because everything looks fine. It fits. Nothing errors. The output is just worse in ways that do not announce themselves.

Anthropic's engineering team makes this argument in their context-engineering post, framing the model as having an "attention budget" and defining good practice as finding "the smallest possible set of high-signal tokens." Take the framing seriously but note the position: the company describing your window as a scarce resource also sells the window. That is a claim worth checking against someone with no such incentive.

There is one. Liu et al., published in TACL, tested how models actually use long inputs on multi-document QA and key-value retrieval. Performance was highest when the relevant information sat at the beginning or the end of the input and dropped when the model had to reach into the middle — and that held even for models explicitly built for long contexts. That is peer-reviewed, it is not selling a context window, and it says position inside the window changes the answer. Capacity is not the variable. Placement is.

The widely-cited "context rot" work points the same direction, and the lab behind it sells a vector database, so "long contexts degrade, retrieve instead" is a conclusion they profit from. It survives the objection mainly because Anthropic cites it against its own product interest.

Where this breaks

The four-way split is vocabulary, and vocabulary has a tax. On a two-person team where the same person owns all four systems, this is overhead — go build something. The split earns its keep when the four jobs have different owners, which is exactly when the argument starts.

It also breaks at the edges. Prompt caching sits across capacity and instruction. A compaction step is session state being rewritten into capacity. I am drawing lines through something continuous because the lines make meetings shorter, not because the territory has them.

The question I ask now

When someone says we need better context, I ask which one before anything else: is this a room problem, a retrieval problem, a rules-file problem, or a state problem?

Room is a budgeting question. Retrieval is a ranking and evals question. A rules file is a twenty-minute edit. State is a persistence question. Four different afternoons, four different owners, one shared noun.

I got the shape of this from Shawn Wallace, who ran the same play on the word "agent" and found three jobs hiding under it. The form travels because the industry keeps doing this: a word gets useful, then absorbs every adjacent thing anyone shipped, then stops carrying information.

Writing this post, I ran the count against my own corpus and got 159,669 active chunks. My workspace guide — the instruction file, job three, the one loaded into every session I start — said 126,000. It had been verified eight days earlier. The file whose job is to describe the corpus was wrong about the corpus, and it had been telling every agent I ran the wrong number all week.

Job three going stale about job two. I had to write the taxonomy to catch it.

Karpathy's LLM Wiki Works. Here's What Broke When I Ran It at Real Scale.

Michael Tuszynski — Thu, 13 Aug 2026 18:38:58 +0000

Andrej Karpathy's LLM Wiki gist from April is right about the important part: for personal knowledge, a markdown wiki that Claude maintains beats a RAG pipeline. I've been running a version of this pattern in production against my own life for six months. Karpathy is right — and he stopped at exactly the point where the interesting problems start.

Here's the pattern, briefly, then what actually breaks.

The claim, in one paragraph

You keep raw sources in one directory. You keep a wiki of markdown pages in another. A CLAUDE.md file tells Claude how the wiki works: naming, folder layout, what to do on ingest, how to handle contradictions. New source comes in, Claude reads it, writes or updates concept pages, links them with [[wikilinks]], updates an index. When you ask a question, Claude reads the wiki, not the raw sources — the synthesis has already been done at ingest time. The whole thing sits under ~100k tokens, so Claude can hold the index in context and reason over it directly. No embeddings, no vector store, no infrastructure. Two clean community implementations to start from: nvk/llm-wiki and Ar9av/obsidian-wiki.

That's the whole thing. It works. It works better than you'd expect. And every problem I've hit with it shows up in the same three places.

My setup, so the failure modes are grounded

Mine is a NEXUS workspace: an Obsidian vault split into 000-inbox/, 200-knowledge/, 300-entities/, 400-daily/, plus per-domain context files under 200-knowledge/context/ — one each for finance, health, content, jobsearch. Every domain has its own schema and its own conventions. A top-level CLAUDE.md sets the operating rules; each domain file specializes them. Entities (people, companies, services, properties) are single markdown files named by their wikilink text, so [[Janney]] grep-searches straight to vault/300-entities/orgs/Janney.md. Daily logs at vault/400-daily/YYYY-MM-DD.md are the append-only session record. A MEMORY.md at the root holds hard-won lessons — 34 entries and counting.

Volume: several hundred wiki pages, four active domains, daily writes going back to February. Well past Karpathy's toy scale, well under any RAG-scale problem. This is the middle band the original gist never addresses.

Where it broke: three failure modes, all the same shape

1. Drift between the "recent changes" section and the "how it works" section of the same file.

The gist assumes ingest and update. It doesn't say what happens when something you documented in April is contradicted by what you did in July. In practice, Claude appends a new dated entry to a changelog at the top of the file, feels done, and leaves the reference section below still describing the dead system as current. A reader — human or agent — trusts the reference section for "how does this work today," and it's wrong. Writing a dated log entry feels like recording the change, so the present-tense prose that people actually read for "how does this work now" never gets revisited. The rule I now enforce: a migration isn't done when the changelog says so; it's done when the reference sections match.

2. Silent contradictions between pages.

Two entity cards for the same account — one calls the institution "Janney," the other calls it "Janney Montgomery Scott LLC" — and a downstream aggregation double-counts because the dedup key is the name. RAG failures are silent because retrieval quietly misses a chunk. Wiki failures are silent for a different reason: two pages both exist, both are internally consistent, and neither knows about the other. Claude doesn't spontaneously notice.

3. Orphan pages and dead wikilinks accumulate faster than lint catches them.

Karpathy's original suggests running lint occasionally. At three hundred pages, "occasionally" is not a frequency — it's a hope. Weekly lint recovers most of it. Nothing recovers a stale page that nobody links to and nobody re-reads.

All three are the same shape: the wiki has no immune system against its own past. It compounds knowledge, but it also compounds staleness, and it needs help distinguishing the two.

Three things I bolted on

A drift linter that fails the build. A Python check that walks the vault, cross-references any "retired" or "decommissioned" term against every file, and flags live mentions of dead systems in the canonical docs. It runs pre-commit and on a scheduled cron. When Ghost got replaced by an Astro static site, the linter caught nine files still describing the Ghost publish path as current — including the top-level project README. Without it, a reader would have reached for dead credentials.

Per-domain sub-schemas, not one giant CLAUDE.md. The gist treats the schema as one file. That doesn't survive the second domain. Finance and health have almost no overlap in vocabulary, conventions, or what a "page" is. My top-level CLAUDE.md sets universal rules (voice, safety, wikilink format, verification protocol). Each domain has its own <domain>-context.md that specializes them. A session working on finance loads the domain file; a session working on content loads a different one. Same architecture as scoped CSS.

A vector index alongside the wiki, not replacing it. Once I passed roughly 100k tokens of wiki content plus 100k tokens of session logs plus tens of thousands of legacy conversations, Claude couldn't hold the index in context anymore. The gist's answer to this is "at that point, use RAG." My answer is: keep the wiki as the authored, curated layer, and add a pgvector index over the whole corpus — vault plus session history plus legacy archives — for recall. Around 126k chunks on Postgres 17 with mxbai-embed-large embeddings running on a Mac mini. Not a replacement for the wiki; a way to find the right wiki page when your brain forgot which slug you wrote it under two months ago. The RAG layer is a retrieval tool for me and my agents. The wiki is still the canonical knowledge.

Where the pattern still wins

For synthesis, entity resolution, and the "how do these three things I noticed relate" question, nothing else comes close. RAG genuinely doesn't do it — the structural argument about retrieval having no accumulation is correct. When I ask "what's the through-line across the last month of daily logs on finance," the answer draws on synthesis pages that were written when each log came in. The work happened once, at ingest.

Entity cards are the other win. A single file per person or company, named by its wikilink text, becomes the source of truth. Everything else references it. When a fact about an entity changes, you change one file. This is the boring pattern that pays for itself every day.

Where it stops — the honest edge cases

The wiki pattern breaks when any of these are true:

Multi-actor writes. If two agents write to the same wiki page concurrently, there is no lock. Mine is single-writer by convention. If you fan out coding agents that all edit SESSION-STATE.md, you need a lockfile — the gist has nothing to say about this.
Recall across the boundary of a Claude session. A wiki isn't memory. Claude reads it fresh each session. Cross-session memory ("what did we decide last Tuesday") needs a separate store, which is what daily logs plus the vector index are for.
Anything past ~100k tokens of hot wiki content. The load-bearing assumption is that the index fits in context. When it doesn't, either you shard by domain (my approach), add a retrieval layer, or accept degraded answers.
Anything anyone else needs to read without Claude in the loop. A markdown wiki is legible to humans. A markdown wiki written by Claude, for Claude, over months has enough house-style shorthand that a new reader without the schema will miss context. The wiki is legible; the accumulated conventions are not.

What to actually do

Start where Karpathy says. One folder, one schema, five sources, in one domain you're actively thinking about. Do it for two weeks before adjusting anything.

At the point it starts feeling like it's working — around thirty or forty pages — put three things in before you regret not having them: a drift check that flags dead terms in live docs, one lint pass a week that actually runs, and a rule that any migration updates the reference section, not just a changelog. That's the operating discipline the pattern needs and doesn't ship with.

Add a vector index only when the wiki genuinely stops fitting. Not before. Karpathy is right about that too — most people reach for RAG when they should have written a schema.

Your $40K GPU Runs 6 Hours a Day: The Utilization Number Every Local-LLM TCO Omits

Michael Tuszynski — Mon, 10 Aug 2026 15:55:49 +0000

Nobody buys a $40,000 GPU and then asks how many hours a day it's actually computing. They ask how many tokens per second it can do at peak, divide the purchase price by a made-up annual token volume, compare that to an API price sheet, and put the winning number on a slide.

The number on that slide is wrong by roughly an order of magnitude. Not because the arithmetic is bad — because it's missing a term.

The term everyone omits

Per-token pricing and per-hour ownership are different units. An API bills you for work performed. A GPU bills you for time elapsed, whether work happens or not. To compare them you need a conversion factor, and that factor is utilization.

Utilization isn't one number, it's two multiplied together:

Duty cycle — what fraction of wall-clock hours the box has any inference in flight. A GPU serving an internal dev team runs during business hours in one timezone. Call it 6 hours a day, five days a week. That's 30 hours out of 168. Duty cycle: 18%.

Saturation — during those busy hours, what fraction of peak throughput you actually sustain. This is where the damage is. An H100 running a 70B-class model with continuous batching wants 25 to 40 concurrent requests to hit its throughput ceiling. Interactive users generate three or four. You're running at maybe 20% of what the silicon can do, while paying for 100% of it.

Multiply them. 18% × 20% = 3.6% effective utilization. That's the conversion factor between your capex and your token bill, and virtually no procurement model contains it.

I've measured this before on much smaller hardware — a Mac mini doing local inference for a real workload sat busy 1.7% of the time over three weeks. The absolute number scales with the box. The shape doesn't.

Running the math both ways

Take a single-GPU node, honestly costed. $40K for the card, three-year straight-line depreciation: $13,333/year. Power at a 1.4 kW node draw and $0.15/kWh commercial rate: about $1,840/year running flat out. Rack and connectivity in a colo: $2,400. Ten percent of one platform engineer to own the thing — patching, driver hell, model upgrades, on-call: $18,000 at a fully loaded $180K.

Annual owned cost: ~$35,500. The staffing line is the biggest single item and it's the one most spreadsheets leave blank. The operational bleed around open-weight deployment is not the GPU. It's everything attached to it.

Now throughput. Assume that node sustains 2,000 output tokens/second aggregate under healthy batching. At 100% utilization that's 63 billion tokens a year.

Effective utilization	Tokens/year	Cost per M tokens
100%	63B	$0.56
25%	15.8B	$2.25
10%	6.3B	$5.63
3.6%	2.3B	$15.65

At full tilt the box is spectacular — half a dollar per million tokens beats every commercial endpoint on the market. At the utilization your team will actually produce, it costs more than Claude Sonnet. Same hardware. Same model. Same electricity bill. The only variable that moved was the one nobody modeled.

Put a percentage on the slide, not a payback date

Here's the reframe I'd argue for, and it fits in one line of a deck.

Stop computing break-even as a token volume. Compute it as a break-even utilization — the fraction of peak throughput you must sustain for the owned box to beat the API you'd otherwise call.

U* = annual_owned_cost / (api_price_per_Mtok × peak_Mtok_per_year)

For the node above, against a $3/M blended API price: $35,500 / (3 × 63,000) = 18.8%. You need to hold nearly a fifth of peak throughput, all year, to win.

Against a $0.60/M hosted open-weights endpoint, the same node needs 94% utilization. That's not a hard target. That's a physically unreachable one for anything but a batch queue.

U* is better than a payback date for three reasons that matter to the person approving the purchase: it's a single number, it's measurable before you buy, and it's falsifiable. "We break even in 14 months" can't be checked until month 14. "We need 19% sustained utilization" can be checked next Tuesday against a week of Prometheus data.

If you're already serving inference somewhere, you can measure both terms today:

# duty cycle — fraction of the last 7d with inference in flight
avg_over_time((DCGM_FI_DEV_GPU_UTIL > bool 5)[7d:1m])

# saturation — how full the batch actually runs (vLLM)
avg_over_time(vllm:num_requests_running[7d]) / <max_num_seqs>

Multiply. That's your real conversion factor. Divide your annual owned cost by it. Then compare.

The denominator keeps moving

There's a second problem, and it's worse than the first. U* isn't a constant. It rises over the life of the asset, because the API price in the denominator keeps falling.

Epoch AI's analysis of inference pricing found that the cost of reaching a fixed capability level has been dropping between 9x and 900x per year depending on the task. Take the conservative end. A 9x annual decline means the $3/M endpoint you benchmarked against is a $0.33/M endpoint a year later.

Your break-even utilization went from 19% to 171%. In year one. Your depreciation schedule did not move.

That's the structural problem with capitalizing inference hardware: you're making a three-year fixed commitment against a price curve that resets quarterly. The academic cost-benefit work on on-prem deployment generally models break-even against today's commercial prices. Fine as a snapshot. Dangerous as a purchase justification, because it treats the comparison baseline as static when it's the fastest-moving number in the stack.

What I'd actually do

Measure before you buy. One week of duty-cycle and saturation data from a rented instance running your real traffic costs a few hundred dollars and answers the question the spreadsheet is guessing at. Rent the exact GPU for a month first. Every cloud has them by the hour.

If your measured U is below 40%, don't buy the box for cost reasons. There aren't any. Route the interactive traffic to an API and keep looking.

If you have batch work — nightly document extraction, embedding regeneration, eval suites, synthetic data generation — that's the workload that fills the trough. Owned hardware pays off when you can schedule against it, because scheduled work is the only kind that saturates. A queue that runs 2am to 6am at full batch depth does more for U* than doubling your user count.

And if the real driver is data residency, a contractual requirement, or a latency floor you can't hit over the public internet — buy it. Those are good reasons. Just price the utilization gap and call it what it is: a control premium, with a dollar figure next to it. That's a defensible line item. "We'll save money on tokens" is not, and it falls apart the first time someone opens Grafana.

The finance team will accept a premium they can see. What they won't forgive is a payback model that assumed 63 billion tokens and delivered 2.3 billion.

Local LLM vs Cloud API at 12 Months: The 3 Line Items Every Per-Token Comparison Omits

Michael Tuszynski — Mon, 10 Aug 2026 00:49:28 +0000

I run a dedicated local inference host. A Mac mini M4 Pro, 14-core CPU, 20-core GPU, 64GB of unified memory, bought open-box for $3,349. It serves embeddings for a retrieval system, a fallback chat model, and a classifier for inbound notes. It has eight models resident, about 140GB on disk.

Over the past three weeks it has been busy 1.7% of the time.

That number comes from its own request log: the summed wall-clock duration of every generate, chat, and embed call, divided by the day. On the heaviest day in the window it hit 3.45%. On several days it did no inference at all. The daily request count looks healthy at roughly 1,440, until you notice that 1,440 is exactly one request per minute: an uptime monitor polling /api/version. The machine's most reliable customer is a health check.

I would still buy it again. But it broke my own cost model, and the way it broke is the part worth writing down.

The month-0 spreadsheet is not the month-12 bill

Almost every local-versus-cloud comparison, including the one I wrote last year, is a snapshot. You price the hardware, price the tokens, divide, and find a break-even volume. That math is fine on the day you run it. It quietly assumes three things that stop being true somewhere around month four.

The gap is not small. McKinsey's July 2026 survey found 93% of respondents exceeding their AI budgets while 62% had moved past experimentation into active deployment. McKinsey puts 20 to 30% of that spend within reach of better accounting. That is a survey of self-reported budgets, so treat the precise figure as directional. The direction is the interesting part: the overruns arrive after deployment, not during the pilot. These are the three line items that show up in that window.

Line item 1: the capacity you bought, not the capacity you used

Cloud inference bills you for work performed. Hardware bills you for work available. At 1.7% utilization those are different products wearing the same label.

Run my numbers forward. Roughly 0.4 busy hours a day is about 146 hours of actual inference a year. Amortize the $3,349 over a single year and each hour of real work costs $22.90. Stretch it across three years, which is fairer to the hardware, and it is still about $7.60 per busy hour. Electricity barely registers. The mini idles near 7W and peaks around 45W under load, so 8,600 idle hours plus 146 busy ones comes to roughly 67 kWh, about $23 a year at California rates.

That last figure is worth pausing on, because my earlier post led with cooling and power draw. At rack scale, with 700W GPUs, that emphasis is right. At the scale most teams actually start at, one box serving one team, power is a rounding error and utilization is the entire story. The line item that matters flipped when the deployment got smaller.

The honest version of the local-inference pitch is not "tokens are cheaper." It is "I am buying a fixed monthly cost and a latency floor, and I will pay it whether or not anyone sends a request."

Line item 2: the cloud price is a moving target and your hardware is not

You commit to hardware at a fixed price. The alternative you rejected keeps getting cheaper underneath you, and not evenly.

Epoch AI's analysis of inference pricing found that prices to reach a fixed performance level have fallen between 9x and 900x per year, depending on which capability you are buying. Matching GPT-4's performance on PhD-level science questions got about 40x cheaper per year. General-knowledge performance fell more slowly, in the 9x-to-40x band. Epoch adds a caveat most people quoting these curves skip: the fastest declines happened in the most recent period, so there is no guarantee they persist.

Two consequences for a 12-month decision. The break-even volume you computed in month 0 moves against local hardware every month you own it, because your capex is sunk while the rented alternative reprices. And the rate it moves depends on your workload, because the decline is task-dependent. A team doing hard reasoning has seen its cloud alternative collapse in price. A team doing bulk classification has seen a gentler slope, and their local box holds its case longer.

Nobody's model has a row for this. Build one: re-run the comparison quarterly with current API pricing rather than the pricing you started with.

Line item 3: you pay per token and you bank per accepted answer

The third omission is the denominator. Both sides of this comparison are usually priced per million tokens, which measures what the model emitted, not what a human kept.

I made this argument in isolation two weeks ago and it landed flat, because an argument without a table is just an assertion. Here is the table. A smaller local model that needs two attempts where a frontier API needs one has doubled its effective token cost, and it has spent your reviewer's attention twice. Reviewer attention is the expensive input here, not the tokens. Tracking where the token spend actually goes is the prerequisite; the acceptance rate is what turns that into a cost.

Line item	Cloud API	Local hardware	How to measure it
Idle capacity	~$0 when idle	Full cost regardless of use	Busy-seconds ÷ elapsed, from your server log
Price decay	Falls 9x–900x/yr, task-dependent	Fixed at purchase; capex is sunk	Re-price the API side quarterly
Rework	Per token, retries billed	Per token, retries billed in idle capacity too	Accepted outputs ÷ total generations

Three rows, all measurable from data you already have. None of them appear in a per-token comparison.

Where this breaks

My 1.7% is a single-operator homelab, and it is the weakest possible case for owned hardware. A shared inference cluster behind a real queue, serving fifty engineers across time zones, lands far higher. Above roughly 40% sustained utilization the capacity argument inverts and hardware wins on cost alone. If your box is genuinely saturated, line item 1 is not your problem.

Two other cases where none of this decides anything. Data residency and compliance can make local inference the only lawful option, at which point the cost comparison is a formality. And batch or overnight work reshapes the math, because idle hours you can schedule into are not idle.

There is also a cost to my own advice. Measuring utilization honestly makes the case against a machine I own and use daily. I kept it, for reasons that never appear in a TCO model: no rate limits, no per-request thinking, models that stay put, and a place to run things I would not send to an API. Those are real. They are just not "cheaper," and calling them cheaper is how a $3,349 box turns into a line item somebody has to defend at renewal.

Go pull your own number first. If you run Ollama, your utilization is sitting in ~/.ollama/logs/server.log right now, and it takes one awk to find out whether you bought a workhorse or a very expensive uptime-monitor endpoint.

Your Local LLM Bill Is Per Token. Your Real Cost Is Per Accepted Answer.

Michael Tuszynski — Wed, 05 Aug 2026 20:26:19 +0000

Finance will hand you a per-token comparison. Local inference at some fraction of a cent per thousand tokens, the frontier API at some multiple of that, and a delta that looks like a budget line worth defending. The arithmetic is fine. The denominator is wrong.

Tokens are a throughput unit. Nobody buys tokens. You buy answers a human is willing to ship, and the ratio between those two things is not a constant — it moves with model size, task class, and how many attempts it takes to land one. The moment you divide by accepted answers instead of tokens, most local-versus-cloud spreadsheets stop saying what they said.

I priced the numerator side of this last year — datacenter GPUs, power, cooling, and the DevOps tax of running your own inference. That post assumed the denominator. This one goes after it.

Five things the per-token quote silently drops

Retries. A 32B model that gets it right 70% of the time on the first pass costs you 1.43 generations per accepted answer, not 1. Every failed attempt burns the same tokens as a successful one and produces nothing.

Escalation. Some fraction of tasks the local model cannot finish at all. Those tasks pay the local cost and the frontier cost. If you don't measure that rate, you're carrying it as an invisible surcharge on the cheap path.

The review pass. This is the big one and it never appears in a token quote. A weaker model shifts work onto a person, and a person costs three to four orders of magnitude more per minute than the accelerator does.

Idle time. Utilization, not throughput, sets unit cost on owned hardware. The box depreciates whether or not anything is in the queue.

Amortization and refresh. Per-token framing treats hardware as free after purchase. It isn't. It's a fixed cost divided by however many tokens you actually generate before you replace it.

What my own box costs, and how I got there

I run a dedicated inference host: a 64GB M4 Pro Mac mini serving llama3.3:70b, qwen3.5:27b, qwen2.5-coder:32b, and deepseek-r1:32b through Ollama. Everything below is measured on that machine, and every assumption is on the table so you can redo it with yours.

Assumptions: $2,000 acquisition, 36-month refresh window, roughly 10W idle and 45W under sustained generation at the wall, about 8 output tokens per second on the 70B at 4-bit quantization. Electricity at $0.31/kWh — that's the rate used in a good watt-by-watt measurement of Apple silicon inference, which also priced per million output tokens and then re-ran the numbers against 30 days of real traffic rather than benchmark loops. Substitute your utility rate.

At my actual duty cycle — about 90 minutes a day of real generation, 45 hours a month:

Amortization: $2,000 ÷ 36 = $55.56/month
Power: 2.03 kWh generating + 6.75 kWh idling = 8.78 kWh = $2.72/month
Output: 45h × 3600s × 8 tok/s = 1.296M tokens
≈ $45 per million output tokens

Now saturate the same box. 24/7 generation, 720 hours, 20.7M tokens, $10.04 of electricity. Same amortization, same tokens per second, same model weights.

≈ $3.17 per million output tokens.

Fourteen-fold swing in unit cost with zero change in hardware or performance. The variable was utilization. This is why an H100 cluster and a Mac mini can both be "cheap per token" on a slide and neither number survives contact with a real request pattern — the cluster amortizes a far larger fixed cost and needs far higher sustained load to get there.

The formula to bring to the meeting

Cost per accepted answer:

CPAA = (A × t_l × c_l) + (e × t_f × c_f) + (m ÷ 60 × W)

A   = attempts per accepted answer (1 ÷ first-pass acceptance rate)
t_l = output tokens per local attempt
c_l = your local cost per token (amortization + power ÷ tokens actually generated)
e   = escalation rate to a frontier model
t_f = output tokens on the escalated call
c_f = frontier rate per token (your rate card)
m   = human review minutes per accepted answer
W   = loaded hourly cost of the reviewer

Worked, with my $45/M and plausible task numbers. A = 1.4, t_l = 900 tokens, e = 0.15, m = 4 minutes, W = $120/hour loaded. For c_f I'll use a $15/M output placeholder — plug in whatever your contract says.

Local tokens: 1.4 × 900 × $0.000045 = $0.057
Escalation: 0.15 × 900 × $0.000015 = $0.002
Human review: 4 ÷ 60 × $120 = $8.00

$8.06 per accepted answer. The token lines are 0.7% of it.

Run the same task class entirely on the frontier model. Acceptance goes up, so A drops to 1.15, e goes to zero, and review drops to 2.5 minutes: $0.016 in tokens plus $5.00 in review. $5.02.

The local box wins the token comparison by about six cents and loses the decision by three dollars. Ninety seconds of extra review per answer is worth roughly fifty times the entire per-token delta at my volumes.

The honest counterargument: m is not fixed. On narrow, well-bounded work — commit message drafting, log triage, structured extraction against a fixed schema — a 32B coder model produces output a reviewer skims rather than audits, and m is identical on both paths. When that's true the local box wins outright and the math is not close. My point isn't that local always loses. It's that the term deciding the outcome is one nobody measures, and the term everybody argues about rounds to noise.

Measure m. Measure A. Measure e. Then argue.

Keep escalation a config line, not a rewrite

None of this is measurable if the model choice is compiled into your application. You need the router in front, and the routing decision has to be data.

That's the pattern in inference-router — an OpenAI-compatible layer that sits between the app and both backends, so which model serves which task class is a config change and every escalation is a countable event. Point the app at one endpoint. Move a task class from qwen2.5-coder:32b to a frontier model by editing a file, watch CPAA, move it back if the review time didn't drop.

Without that, e is unknowable and you're arguing from vibes. With it, escalation rate is a metric you can put on a dashboard next to acceptance rate, and the local-versus-cloud question becomes a per-task-class answer instead of a religious one.

While you're in there, cap output length. One team traced a large share of their bill to uncontrolled, conversational output tokens — verbose responses cost the same whether the extra tokens help or not, and on owned hardware they cost you queue time too.

Buy local for the reasons that hold up

There are three defensible reasons to run inference on hardware you own: you need the data to never leave your boundary, you need latency a network round trip can't give you, or you need to keep serving when a vendor changes a model version out from under you. All three are real, and none of them require the token math to work out in your favor.

"It's cheaper per token" is not on that list. It's the claim finance will test first and the one most likely to fall apart, because it's the only one that depends on a denominator nobody is tracking.

My mini pays for itself on data residency. At $45 per million output tokens and a 4% duty cycle, it does not pay for itself on price, and I'd rather say that out loud than have someone else find it in a spreadsheet.

The 26% Who Never Rolled Back an Agent Aren't Winning

Michael Tuszynski — Wed, 29 Jul 2026 00:00:48 +0000

Somewhere right now there's a slide in a board deck claiming a clean record: zero agent rollbacks since launch. It reads like a win. It's closer to a smoke detector with the battery pulled out.

New survey data from Sinch puts the number at 74% of enterprises that have rolled back a deployed AI agent after it went live — customer-communications agents specifically, pulled over governance failures. That figure got picked up everywhere as an indictment — three out of four agent deployments blowing up in production. But the interesting number is the one underneath it. Among organizations with the most mature guardrails, the rollback rate goes up, to 81%.

Read that again. Better safety infrastructure correlates with more rollbacks, not fewer.

Rollback rate measures your eyes, not your agent

There's only one clean way to explain that inversion. The teams with better instrumentation aren't shipping worse agents. They're catching things the other teams are shipping past.

Every agent in production is doing something wrong at some rate. Hallucinated policy details, tool calls against stale records, a tone that drifts on the eighth turn of an angry conversation, a refusal loop that pushes a customer to a human queue that's already 40 deep. The question was never whether the failure exists. It's whether anything in your stack notices it before your customer does.

So the 26% split into two very different groups. A small number genuinely nailed scope — narrow task, tight tool surface, a human approving anything consequential. The rest have no detector. Their agent is failing at whatever the base rate is, silently, and the absence of a rollback is being reported upward as quality.

SRE teams learned this lesson two decades ago and it stuck: an incident count is a function of your monitoring, not your reliability. A team that goes from 3 incidents a quarter to 30 after installing real alerting did not get 10x worse. They got 10x more honest. It's the same principle the write-ups on this data keep circling — the model rarely breaks, the infrastructure around it buckles. Which means model choice isn't what separates the 26% from everyone else.

Treating rollback rate as a failure metric creates the exact incentive you don't want. A VP who gets graded on rollbacks will not build better agents. They'll build quieter ones — loosen the eval thresholds, downgrade the alert to a weekly digest, route the complaint channel to a shared inbox nobody owns. The metric improves. The agent doesn't.

The guardrail tax nobody budgeted for

The second number in that survey is the one that should reset your staffing plan. 84% of AI engineering teams spend at least half their time on safety infrastructure rather than on the agent itself. And enterprise investment now skews toward trust, security, and compliance (76%) over AI development proper (63%).

That's not overhead creeping in at the edges. That's the majority of the work.

It also matches what production looks like. The agent is a prompt, a model call, and a tool list — a week of work for a competent engineer. The other eleven weeks go to the eval set, the PII redaction layer, the escalation path, the audit log that survives a compliance review, the shadow-mode comparison, the per-version rollback switch, and the on-call rotation that owns all of it at 2 a.m.

Microsoft's 2026 Work Trend Index frames the organizational side of this as a capacity question — whether organizations are built to capture the agency that agents free up. Most aren't, and the guardrail tax is a good part of why. The headcount that was supposed to move up the value chain is instead maintaining the machinery that keeps the agent honest.

Who paid for this, and what it actually covers

Now the part the LinkedIn reposts skipped.

Sinch is a CPaaS vendor. They sell customer-communications infrastructure. A survey they sponsored concluding that infrastructure quality predicts agent success is a survey concluding that you should buy more of what Sinch sells. That doesn't make the data wrong, but it does mean the framing was chosen before the responses came in.

The scope is narrower than the headline suggests, too. This is customer-communications agents — support, messaging, conversational flows. Not coding agents, not internal RAG assistants, not the finance-ops bot reconciling invoices. A customer-facing agent has an unusually loud failure mode: the customer complains, and the complaint is logged in a system somebody already reads. Detection is close to free. In a domain where the agent's mistakes land quietly in a document nobody re-reads for six weeks, the 26% number would almost certainly be higher — and mean even less.

Hold the core finding anyway. The correlation between guardrail maturity and rollback frequency is directionally strong enough to survive the sponsorship discount, because it points the opposite direction from the sponsor's simplest sales pitch. "Buy our stuff and you'll roll back more often" isn't a slogan anyone reverse-engineers into a survey.

Instrument the rollback, then go trigger one

Stop reporting rollback count. Start reporting rollback provenance. One row per deployed agent version, and the field that matters is who noticed:

release_id      agent-support-v14
model           claude-sonnet-5
prompt_sha      a91f3c2
tools_sha       7de0b41
deployed_at     2026-07-14T09:12Z
rolled_back_at  2026-07-16T23:41Z
detector        eval_regression | slo_breach | abuse_signal
                | agent_self_report | human_complaint
time_to_detect  3410 minutes

Now run the distribution on detector. If 80% of your rollbacks say human_complaint, your detection layer is your customers, and your time_to_detect is however long it takes an annoyed person to find the contact form. Whatever that interval turns out to be in your own data, multiply it by your conversation volume — that's how many exchanges you can't un-send.

Three things follow from that ledger.

Set a target on time-to-detect, not on rollback count. An hour is achievable with automated eval replay on a sampled slice of live traffic. Days is what you get when complaint volume is the detector.

If you've never rolled back, go cause one. Inject a deliberately degraded prompt into a canary slice — drop a rule from the system prompt, or point a tool at a stale index — and time how long your stack takes to flag it. A rollback path that has never been exercised is a hypothesis, not a control. Same logic as restoring from backup: nobody has a backup, they have a restore they've tested or a file they hope is fine.

And when a vendor pitches you an agent, ask how many times they've rolled back their own. A confident zero means they aren't looking. The right answer sounds like "four times last quarter, median detection 22 minutes, here's the ledger."

The 81% cohort isn't failing more often. They're the only ones who can prove what their agent did last Tuesday at 3 a.m.

AI Use Is a 50/50 Coin Flip. Coding Already Tipped 61% Autonomous.

Michael Tuszynski — Thu, 23 Jul 2026 20:08:27 +0000

The Copilot Framing Is Already the Minority Report

Every vendor deck for a coding tool sells you the same picture: a developer in the driver's seat, the AI riding shotgun, a human hand on every merge. Copilot. Human-in-the-loop. Pair programming with a machine. It's the comfortable frame because it keeps the person central and the tool subordinate.

For software development, that frame already describes the smaller half of what's actually happening.

Anthropic's Economic Index publishes real data on how people use Claude, and it tags every conversation one of two ways. Augmentation means the person stays actively in the loop — back and forth, iterating, learning, steering each step. Automation means the person hands off a task and directs Claude to complete it. Across everything, it's nearly a coin flip: 51 percent augmentation, 49 percent automation globally, and a clean 50/50 in the US.

Narrow to software development tasks and the coin lands differently. 39 percent augmentation, 61 percent automation. The pattern everyone quotes in their pitch is the one coding has already moved past.

What the Split Actually Measures

Read the label carefully, because it's easy to turn this into something it isn't. Augmentation versus automation is a conversation style, not a jobs claim. It says nothing about displacement, headcount, or whether a role survives. It measures one thing: in a given exchange, is the human collaborating turn by turn, or delegating the whole task?

That distinction matters because the two styles want different products underneath them. An augmentation session is a dialogue — the tool's job is to be responsive, explain itself, and stay legible while a person drives. An automation session is a delegation — the tool's job is to run to completion and come back with something you can check.

Most coding tools are still built for the first job. The interface assumes you're watching, approving diffs one at a time, keeping a hand on the wheel. When 61 percent of the actual work is delegation, that assumption is backwards for the majority of what your developers are doing.

The Number That Keeps This Honest

Here's the counter-weight, and it's in the same dataset. Software development is only about 11.5 percent of global Claude requests, and 8.1 percent in the US. Coding is not the center of gravity for how people use these models. It's a slice.

That cuts against the temptation to read "61 percent automation in coding" as "AI is taking over programming." It isn't taking over anything. It's one task category, and inside that category the delegation style leads. Both things are true, and the small share is the part that keeps the big number from getting oversold.

One more caveat worth stating plainly: this is a single snapshot from May 2026. No trend line, no trajectory, no "up 12 points from last quarter." One reading. The 61/39 split is a photograph, not a movie. I'd bet the direction of travel is toward more automation as tools get better at running unattended — but that's my read, not something the data shows. Treat it as a fixed point, not a slope.

And note the seam in the categories: Anthropic reports "Computer and Mathematical" as the number one task category overall, yet software development specifically sits at 11.5 percent. Those aren't in conflict — the top category is broad and covers a lot more than writing application code. But if you conflate them, you'll overstate how much of AI usage is engineers shipping features. Most of it isn't.

What a Buyer Should Do With This

If you're choosing or building a coding agent, design for the automation majority. That's the whole recommendation, and it changes what "good" looks like.

The copilot default optimizes for a person watching every step. The automation pattern optimizes for a person checking the result. Those need different architecture. I've written before that a coding agent stack has two layers — the model and the scaffolding around it — and that the scaffolding decides the outcome more than the model choice does. This data is why. When 61 percent of real coding work is delegation, the layer that runs the agent unattended and catches its mistakes is doing the heavy lifting, not the chat window.

Concretely, designing for automation means:

Autonomous execution with verification gates, not approval-per-diff. The agent should run the task end to end, then hand you a result gated by checks it had to pass — tests green, types clean, lint quiet, a diff that touches only what it claimed. You review the outcome and the evidence, not every keystroke. A human approving line-by-line is the augmentation product, and it doesn't scale to 61 percent of your work.

Real gates, not vibes. The gates are the whole safety story once the human steps back from each step. That means a test suite the agent can't skip, a build it can't merge past when red, and provenance on what changed. If you can't articulate what has to be true before an agent's output reaches main, you haven't built for automation — you've built a faster way to generate unreviewed code.

Legibility at the boundary, not throughout. Augmentation needs the tool legible at every turn. Automation needs it legible at one point: the handoff. Invest your explanation budget there — what did it do, what did it check, what should you look at first.

The tools that ship copilot-by-default are optimizing for the 39 percent. That's a real 39 percent, and for exploratory work, learning a new codebase, or high-stakes changes where you genuinely want to drive, augmentation is the right mode. Don't rip it out. But if your default interaction model assumes a human hand on every commit, you've built the minority product and called it the platform.

The Frame Is Behind the Practice

The interesting gap here isn't between humans and machines. It's between how the industry talks about coding agents and how people actually use them. The marketing still runs on the copilot metaphor — reassuring, human-centered, a decade old. The usage data says developers crossed over to delegation a while ago and mostly stopped narrating it.

61 percent. That's the number to bring to your next tool evaluation. Ask the vendor what happens after the human stops watching — what runs, what gets checked, what stops a bad change cold. If the answer is "well, you approve each diff," they built for the half that's shrinking.

The Meter Is Always Running

Michael Tuszynski — Tue, 21 Jul 2026 14:02:11 +0000

The Meter Is Always Running

For forty years, enterprise software rested on a comfortable assumption: that it was a fixed cost. You paid a license, amortized it over a depreciation schedule, and every unit of value you squeezed out afterward felt like it was approaching free. Seats, not usage. Capex, not cost of goods. A CFO could draw a straight line from spend to return and sleep at night.

AI breaks that line. Most organizations adopted it without noticing.

The thing that actually changed

The interesting shift in enterprise AI is not cloud versus on-premises. That is a deployment question, and deployment questions are the kind of thing infrastructure teams have solved a hundred times. The deeper shift is quieter: software stopped being a fixed cost and became a variable one.

A large language model does not bill like a license. It bills like a taxi. The meter runs on every token, upstream and downstream, and input and output are priced per million tokens at rates that differ by model and by direction. The fare depends on how long the conversation ran, how much reasoning it demanded, how many times a user hit retry. Two customers doing the "same" task can cost 10x different amounts because one of them pasted a novel into the prompt. There is no seat count that caps it. There is no version you buy once and freeze. The cost scales with success: the more people use the feature, the more it costs you to have built it.

That is a new financial object, and the mental model most companies applied to it, SaaS licensing or depreciable capital, is the wrong shape. You cannot amortize a variable cost. You can only manage it.

The on-prem mirage

The popular rebuttal goes like this: just run it yourself. Buy the hardware once, own it forever, and your marginal cost per query approaches zero. Convert the scary variable cost back into a familiar fixed one.

It is a seductive move, and it is mostly wrong.

The "own it forever" asset is not the GPU. It is the model, and the model is spoiling in real time. The cluster you justified this year runs a frontier model that will be mid-tier in twelve months. The hardware ages too: how long an AI accelerator stays economically useful is now an open argument on Wall Street, with investors questioning whether the six-year depreciation schedules on the books match a reality where a new generation lands every eighteen months. You did not buy a depreciable asset amortized over seven years. You bought a seat on a refresh treadmill, plus a standing bill for power, cooling, and the engineers who keep it fed.

And unlike the cloud meter, that silicon costs the same whether it runs at 90% load or 5%. Idle capacity is the norm, not the exception: it is common for GPU clusters to sit 70 to 80% idle between bursts of real work. You did not eliminate the variable cost. You traded a usage-based meter for a utilization risk, and for most enterprises, whose demand is spiky rather than steady, that is the worse trade.

Marginal-cost-to-zero is real, but only at high, sustained load on a model you are content to keep running. That describes a narrow band of workloads. It does not describe "our AI strategy."

Inference is cost of goods, not capex

Here is the reframe that survives contact with a spreadsheet: inference is cost of goods sold.

Not a license you buy. Not a machine you depreciate. It is a per-transaction input cost, like the cloud compute behind a web request or the fee on a card payment. Companies already know how to run a business where serving each customer costs money. They have gross-margin discipline, unit economics, dashboards that flag when a product line goes underwater. They simply never had to apply that discipline to software features before, because software features used to be free to serve once written.

File inference under cost of goods instead of "IT capex," and the right behaviors fall out on their own. You start asking the questions a margin-conscious operator always asks.

What that discipline looks like

Know your unit cost per feature. If you cannot say what a single run costs, you are blind on the one number that decides whether the feature is a business or a liability. Instrument tokens the way you already instrument latency.

Then route by need. Most requests do not warrant a frontier model; send the cheap, frequent, predictable work to smaller or local models and reserve the expensive reasoning for the calls that actually earn it. A good router is worth more than a bigger model.

Cache aggressively. Repeated system prompts, retrieved context, common answers: every hit is a fare you skip. Anthropic reports that prompt caching can cut input costs by up to 90% and latency along with it. Caching is not a nice-to-have optimization here. It is margin.

Put a ceiling on it. Variable cost without guardrails is how one looping agent turns into a five-figure surprise. Budget caps, rate limits, and per-tenant quotas are the circuit breaker. Build them before the invoice, not after.

And decide placement per workload, not per company. Some things belong on a local model you own; some belong on a metered API. "Cloud-first" and "on-prem-first" are both the wrong altitude. The unit is the workload, and the answer is a portfolio.

The reckoning is real. It is not the one on the thumbnail.

A reckoning is coming for a lot of AI initiatives, but it will not arrive as the dramatic collapse of anyone's cloud strategy. It will arrive as a quiet quarterly review where someone finally divides the AI line item by the number of customers it served and does not like the answer. The projects that survive that meeting will not be the ones that picked the right deployment location. They will be the ones that treated inference as what it is, a variable operating cost with no natural ceiling, and built the discipline to manage it from the first day.

The meter was always running. The only question is whether you were watching it.

Your AI Coding Pilot Cost $80K and Shipped 9% Faster. Here's the Line Item Finance Missed.

Michael Tuszynski — Mon, 20 Jul 2026 14:07:50 +0000

The slide always looks the same. Pilot spend: $80K. Velocity: up 9%. Recommendation: expand to the full org. I have sat through enough of these readouts, on both sides of the table, to recite the speaker notes from memory. The room approves it, because $80K buying a 9% faster engineering org of any size is obviously a deal.

The math on the slide is fine. The problem is a line item that never made it onto the slide, and it is the most expensive input in the building.

Where the $80K Went

Run the composite pilot. Fifty seats of an AI coding assistant at roughly $20 a head is about $12K a year. Agent and API tokens for the teams that went past autocomplete, call it $3K a month once real workloads land — I've published my own bill, so I know what this curve looks like. Add the integration sprint, the eval work, and the enablement time nobody bills to the pilot, and $80K is a fair all-in number for a mid-size org's first serious year.

Every dollar of that is metered, invoiced, and visible. Seats show up on a purchase order. Tokens show up on a usage dashboard. Finance can read the pilot's cost to the penny, which is exactly why the readout feels rigorous.

Now the return side. A 9% velocity gain across a 12-engineer group is roughly one extra engineer's worth of throughput, and a loaded engineer runs $250K, which is the number that dwarfs every token bill in sight. So the slide implies $80K bought about $270K of capacity. A 3.4x return, approved before the coffee gets cold.

The Line Item That Never Shows Up

Here is what the slide does not carry: the review and verification attention the pilot consumed, priced at senior-engineer rates.

AI-assisted pull requests change shape. Diffs get bigger. Volume goes up. And the failure mode shifts from obviously-broken to plausible-but-wrong, which is the expensive kind, because plausible-but-wrong has to be caught by a human who understands the system. Suppose each engineer ships five AI-assisted PRs a week and each one takes a reviewer just twenty extra minutes. Across 12 engineers that is 20 hours a week of review time, half a full-time engineer, roughly $125K a year. And it does not land evenly: it lands on the two or three senior people qualified to catch the plausible-but-wrong category, the same scarce attention that was already the bottleneck.

Take the $270K gain, subtract $80K in tool spend and $125K in unpriced senior attention, and the triumphant 3.4x collapses to about 1.2x, before counting rework. That might still be worth doing. But nobody in the room approved that deal, because nobody saw it.

The reason nobody saw it is structural, and it is worth saying plainly: finance manages what is metered. Seats and tokens arrive as invoices. Attention arrives as nothing at all. It is smeared invisibly across a salary line that looks identical whether your seniors spent the quarter building or spent it correcting a machine's confident guesses.

The Evidence Says the Slide Flatters Itself

The 9% is usually self-reported or lightly measured, and self-report runs hot. In METR's randomized study of experienced open-source developers, participants estimated AI made them 20% faster while the measured result was 19% slower. The perception gap is the mechanism that hides the missing line item: the tool feels fast because typing is fast, while the slow part — verifying, correcting, re-prompting — doesn't register as "using the tool."

The vendor numbers run hotter still. GitHub's own study reported 55% faster task completion, a first-party figure measured on a greenfield toy task, and it should be discounted accordingly. Meanwhile DORA's research keeps finding the bill on the other side of the ledger: as AI adoption rises, software delivery stability dips. Faster generation, wobblier delivery. That wobble is rework, and rework is more senior attention, unpriced.

To be fair to the other side of the argument: METR's result is one population on one kind of task, not a universal law, and pilots built on scoped tasks with an explicit verification budget genuinely do clear the honest bar. The point is narrower and harder to dodge. A pilot that measured only what was invoiced has not measured its return. It has measured its receipts.

The Five-Line Version

The fix costs nothing but honesty. Here is the pilot P&L with all the lines on it:

AI coding pilot P&L

+ Velocity delta
    measured (cycle time,
    merged scope), never
    surveyed
- Seats + tokens
    the invoice (the only
    line the readout had)
- Review-hours delta
    PR review time before
    vs. during, x reviewer
    cost
- Rework delta
    change-failure/rollback
    rate, before vs. during
- Comprehension debt
    merged code nobody on
    the team can explain, %

The first two lines exist in every pilot. The last three exist in almost none, and they are all measurable with tooling you already run: your git host timestamps every review, your incident tracker already counts rollbacks, and the comprehension question takes one uncomfortable team meeting.

If the pilot still clears the bar with five lines on the page, expand it with confidence — mine does, and that's precisely why I keep publishing the full bill instead of the flattering half. A tool that survives honest accounting is a tool worth scaling.

Ask for the Line

The 9% on the slide is neither a lie nor a result. It is an unfinished calculation, presented with the confidence of a finished one, to a room that is being asked to multiply it by the whole org.

So ask the one question that finishes it: show me the review-hours delta. If the answer is a number, you are looking at a rare, well-run pilot, and you should probably fund it. If the answer is a pause, then the 9% was never a measurement. It was a survey wearing one, and the most expensive people in the building are quietly paying the difference.

The Bubble Popper and the Payoff Are the Same Thing

Michael Tuszynski — Thu, 16 Jul 2026 23:59:00 +0000

Cory Doctorow thinks AI is a bubble, and he argues it better than almost anyone who cheers him on. In his December speech at the University of Washington, the case runs like this: the tech giants stopped growing years ago, and a company priced as a growth stock faces catastrophe the moment the market stops believing. So they pump whatever keeps the multiple alive. Pivot to video, crypto, NFTs, Metaverse, now AI, each pitched with the same conviction, each abandoned when the next vehicle arrives. "Superintelligence" is the pitch this cycle because science fiction makes a better P/E story than enterprise software ever did.

His sharpest tool is the reverse centaur. A centaur is a human assisted by a machine: you drive the car, the car amplifies you. A reverse centaur is a human bolted onto a machine as its peripheral, hired to absorb the failures the machine can't, at machine pace, under machine supervision. Doctorow's thesis is that the money behind AI is betting on reverse centaurs: workers demoted into error-cleanup for systems sold to their bosses as replacements.

Take him seriously. The losses are real, the incentives he names are real, and I have sat through enough board-mandated AI pilots to know the reverse centaur is not a hypothetical. And then notice that his argument, stated carefully, is about two different balance sheets that the doom discourse keeps smearing into one.

Two Balance Sheets, One Shouting Match

Seller economics asks whether the labs make money. Company-level losses are enormous, but look at where they live: training runs, data-center capex, sales, and the free tier. The serving unit underneath is a different story. SemiAnalysis puts Anthropic's gross margins in the mid-60s — an independent analyst number, though The Information reported the same company projecting 40% when inference costs spiked, so treat the exact figure as contested. The direction is not. Serving paid tokens is gross-margin-positive; what bleeds is everything wrapped around it.

Even the famous counterexample proves the shape. When Sam Altman admitted OpenAI loses money on the $200 Pro plan, the reason was that heavy users outran the flat price. Sell unbounded agent workloads at a fixed monthly fee and the top of the usage tail eats you. That is a pricing-model problem, and pricing models get fixed. Physics problems don't. This one is being fixed right now, mostly at the expense of people like me: usage caps, tier splits, metered agents.

Buyer economics asks a different question: does a scoped deployment pay for itself? That is where operators live, and none of the training-run losses show up on this side of the table. I've written down what my own stack costs per month, and why "$20K in tokens against a $250K engineer" is the wrong math even when it flatters the tools. A deployment either produces measured value over measured cost or it doesn't. The lab's income statement has nothing to do with it.

Doctorow is right about the first balance sheet. The mistake is letting the first one answer for the second.

The Question That Does All the Work

Here is my actual thesis, earned from reps rather than from a valuation model: the economics work themselves out, but only for operators disciplined enough to ask "what problem am I solving" before pointing the tool at anything, and rigorous enough to measure the answer afterward.

An efficiency you don't understand is not an efficiency. It is an unmeasured cost wearing a demo. The team that deploys a coding agent with no baseline, no success metric, and no owner has not automated anything; it has hired a very fast intern nobody supervises and booked the salary as savings. That is blind deployment, and blind deployment is exactly the reverse-centaur failure: the humans end up serving the machine's output because nobody defined what the machine was for.

Which means my thesis and Doctorow's frame are the same observation read from opposite ends. Scoping rigor is what makes you a centaur instead of a reverse one. The human who decides what the machine is for stays the head. The human who cleans up after an unscoped machine becomes the peripheral.

The scoping questions I use, written down (writing them down matters, and the last section says why):

1. What is the problem, stated without naming a tool?
2. What does it cost today, in hours or dollars I can point at?
3. What does "working" look like, measured how, by whom?
4. What is the failure mode, and who catches it?
5. What is the kill criterion, decided before the pilot starts?

The Payoff and the Pin Are the Same Force

Now the part I have not seen anyone say plainly: the rigor that makes AI pay off is the same force that pops the bubble.

Walk through what happens if scoping discipline actually spreads. Every FOMO seat license gets audited against question 2, and half of them fail. Every board-mandated pilot meets question 3 and dies for lack of a metric. The all-you-can-eat experimentation budgets get metered. A real slice of today's AI revenue is hype-spend, and disciplined buyers stop paying for hype. Revenue evaporates precisely as deployments get better.

That is not bearish on AI. It is bearish on the bubble, and those are different positions. It is Doctorow's own "some bubbles leave behind something productive" scenario made concrete. He reaches for Worldcom, a fraud whose CEO died in prison while the dark fiber it buried still carries Doctorow's 2-gigabit home connection. Swap the fiber for scoped deployments quietly compounding inside businesses while the FOMO spend burns off. The economics working out and the bubble deflating are the same event, narrated by an optimist and a pessimist.

The Craftsman Clause

There is an investor-side implication buried in this that is more uncomfortable than any crash prediction.

If every deployment that pays needs a human who understands both the domain and the tool's failure modes, then AI is not the frictionless labor replacement the trillion-dollar valuations are priced on. It is a power tool that still needs a craftsman. A nail gun does not fire the carpenter; it makes the carpenter's judgment the binding constraint on every house. Attention was always the bottleneck; the tools just moved it.

Great news for buyer ROI. Quietly fatal for the "human exits the loop" story being sold upstream, because the wholesale-replacement bet only pays if the craftsman requirement goes away, and every honest deployment I have run or reviewed says it doesn't. The augmentation bet keeps cashing small checks. The replacement bet keeps pre-spending checks nobody has figured out how to write.

Two Half-Lives, One Fork

So if the scarce input is the human who can scope, the personal question is what that scarcity is worth and how long it lasts.

The scoping skill is teachable in principle and bottlenecked in practice, and the bottleneck is structural, not a fad. The skill is tacit. It lives on the seam between a domain and a tool, where few people sit. And organizations promote people for shipping, not for the scoping that made the shipping cheap; nobody's OKRs reward the pilot that didn't happen.

But the edge has two halves with opposite half-lives, and confusing them is the trap. Tool mechanics — prompt-craft, knowing this month's model's failure modes, the tricks — depreciates fast, because every model release is the labs folding exactly that knowledge into the product. The agent-loop mechanics I wrote up are already aging out from under that post. Problem formulation — translating a business's real problem out from under its stated one — depreciates slowly, because it is organizational translation, and organizations stay human. The trap is that the fast-depreciating half is the legible, demoable half. It feels like the edge because you can show it off. The durable half looks like "just asking questions."

Which leaves the fork, and I am going to leave it as a fork rather than resolve it for you. Stay the indispensable bottleneck and you extract rent: real money, now, capped at your personal throughput, gone the day the scarcity ends. Or codify the tacit half into artifacts a colleague can run without you — a written scoping method like the checklist above, reference patterns from the deployments that worked — and you dissolve your own bottleneck while capturing the value of having systematized it. Rent pays this quarter. The moat is owning the codified version when everyone else finally needs it.

The bubble being real is what creates the opening; nobody pays a premium for discipline in a calm market. The economics work out for whoever is on the right side of the rent-to-moat conversion. And the thing to be dismantling, deliberately, starting now, is the bottleneck being you personally — before the labs dismantle the cheap half of your edge for free and leave you holding only the part you never bothered to write down.

Torvalds' Best Review Trick Just Stopped Working

Michael Tuszynski — Thu, 16 Jul 2026 18:11:41 +0000

Linus Torvalds has spent years reviewing the most consequential codebase on Earth without reading much of the code. He said so himself in a 2020 interview with Tag1 Consulting: "While I still look at patches, I actually tend to look more at the explanations, and the history of how the patch came to me."

That habit was never laziness. It was the sharpest quality filter in software, and it rested on an economic fact: a coherent explanation of why a change belongs in the kernel was expensive to produce. You had to understand the subsystem, the failure mode, and the design decisions that shaped the current code. Faking the explanation cost more than having the understanding. So the explanation worked as proof of understanding, and the man at the top of the review pyramid could read intent instead of implementation.

At Open Source Summit India 2026 in Mumbai this month, Torvalds described the thing that broke that filter. He never framed it as broken. The quotes do it for him.

The Explanation Was Proof of Work

Tests can be gamed. Diffs can be pattern-matched from similar fixes without understanding either one. But for decades, a paragraph that correctly situated a change in the design history of a kernel subsystem could not be written by someone who lacked the mental model. Reviewers up and down the kernel hierarchy leaned on that correlation. Torvalds built his entire post-programming job on it.

The correlation was always a proxy, though. Nobody cares about the explanation itself. They care about the understanding it demonstrates, and Goodhart's law says any proxy works right up until it becomes a target. A proxy survives on the cost of faking it. This one survived for thirty years because the fake was more work than the real thing.

LLMs Made the Proxy Free

Large language models generate code that is sometimes wrong. They generate prose about code that is nearly always fluent. The explanation, the one artifact that used to be hardest to fake, is now the cheapest part of the submission.

Torvalds described what that looks like from the receiving end: bug reports that read as entirely valid and turn out to be fabricated. "It can actually be a huge drain on resources when it takes humans a lot of effort to figure out that, hey, this machine-generated report was not true," he said. Note the asymmetry: the report costs nothing to generate and real investigative work to disprove. That is a filter running backwards, taxing the reviewers it used to protect.

The patches carry the same signature. He called many of them "mindless band-aid kind of patches... they may fix the immediate problem, but the kind of bug remains, and it just is waiting in the hallway to hit you in another place." A band-aid patch in 2019 usually arrived with a thin, awkward description, and the description gave it away. The same patch in 2026 arrives wrapped in a confident, well-structured explanation of a root cause the author never found. The tell is gone. Prose quality no longer predicts anything about the understanding behind it.

The Fix He Announced Is a Provenance Requirement

Buried in the Mumbai remarks is the actual news, and it got almost no coverage: "If you find a bug with an LLM, it's not enough to just ask the LLM to make a bug report and then throw it over the fence to us. We want to see a suggested patch; we want to see the human who ran the LLM act as a kind of back-and-forth."

Read that as a submission requirement, not a complaint. He is no longer asking the explanation to prove anything. He is asking for evidence of the process: show me the dialogue, show me where you pushed back, show me what the model got wrong before you fixed it. The unit of review is shifting from the artifact to its provenance. For decades the kernel asked whether the description demonstrated understanding. The new question is whether the human can show their working relationship with the tool that produced it.

That is a bigger change than it sounds. The transcript of the back-and-forth, not the polished description sitting on top of it, is becoming the thing a maintainer actually wants to see.

Nobody's Tooling Captures This Yet

PR templates ask what changed and why. CI gates check tests, coverage, lint. None of them ask the question Torvalds is now asking: how do you know? The description field is where the fluent fake lives. The evidence of a genuine back-and-forth lives in session logs most teams throw away.

Closing that gap does not need a platform. A template section does most of it:

## Provenance
- Generated with: <tool/model, or "by hand">
- First-pass mistake I caught: <what you pushed back on>
- What I checked myself: <command you ran, output you read>

The middle field is the tell. A developer who worked the problem with a model fills it in ten seconds, because every real session has one. A developer who piped output over the fence has nothing to put there, and an empty field is a louder signal than a beautiful description.

I started keeping this kind of evidence before I had a name for it. When I published a self-improving agent demo earlier this month, the repo shipped with the actual run logs, a 13-of-15 test pass climbing to 15-of-15 across iterations, because without the logs the claim is indistinguishable from every other AI demo on the internet. The logs are the proof the loop ran. The README is just the description.

Where This Breaks

The obvious objection: transcripts can be faked too. Ask the model that wrote your patch to also write a plausible back-and-forth and it will produce one, complete with staged pushback. If teams start grading transcripts, Goodhart eats the transcript next. Grading them with an LLM judge inherits the same problem one level up.

The kernel's answer is already visible, and it is not paperwork. Torvalds has been blunt that the AI slop problem will not be solved with documentation, and kernel maintainers already deprioritize drive-by reports whose submitters won't answer questions. The gate is not the artifact. The gate is whether you can continue the conversation. A fabricated transcript buys exactly one round; the first follow-up question a maintainer asks lands on the human, in real time, with no model in the loop to sound fluent for them.

So Torvalds' test did not die in Mumbai. It moved. Reading the explanation stopped working, so he replaced it with watching you respond.

The next time a pull request lands in your queue with a suspiciously fluent description, run the kernel's version of the test. Ask one follow-up question the description does not answer, and watch the clock. The answer that comes back in five minutes was always the author's. The one that takes a day was written by whatever wrote the description.

Attention Is the New Bottleneck. Engineer It Like One.

Michael Tuszynski — Tue, 14 Jul 2026 16:43:39 +0000

The Diagnosis Is Right

Atomic Object published a piece last month arguing that with AI agents, attention is the new bottleneck. Agents made execution cheap. You can fire off four parallel coding agents before your coffee cools. What you can't do is review four streams of output at once, hold the context of each in your head, and catch the one that quietly wrote a migration that drops a column.

The diagnosis is correct. Simon Willison described running four parallel agents and being wiped out by 11am — not because the tools failed, but because he became the rate limiter. Michael Novati made the same point from a different angle: AI removed the production bottleneck and revealed the real one underneath — the human system that surrounds production. Everyone circling this problem is seeing the same thing. Execution went to zero and human attention became the scarce resource.

So the diagnosis holds. The prescription is where it falls apart.

The "Type Faster" Answer

Atomic Object's advice is all personal discipline. Automate trivial decisions. Plan for cognitive load. Take deliberate breaks. Budget your attention like a finite resource. Good advice for a person. Useless as a system.

We've seen this exact move before. When execution was the bottleneck — when developers spent their day typing, compiling, catching their own syntax errors — nobody's answer was "type faster." Nobody wrote a productivity blog telling engineers to be more disciplined about their keystrokes. We built compilers so you didn't hand-check types. We built linters so nobody argued about a missing semicolon. We built CI so the machine ran the test suite at 2am and told you which commit broke it.

The bottleneck moved and we answered with infrastructure, not willpower. Telling a developer in 1998 to concentrate harder on avoiding null-pointer bugs would have been absurd. Telling a developer in 2026 to budget their attention better is the same absurdity wearing new clothes.

Discipline doesn't scale. Infrastructure does. If your plan for the attention bottleneck is "I'll be more focused," you've already lost, because the failure mode of human attention isn't insufficient effort — it's that there's a finite amount of it and agents produce work faster than any amount of focus can absorb.

Engineer the Bottleneck Instead

Here's the reframe. Attention is a resource your system spends on your behalf. Most systems spend it wastefully — they make a human look at everything. The job is to build the parts that spend it carefully, the same way a good compiler spends your debugging time carefully by pointing at line 47 instead of making you read the whole file.

Four pieces do most of the work.

Verification gates so humans review exceptions, not everything. The default agent workflow asks you to eyeball every output. That's the waste. Instead, have the agent verify its own work and only surface what it can't confirm. My content pipeline ships blog posts across five channels. After it publishes, a verifyPublishedPost() step re-fetches the committed markdown and checks for the two failures that actually happened to me — a missing feature image, and a leading H1 that double-rendered the title. If it finds one, it fires a 🚨 line into my notification channel. If everything's clean, I hear nothing. I went from reading every published post to reading only the broken ones. That's not discipline. That's a gate.

Structured escalation so one line beats a log stream. Uptime monitoring is the canonical trap. You can watch a dashboard, or you can read logs, or you can have every alert route to a single channel with a single flagged line: what broke, where, since when. I don't read my nightly job logs. They write to disk. If a job fails, one message arrives. The difference between "check the logs" and "here is the one thing that needs you" is the difference between spending attention and saving it.

Trust tiers per task class. Not all work deserves the same scrutiny, and pretending it does is how you burn attention on things that don't need it. Low-stakes work ships on green checks — my content pipeline's Tier 1 curated shares go out on a passing lint gate, no human in the loop. High-stakes work queues for review. The on-demand blog publisher treats my explicit request as the approval and ships without a gate, because I asked for it. The cron pipeline that drafts on its own routes through an approval message first. Same infrastructure, different trust tier, matched to the blast radius of being wrong.

Decision logs so the same call never reaches a human twice. This is the one people skip, and it's the one that compounds. Every time you make a judgment call for an agent, write it down where the agent reads it. My project instructions carry a running list of hard-won lessons — never hand-author markdown into the blog repo, opt into Instagram explicitly because the publishing API returns false-success statuses, strip any leading body H1 unconditionally. Each of those is a decision I made exactly once. The agent now makes it every time without me. A decision log is a cache for judgment. Without it, you re-answer the same question forever, which is the most expensive way there is to spend attention.

The Objection Worth Taking Seriously

The honest counterargument: infrastructure has a build cost, and discipline is free today. Writing a self-verification step, wiring escalation, defining trust tiers — that's real work, and for a solo developer running one agent occasionally, personal focus genuinely is cheaper. Atomic Object isn't wrong that a human should also automate their own trivial decisions and take breaks.

But that's a bet on the problem staying small, and it won't. The whole premise is that agents multiply output. The developer running one agent this month runs six next quarter. Discipline that works at one stream collapses at six — that's the entire bottleneck being described. You pay the infrastructure cost once and it holds as you scale. You pay the discipline cost every single day, and it fails exactly when the load gets heavy enough to matter. Build the gate before you need it, because the moment you need it you won't have attention left to build it.

What Actually Separates the Winners

The framing that treats attention as a personal-productivity problem is going to age like every other "work smarter" answer to a structural constraint. It puts the burden on the operator to be superhuman, when the whole point of the last forty years of tooling was to stop requiring humans to be superhuman.

The people who win with agents won't be the ones with the most discipline about their attention. Discipline is a fixed, small, human quantity, and the workload is about to be neither fixed nor small. The winners will be the ones whose systems spend their attention the least — whose agents verify themselves, escalate the one thing that matters, ship low-stakes work without asking, and never bring the same decision back a second time.

Attention is the new bottleneck. So engineer it like one. We didn't beat the execution bottleneck by typing faster, and we won't beat this one by concentrating harder.