DEV Community: Michael Tuszynski

6 Agent Licenses, 1 Runtime: You Are Paying Per Seat for the Layer That Is Not Scarce

Michael Tuszynski — Mon, 07 Sep 2026 20:21:20 +0000

Six licenses. One runtime. That's the shape of most agent quotes on the table right now, and the invoice is inverted: the six licenses are the big line, and the runtime, the thing that actually consumes scarce compute, is either a rounding error or a separate purchase order nobody brought to the renewal meeting.

Microsoft published the split more clearly than anyone else did. The E7 bundle lists at $99 per user per month. To be fair to that number, it buys the whole suite, Office and Teams and the security stack, with agent capability folded in; it is the cleanest public example of the split, not a pure agent-seat price. SAMexpert's breakdown of the license puts the split plainly: that layer grants agent capability, it does not run them. Execution bills separately, in Agent Compute Units, through a pre-purchase plan sold in blocks. Two layers, two meters, one signature.

Treat that as the reference architecture for every agent quote you get this year, whoever the vendor is.

The seat layer and the runtime layer are priced by different physics

The interface layer is software. Marginal cost of a seventh seat: approximately zero. The vendor's cost doesn't move when you add a user who opens the panel twice a week.

The runtime layer is electricity, silicon, and a queue. Marginal cost of the ten-thousandth agent run is real, and it's the only part of the stack where the vendor is buying something scarce from someone else.

So when a quote grows by multiplying the free-to-produce layer and holds the expensive layer flat or unbundled, that's not a pricing accident. That's the whole design. You're being metered on the thing that isn't scarce.

Run the six seats: 6 × $99 = $594/month, $7,128/year, before a single agent executes anything. Now ask how many of those six drove an agent in the last 30 days. If it's two, one platform engineer and one person in support automation, the real cost per active agent user is $297/month, and the other four seats bought a login.

"Per seat" hides decisions your contract already made

The phrase does a lot of undisclosed work. Per-seat licensing is an umbrella covering separate decisions about what the unit attaches to, what time window counts it, and which type of user holds it. Vendors pick one combination, print two words, and let you assume you know which one you bought.

Two of those decisions bite agent deals in particular. Named-user counts never go down mid-term; concurrent counts can, so ask which one is in your paper. It's usually in a definitions exhibit, not the order form. And the service identity your agent runs under may be counted as a seat even though no human ever touches it. That one line can double a bill.

The runtime is getting cheaper faster than the seat is

The direction of travel is the part procurement keeps missing. Inference prices have fallen with every model generation, and a vendor's runtime rate card follows them down inside a contract term. Per-seat list prices move once a year, and not downward.

If a seat price was originally set to recover runtime cost, and runtime cost drops 60% over a term, the seat price should move. It won't unless you make it a renewal condition. Put a benchmark clause in: seat price adjusts with the vendor's own published runtime rate card, or you get the delta as runtime credits.

What the seat is actually buying, and why that argument doesn't survive contact

The strongest counterargument is real, so take it seriously: the seat isn't paying for compute at all. It pays for identity and SSO, audit logging, data-loss prevention, admin policy, connectors into your existing data, and a support contract with a throat to choke. Those are per-human costs. Charging per human is honest.

Fine. Then price them as such, and watch the argument come apart on its own mechanism.

Governance costs scale with directory objects and policy scope, not with who happens to use an agent. If audit and DLP are what the seat buys, they should survive the removal of the agent entitlement. So ask for that quote: the seat, priced without agent capability. Most vendors won't produce it, because the number would be embarrassing next to the one with the agent checkbox on.

The gap between those two quotes isn't governance. It's a right to consume, sold as a subscription. And a right to consume, when the consumption is metered separately anyway, is a toll booth on a road you already paid to drive.

The vendor's own bundle proves this. E7's seat layer explicitly doesn't execute agents. The execution meter is a different SKU with a different plan. Every argument that the seat covers runtime is refuted by the price page it came from.

The worksheet to take into the renewal

Split the quote before you sign it. It fits in your head. Do it on paper anyway, before you sign:

// renewal-worksheet.ts — split the bill by layer before signing
const quote = {
  seats: 6,
  seatPrice: 99,          // $/user/month — interface layer, ~$0 marginal cost to vendor
  activeAgentUsers: 2,    // drove an agent at least once in last 30 days
  runtimeSpendMonthly: 0, // metered execution, separate PO
};

const seatBill    = quote.seats * quote.seatPrice;              // 594
const perActive   = seatBill / quote.activeAgentUsers;          // 297
const runtimeShare = quote.runtimeSpendMonthly /
                     (seatBill + quote.runtimeSpendMonthly);    // 0.00

// If runtimeShare < 0.25, you are buying interface, not capability.

That last line is the test. When less than a quarter of your total agent spend is execution, the vendor has moved the money to the layer where scarcity doesn't apply, and your growth path is buying more logins rather than more work.

In the agent pipeline I run there is no seat at all. The interface is a terminal and a queue directory, and every dollar is metered execution: drafting this post cost $1.11 in model calls, and a second operator would have added nothing to the bill.

Where this argument breaks

Three places, honestly.

Small teams under ten people often can't get consumption pricing at all. Vendors gate metered plans behind enterprise minimums, and a flat seat price is the cheaper administrative outcome even when the unit economics are ugly. Take the seats and stop optimizing.

Regulated buyers who need contractual data residency, retention guarantees, and a named DPA are buying something the runtime layer doesn't include. That's a legitimate per-tenant cost that a per-seat model approximates badly but does cover.

And if your agent usage is genuinely uniform, every seat driving the runtime every day, the two models converge and the seat is just a prepaid meter with less paperwork. Usage is rarely that flat. The common shape is a few heavy operators and a long tail of people who opened the panel twice.

None of that changes the ask. Get the runtime quoted as its own line, in its own units, with its own rate card. Then look at the two numbers side by side.

Six seats at $99 is $7,128 a year. Ask your vendor what the runtime under those six seats cost last quarter. If they can't tell you, they're not selling you compute. And if they can, you finally know which of the two numbers to negotiate.

Fit Is a Claim You Have to Be Able to Test

Michael Tuszynski — Thu, 03 Sep 2026 02:08:21 +0000

Every rejection note has a house style. We've thought more about what this role needs, and it isn't the right match. Sometimes that's the honest summary of a real finding. More often it's a verdict from a process that never collected the evidence for it.

Fit is the most load-bearing word in startup hiring and the least disciplined one. It decides more outcomes than the take-home and the panel debrief combined, and unlike those it goes unwritten. Nobody signs their name to it. It shows up at the end, in the summary email, after a loop that tested something else entirely.

Nobody is a fit in the abstract

The claim always has four parts: this person, in this seat, under these constraints, against this bar. Drop one and you're describing a personality, which is neither testable nor any of your business.

Two versions of the same criterion. First: "works well with others." Second: "will carry the pager alone through a weekend for a system they didn't build, and will wake a colleague when the runbook is wrong rather than guess."

The first is a vibe. The second can be wrong, which means it can be checked. Build a round that produces evidence against it and you'll find out. There's no round you can build for "works well with others," which is why teams that lean on it end up grading candidates against whichever colleague they happen to resemble.

The rewrite costs an afternoon. Most startups skip it because the founders think they already agree on what the seat is. They don't, and the loop is where that surfaces.

If no round can test it, it isn't a reason

Every criterion in your fit definition gets a named round attached to it — the round that could produce evidence against it. If nothing on the schedule can do that, the criterion is an impression, and impressions don't get to reject people.

Interviewers aren't neutral instruments. Jason Dana, Robyn Dawes, and Nathanial Peterson ran the cleanest demonstration I know of. In the first of three studies, participants predicting other students' semester GPAs formed impressions just as confidently when the interviewee was answering with a random response system as when the answers were real, and they predicted worse than if they'd skipped the interview and used prior GPA alone. Watching an interview didn't hurt. Conducting one did. Their abstract ends on a single sentence: "Our simple recommendation for those making screening decisions is not to use them."

That's students predicting GPAs, not managers predicting job performance, and the gap matters. What transfers is the mechanism: a confident read built on noise, crowding out the information you already had.

Structure is the fix. Google reports that its structured loops predicted on-the-job performance across functions and levels, saved about 40 minutes per interview in interviewer time, and left rejected candidates 35% happier than those who went through an unstructured one. That's a company grading its own homework, so take it as a ceiling. The independent version is stronger than Google's marketing needs it to be. When Sackett and colleagues re-ran the personnel-selection meta-analyses and corrected for decades of overstated range-restriction adjustments, most selection methods lost validity. Structured interviews came out top-ranked anyway.

Sit with the rejected-candidates number for a second. People can accept losing to a bar. Losing to a mood is what they don't forgive.

You didn't fail by interviewing. You failed by grading a criterion nothing in the loop assessed. Reject someone on collaboration style after four rounds that only ever probed technical depth and you've written a conclusion with no data under it. "Fit" is the word that covers the gap.

The seat has to exist before anyone can miss it

The most common fit failure is a role-definition failure wearing a candidate's face.

Watch a fast-moving company open a seat nobody has held yet. Call it the first data engineer. One interviewer describes someone who builds the warehouse and owns the models end to end. Another describes a partner to the analysts, mostly unblocking their queries. A third thinks the job is finally making the dashboards trustworthy. All three are senior. All three are sincere. All three are describing different jobs. The candidate answers each of them accurately and then gets measured against a composite that exists in nobody's head.

This isn't only a hiring problem. Gallup's engagement indicator puts the share of U.S. employees who say they know what's expected of them at work at 49% as of May 2026. Role clarity is scarce for people who already have the job. Expecting it to show up spontaneously for a job that doesn't exist yet is optimistic.

Here's a cheap diagnostic, and a slightly brutal one. Before the req goes out, have every interviewer write one paragraph, separately: what does this person do in week six, who do they answer to day to day, what do they own that nobody else touches. Read the paragraphs side by side. If they don't converge, there's no seat yet. There's a wish, and nobody can fit a wish.

Your loop is the honest demo

Culture is behavior under constraint. A hiring loop is a constraint, so the loop is the most honest thing a candidate sees.

They read it as an operating model, because that's what it is. Rounds that appear and disappear mid-process. Headcount numbers that don't agree between two conversations a week apart. A decision-maker named early who never materializes. A timeline that quietly doubles. None of that reads as a scheduling problem — it reads as how you behave when you're busy. The people who pick up on it fastest are the ones with other options, which is a filter you didn't design and wouldn't choose.

If your loop can't hold its shape for six weeks under normal load, better to learn that before a real deadline tests the same thing.

Write the fit spec

The artifact is small: one table, agreed before the req goes live, a row per criterion. Here's one filled in for that first data engineer. The rows change per role. The shape doesn't.

Claim about the seat	What would disprove it	Round that tests it	Disqualifying signal
Ships against a charter nobody has written yet	Needs the spec settled before starting	Scenario round where the requirements contradict each other	Asks for the org chart before the problem
Makes the analysts self-sufficient instead of becoming their queue	Optimizes for doing the work themselves	Live teach-back on something they built	Can't explain the work to a non-specialist
Tolerates a role whose shape changes every quarter	Wants a fixed remit before committing	Direct question, asked the same way of every candidate	Treats a scope change as a broken promise
Picks a schema under incomplete information and defends it later	Waits for more data before deciding	Design round with a deliberately thin brief	Won't commit to a call under uncertainty

Then one governing rule. Rejection reasons must come off the table. If someone in the debrief raises a reason that isn't on it, either the spec was incomplete — add the row, re-run the round that tests it — or the reason wasn't real, and it doesn't get to decide.

I build this kind of thing for a living, though usually for machines. rubric-bench exists because an LLM scoring free-text answers is a production dependency: you pin a golden set of answers to known verdicts, re-score after every prompt edit or model bump, and fail the build when the verdicts move. Nobody would ship a grader that way and skip the tests. A hiring panel is the same object with worse instrumentation — a grader nobody regression-tests, running a rubric nobody wrote down.

The spec isn't free. Four rows with a round attached to each adds a week to a loop that a twenty-person company may not have. Published criteria are rehearsable: say "we test for scope pushback" out loud and you'll meet candidates who've practiced pushing back on scope. And a spec tight enough to be testable will screen out the atypical hire who would have redefined the seat. Pay it anyway. The alternative isn't "no spec." It's a spec living in four people's heads, applied at the end, where nobody has to defend it.

The counter-argument

Fit isn't fake, and treating it as pure bias overcorrects.

Lauren Rivera's study of elite professional service firms found that evaluators sought candidates culturally similar to themselves in leisure pursuits, experiences, and self-presentation, and that concerns about shared culture "often outweighed concerns about absolute productivity". Read closely, that's a finding about unstated fit rather than fit as such. Her setting was law, banking, and consulting, not early-stage startups, and it's a case study built from 120 interviews rather than a controlled trial. The mechanism travels anyway. A criterion nobody wrote down collapses into similarity-to-me, because that's the only yardstick left in the room.

Genuine style mismatches exist too. Someone who does their best work in a quiet queue will be miserable in a seat that's five customer calls a day, and hiring them is unkind to both parties. So style can be a real reason. What it can't be is unstated, because unstated fit is unfalsifiable, and unfalsifiable reasons attract every bias you have. Name the constraint in the job post and you've been fair. Discover it in round four and call it fit, and you haven't.

Back to the rejection note

Write the spec before the req, attach a round to every row, and forbid off-spec reasons in the debrief. That part's mechanical.

The sentence at the end is the harder part. Tell people what you actually decided against. "We wanted someone who's built a warehouse from nothing before, and we couldn't establish that from the loop" is a sentence a person can act on, and one you can only write if you did the work to know it. "Not the right match on either side" is what you send when you didn't.

Context Is Four Different Words: The Window, The Corpus, The File, and The Session

Michael Tuszynski — Sun, 16 Aug 2026 16:34:16 +0000

Someone on your team says the agent "needs better context." Four people nod. They are agreeing about a word and disagreeing about the work, and nobody finds out until two of them have built the wrong thing.

I run four systems that all get called context, and they have almost nothing in common. One is a hard capacity limit. One is a 35-million-token corpus. One is a 4.7KB file I hand-edited last week. One evaporates when the session ends. Same word, four owners, four failure modes, four different afternoons of work.

Here is how I keep them apart.

Job one: context as capacity

This is context as a number. The window. It has a ceiling, you can exceed it, and when you do, something gets dropped or summarized whether you like it or not.

Capacity is the only one of the four you can measure without judgment, which is why it dominates conversation. It is also the least interesting, because being under the limit tells you nothing about whether the right things are in there.

My own number is worth stating plainly. Before I type a single word, my agent has already loaded a global instruction file, a shared cross-tool rules file, a workspace guide, a memory index, and a lessons file. Together that is about 75,000 characters, call it 19,000 tokens, spent every session on standing orders. That is my floor, not my usage.

Job two: context as corpus

This is context as a pile of things you could retrieve. Vector store, RAG index, semantic search. It is not in the window; it is the universe you draw from to fill the window.

Mine is Postgres with pgvector, embeddings served by a local model:

select sum(token_count), count(*), round(avg(token_count))
  from chunks where status = 'active';
-- 35,035,979 | 159,669 | 219

Thirty-five million tokens of retrievable material against a window that holds a fraction of a percent of it. The binding constraint was never storage. It is selection.

The breakdown is the part that changed how I think about this. Of those 159,669 chunks, session transcripts account for 99,259. Archived history from two prior systems adds another 51,165. The hand-curated knowledge vault — the notes I actually wrote on purpose — is 2,515 chunks, about 1.6% of the corpus. The rest is exhaust.

That ratio is not a bug I am about to fix. It is what a retrieval corpus looks like after eighteen months: mostly a record of what happened, with a thin seam of what you decided.

Job three: context as instruction

This is context as a durable file you hand-author and version. CLAUDE.md, AGENTS.md, cursor rules, system prompts checked into the repo. It loads every session, it is read by a machine but written by a person, and it goes stale silently.

The artifact:

$ ls -la ~/.codex/AGENTS.md
~/.codex/AGENTS.md -> /Users/mpt/.claude/AGENTS.core.md

One 4.7KB file of rules — never commit secrets, never deploy without asking, batch privileged steps into one paste-able block — symlinked so that two different vendors' coding agents read the identical text. I edit one file; Claude Code and Codex both change behavior. That is context as configuration, and it has more in common with a dotfile than with anything in jobs one or two.

Its failure mode is drift, not overflow. A stale rule is worse than a missing one, because the agent follows it confidently.

Job four: context as session state

This is what is live right now: the conversation so far, what you already tried, the decision you made twenty minutes ago and have not written down. It is the only one of the four that dies on restart.

Everyone underestimates this one until a session compacts mid-task and the agent cheerfully re-litigates something it settled an hour ago. The fix is unglamorous. Write state to disk on purpose, or accept that it is gone.

The two axes nobody separates

Capacity and curation are not the same axis, and treating them as one is where the money goes.

	Small window	Large window
Curated input	Fast, cheap, works	Works, costs more than it needs to
Uncurated input	Overflows immediately	Fits, and quietly degrades

The bottom-right cell is the expensive one, because everything looks fine. It fits. Nothing errors. The output is just worse in ways that do not announce themselves.

Anthropic's engineering team makes this argument in their context-engineering post, framing the model as having an "attention budget" and defining good practice as finding "the smallest possible set of high-signal tokens." Take the framing seriously but note the position: the company describing your window as a scarce resource also sells the window. That is a claim worth checking against someone with no such incentive.

There is one. Liu et al., published in TACL, tested how models actually use long inputs on multi-document QA and key-value retrieval. Performance was highest when the relevant information sat at the beginning or the end of the input and dropped when the model had to reach into the middle — and that held even for models explicitly built for long contexts. That is peer-reviewed, it is not selling a context window, and it says position inside the window changes the answer. Capacity is not the variable. Placement is.

The widely-cited "context rot" work points the same direction, and the lab behind it sells a vector database, so "long contexts degrade, retrieve instead" is a conclusion they profit from. It survives the objection mainly because Anthropic cites it against its own product interest.

Where this breaks

The four-way split is vocabulary, and vocabulary has a tax. On a two-person team where the same person owns all four systems, this is overhead — go build something. The split earns its keep when the four jobs have different owners, which is exactly when the argument starts.

It also breaks at the edges. Prompt caching sits across capacity and instruction. A compaction step is session state being rewritten into capacity. I am drawing lines through something continuous because the lines make meetings shorter, not because the territory has them.

The question I ask now

When someone says we need better context, I ask which one before anything else: is this a room problem, a retrieval problem, a rules-file problem, or a state problem?

Room is a budgeting question. Retrieval is a ranking and evals question. A rules file is a twenty-minute edit. State is a persistence question. Four different afternoons, four different owners, one shared noun.

I got the shape of this from Shawn Wallace, who ran the same play on the word "agent" and found three jobs hiding under it. The form travels because the industry keeps doing this: a word gets useful, then absorbs every adjacent thing anyone shipped, then stops carrying information.

Writing this post, I ran the count against my own corpus and got 159,669 active chunks. My workspace guide — the instruction file, job three, the one loaded into every session I start — said 126,000. It had been verified eight days earlier. The file whose job is to describe the corpus was wrong about the corpus, and it had been telling every agent I ran the wrong number all week.

Job three going stale about job two. I had to write the taxonomy to catch it.

Karpathy's LLM Wiki Works. Here's What Broke When I Ran It at Real Scale.

Michael Tuszynski — Thu, 13 Aug 2026 18:38:58 +0000

Andrej Karpathy's LLM Wiki gist from April is right about the important part: for personal knowledge, a markdown wiki that Claude maintains beats a RAG pipeline. I've been running a version of this pattern in production against my own life for six months. Karpathy is right — and he stopped at exactly the point where the interesting problems start.

Here's the pattern, briefly, then what actually breaks.

The claim, in one paragraph

You keep raw sources in one directory. You keep a wiki of markdown pages in another. A CLAUDE.md file tells Claude how the wiki works: naming, folder layout, what to do on ingest, how to handle contradictions. New source comes in, Claude reads it, writes or updates concept pages, links them with [[wikilinks]], updates an index. When you ask a question, Claude reads the wiki, not the raw sources — the synthesis has already been done at ingest time. The whole thing sits under ~100k tokens, so Claude can hold the index in context and reason over it directly. No embeddings, no vector store, no infrastructure. Two clean community implementations to start from: nvk/llm-wiki and Ar9av/obsidian-wiki.

That's the whole thing. It works. It works better than you'd expect. And every problem I've hit with it shows up in the same three places.

My setup, so the failure modes are grounded

Mine is a NEXUS workspace: an Obsidian vault split into 000-inbox/, 200-knowledge/, 300-entities/, 400-daily/, plus per-domain context files under 200-knowledge/context/ — one each for finance, health, content, jobsearch. Every domain has its own schema and its own conventions. A top-level CLAUDE.md sets the operating rules; each domain file specializes them. Entities (people, companies, services, properties) are single markdown files named by their wikilink text, so [[Janney]] grep-searches straight to vault/300-entities/orgs/Janney.md. Daily logs at vault/400-daily/YYYY-MM-DD.md are the append-only session record. A MEMORY.md at the root holds hard-won lessons — 34 entries and counting.

Volume: several hundred wiki pages, four active domains, daily writes going back to February. Well past Karpathy's toy scale, well under any RAG-scale problem. This is the middle band the original gist never addresses.

Where it broke: three failure modes, all the same shape

1. Drift between the "recent changes" section and the "how it works" section of the same file.

The gist assumes ingest and update. It doesn't say what happens when something you documented in April is contradicted by what you did in July. In practice, Claude appends a new dated entry to a changelog at the top of the file, feels done, and leaves the reference section below still describing the dead system as current. A reader — human or agent — trusts the reference section for "how does this work today," and it's wrong. Writing a dated log entry feels like recording the change, so the present-tense prose that people actually read for "how does this work now" never gets revisited. The rule I now enforce: a migration isn't done when the changelog says so; it's done when the reference sections match.

2. Silent contradictions between pages.

Two entity cards for the same account — one calls the institution "Janney," the other calls it "Janney Montgomery Scott LLC" — and a downstream aggregation double-counts because the dedup key is the name. RAG failures are silent because retrieval quietly misses a chunk. Wiki failures are silent for a different reason: two pages both exist, both are internally consistent, and neither knows about the other. Claude doesn't spontaneously notice.

3. Orphan pages and dead wikilinks accumulate faster than lint catches them.

Karpathy's original suggests running lint occasionally. At three hundred pages, "occasionally" is not a frequency — it's a hope. Weekly lint recovers most of it. Nothing recovers a stale page that nobody links to and nobody re-reads.

All three are the same shape: the wiki has no immune system against its own past. It compounds knowledge, but it also compounds staleness, and it needs help distinguishing the two.

Three things I bolted on

A drift linter that fails the build. A Python check that walks the vault, cross-references any "retired" or "decommissioned" term against every file, and flags live mentions of dead systems in the canonical docs. It runs pre-commit and on a scheduled cron. When Ghost got replaced by an Astro static site, the linter caught nine files still describing the Ghost publish path as current — including the top-level project README. Without it, a reader would have reached for dead credentials.

Per-domain sub-schemas, not one giant CLAUDE.md. The gist treats the schema as one file. That doesn't survive the second domain. Finance and health have almost no overlap in vocabulary, conventions, or what a "page" is. My top-level CLAUDE.md sets universal rules (voice, safety, wikilink format, verification protocol). Each domain has its own <domain>-context.md that specializes them. A session working on finance loads the domain file; a session working on content loads a different one. Same architecture as scoped CSS.

A vector index alongside the wiki, not replacing it. Once I passed roughly 100k tokens of wiki content plus 100k tokens of session logs plus tens of thousands of legacy conversations, Claude couldn't hold the index in context anymore. The gist's answer to this is "at that point, use RAG." My answer is: keep the wiki as the authored, curated layer, and add a pgvector index over the whole corpus — vault plus session history plus legacy archives — for recall. Around 126k chunks on Postgres 17 with mxbai-embed-large embeddings running on a Mac mini. Not a replacement for the wiki; a way to find the right wiki page when your brain forgot which slug you wrote it under two months ago. The RAG layer is a retrieval tool for me and my agents. The wiki is still the canonical knowledge.

Where the pattern still wins

For synthesis, entity resolution, and the "how do these three things I noticed relate" question, nothing else comes close. RAG genuinely doesn't do it — the structural argument about retrieval having no accumulation is correct. When I ask "what's the through-line across the last month of daily logs on finance," the answer draws on synthesis pages that were written when each log came in. The work happened once, at ingest.

Entity cards are the other win. A single file per person or company, named by its wikilink text, becomes the source of truth. Everything else references it. When a fact about an entity changes, you change one file. This is the boring pattern that pays for itself every day.

Where it stops — the honest edge cases

The wiki pattern breaks when any of these are true:

Multi-actor writes. If two agents write to the same wiki page concurrently, there is no lock. Mine is single-writer by convention. If you fan out coding agents that all edit SESSION-STATE.md, you need a lockfile — the gist has nothing to say about this.
Recall across the boundary of a Claude session. A wiki isn't memory. Claude reads it fresh each session. Cross-session memory ("what did we decide last Tuesday") needs a separate store, which is what daily logs plus the vector index are for.
Anything past ~100k tokens of hot wiki content. The load-bearing assumption is that the index fits in context. When it doesn't, either you shard by domain (my approach), add a retrieval layer, or accept degraded answers.
Anything anyone else needs to read without Claude in the loop. A markdown wiki is legible to humans. A markdown wiki written by Claude, for Claude, over months has enough house-style shorthand that a new reader without the schema will miss context. The wiki is legible; the accumulated conventions are not.

What to actually do

Start where Karpathy says. One folder, one schema, five sources, in one domain you're actively thinking about. Do it for two weeks before adjusting anything.

At the point it starts feeling like it's working — around thirty or forty pages — put three things in before you regret not having them: a drift check that flags dead terms in live docs, one lint pass a week that actually runs, and a rule that any migration updates the reference section, not just a changelog. That's the operating discipline the pattern needs and doesn't ship with.

Add a vector index only when the wiki genuinely stops fitting. Not before. Karpathy is right about that too — most people reach for RAG when they should have written a schema.

Your $40K GPU Runs 6 Hours a Day: The Utilization Number Every Local-LLM TCO Omits

Michael Tuszynski — Mon, 10 Aug 2026 15:55:49 +0000

Nobody buys a $40,000 GPU and then asks how many hours a day it's actually computing. They ask how many tokens per second it can do at peak, divide the purchase price by a made-up annual token volume, compare that to an API price sheet, and put the winning number on a slide.

The number on that slide is wrong by roughly an order of magnitude. Not because the arithmetic is bad — because it's missing a term.

The term everyone omits

Per-token pricing and per-hour ownership are different units. An API bills you for work performed. A GPU bills you for time elapsed, whether work happens or not. To compare them you need a conversion factor, and that factor is utilization.

Utilization isn't one number, it's two multiplied together:

Duty cycle — what fraction of wall-clock hours the box has any inference in flight. A GPU serving an internal dev team runs during business hours in one timezone. Call it 6 hours a day, five days a week. That's 30 hours out of 168. Duty cycle: 18%.

Saturation — during those busy hours, what fraction of peak throughput you actually sustain. This is where the damage is. An H100 running a 70B-class model with continuous batching wants 25 to 40 concurrent requests to hit its throughput ceiling. Interactive users generate three or four. You're running at maybe 20% of what the silicon can do, while paying for 100% of it.

Multiply them. 18% × 20% = 3.6% effective utilization. That's the conversion factor between your capex and your token bill, and virtually no procurement model contains it.

I've measured this before on much smaller hardware — a Mac mini doing local inference for a real workload sat busy 1.7% of the time over three weeks. The absolute number scales with the box. The shape doesn't.

Running the math both ways

Take a single-GPU node, honestly costed. $40K for the card, three-year straight-line depreciation: $13,333/year. Power at a 1.4 kW node draw and $0.15/kWh commercial rate: about $1,840/year running flat out. Rack and connectivity in a colo: $2,400. Ten percent of one platform engineer to own the thing — patching, driver hell, model upgrades, on-call: $18,000 at a fully loaded $180K.

Annual owned cost: ~$35,500. The staffing line is the biggest single item and it's the one most spreadsheets leave blank. The operational bleed around open-weight deployment is not the GPU. It's everything attached to it.

Now throughput. Assume that node sustains 2,000 output tokens/second aggregate under healthy batching. At 100% utilization that's 63 billion tokens a year.

Effective utilization	Tokens/year	Cost per M tokens
100%	63B	$0.56
25%	15.8B	$2.25
10%	6.3B	$5.63
3.6%	2.3B	$15.65

At full tilt the box is spectacular — half a dollar per million tokens beats every commercial endpoint on the market. At the utilization your team will actually produce, it costs more than Claude Sonnet. Same hardware. Same model. Same electricity bill. The only variable that moved was the one nobody modeled.

Put a percentage on the slide, not a payback date

Here's the reframe I'd argue for, and it fits in one line of a deck.

Stop computing break-even as a token volume. Compute it as a break-even utilization — the fraction of peak throughput you must sustain for the owned box to beat the API you'd otherwise call.

U* = annual_owned_cost / (api_price_per_Mtok × peak_Mtok_per_year)

For the node above, against a $3/M blended API price: $35,500 / (3 × 63,000) = 18.8%. You need to hold nearly a fifth of peak throughput, all year, to win.

Against a $0.60/M hosted open-weights endpoint, the same node needs 94% utilization. That's not a hard target. That's a physically unreachable one for anything but a batch queue.

U* is better than a payback date for three reasons that matter to the person approving the purchase: it's a single number, it's measurable before you buy, and it's falsifiable. "We break even in 14 months" can't be checked until month 14. "We need 19% sustained utilization" can be checked next Tuesday against a week of Prometheus data.

If you're already serving inference somewhere, you can measure both terms today:

# duty cycle — fraction of the last 7d with inference in flight
avg_over_time((DCGM_FI_DEV_GPU_UTIL > bool 5)[7d:1m])

# saturation — how full the batch actually runs (vLLM)
avg_over_time(vllm:num_requests_running[7d]) / <max_num_seqs>

Multiply. That's your real conversion factor. Divide your annual owned cost by it. Then compare.

The denominator keeps moving

There's a second problem, and it's worse than the first. U* isn't a constant. It rises over the life of the asset, because the API price in the denominator keeps falling.

Epoch AI's analysis of inference pricing found that the cost of reaching a fixed capability level has been dropping between 9x and 900x per year depending on the task. Take the conservative end. A 9x annual decline means the $3/M endpoint you benchmarked against is a $0.33/M endpoint a year later.

Your break-even utilization went from 19% to 171%. In year one. Your depreciation schedule did not move.

That's the structural problem with capitalizing inference hardware: you're making a three-year fixed commitment against a price curve that resets quarterly. The academic cost-benefit work on on-prem deployment generally models break-even against today's commercial prices. Fine as a snapshot. Dangerous as a purchase justification, because it treats the comparison baseline as static when it's the fastest-moving number in the stack.

What I'd actually do

Measure before you buy. One week of duty-cycle and saturation data from a rented instance running your real traffic costs a few hundred dollars and answers the question the spreadsheet is guessing at. Rent the exact GPU for a month first. Every cloud has them by the hour.

If your measured U is below 40%, don't buy the box for cost reasons. There aren't any. Route the interactive traffic to an API and keep looking.

If you have batch work — nightly document extraction, embedding regeneration, eval suites, synthetic data generation — that's the workload that fills the trough. Owned hardware pays off when you can schedule against it, because scheduled work is the only kind that saturates. A queue that runs 2am to 6am at full batch depth does more for U* than doubling your user count.

And if the real driver is data residency, a contractual requirement, or a latency floor you can't hit over the public internet — buy it. Those are good reasons. Just price the utilization gap and call it what it is: a control premium, with a dollar figure next to it. That's a defensible line item. "We'll save money on tokens" is not, and it falls apart the first time someone opens Grafana.

The finance team will accept a premium they can see. What they won't forgive is a payback model that assumed 63 billion tokens and delivered 2.3 billion.

Local LLM vs Cloud API at 12 Months: The 3 Line Items Every Per-Token Comparison Omits

Michael Tuszynski — Mon, 10 Aug 2026 00:49:28 +0000

I run a dedicated local inference host. A Mac mini M4 Pro, 14-core CPU, 20-core GPU, 64GB of unified memory, bought open-box for $3,349. It serves embeddings for a retrieval system, a fallback chat model, and a classifier for inbound notes. It has eight models resident, about 140GB on disk.

Over the past three weeks it has been busy 1.7% of the time.

That number comes from its own request log: the summed wall-clock duration of every generate, chat, and embed call, divided by the day. On the heaviest day in the window it hit 3.45%. On several days it did no inference at all. The daily request count looks healthy at roughly 1,440, until you notice that 1,440 is exactly one request per minute: an uptime monitor polling /api/version. The machine's most reliable customer is a health check.

I would still buy it again. But it broke my own cost model, and the way it broke is the part worth writing down.

The month-0 spreadsheet is not the month-12 bill

Almost every local-versus-cloud comparison, including the one I wrote last year, is a snapshot. You price the hardware, price the tokens, divide, and find a break-even volume. That math is fine on the day you run it. It quietly assumes three things that stop being true somewhere around month four.

The gap is not small. McKinsey's July 2026 survey found 93% of respondents exceeding their AI budgets while 62% had moved past experimentation into active deployment. McKinsey puts 20 to 30% of that spend within reach of better accounting. That is a survey of self-reported budgets, so treat the precise figure as directional. The direction is the interesting part: the overruns arrive after deployment, not during the pilot. These are the three line items that show up in that window.

Line item 1: the capacity you bought, not the capacity you used

Cloud inference bills you for work performed. Hardware bills you for work available. At 1.7% utilization those are different products wearing the same label.

Run my numbers forward. Roughly 0.4 busy hours a day is about 146 hours of actual inference a year. Amortize the $3,349 over a single year and each hour of real work costs $22.90. Stretch it across three years, which is fairer to the hardware, and it is still about $7.60 per busy hour. Electricity barely registers. The mini idles near 7W and peaks around 45W under load, so 8,600 idle hours plus 146 busy ones comes to roughly 67 kWh, about $23 a year at California rates.

That last figure is worth pausing on, because my earlier post led with cooling and power draw. At rack scale, with 700W GPUs, that emphasis is right. At the scale most teams actually start at, one box serving one team, power is a rounding error and utilization is the entire story. The line item that matters flipped when the deployment got smaller.

The honest version of the local-inference pitch is not "tokens are cheaper." It is "I am buying a fixed monthly cost and a latency floor, and I will pay it whether or not anyone sends a request."

Line item 2: the cloud price is a moving target and your hardware is not

You commit to hardware at a fixed price. The alternative you rejected keeps getting cheaper underneath you, and not evenly.

Epoch AI's analysis of inference pricing found that prices to reach a fixed performance level have fallen between 9x and 900x per year, depending on which capability you are buying. Matching GPT-4's performance on PhD-level science questions got about 40x cheaper per year. General-knowledge performance fell more slowly, in the 9x-to-40x band. Epoch adds a caveat most people quoting these curves skip: the fastest declines happened in the most recent period, so there is no guarantee they persist.

Two consequences for a 12-month decision. The break-even volume you computed in month 0 moves against local hardware every month you own it, because your capex is sunk while the rented alternative reprices. And the rate it moves depends on your workload, because the decline is task-dependent. A team doing hard reasoning has seen its cloud alternative collapse in price. A team doing bulk classification has seen a gentler slope, and their local box holds its case longer.

Nobody's model has a row for this. Build one: re-run the comparison quarterly with current API pricing rather than the pricing you started with.

Line item 3: you pay per token and you bank per accepted answer

The third omission is the denominator. Both sides of this comparison are usually priced per million tokens, which measures what the model emitted, not what a human kept.

I made this argument in isolation two weeks ago and it landed flat, because an argument without a table is just an assertion. Here is the table. A smaller local model that needs two attempts where a frontier API needs one has doubled its effective token cost, and it has spent your reviewer's attention twice. Reviewer attention is the expensive input here, not the tokens. Tracking where the token spend actually goes is the prerequisite; the acceptance rate is what turns that into a cost.

Line item	Cloud API	Local hardware	How to measure it
Idle capacity	~$0 when idle	Full cost regardless of use	Busy-seconds ÷ elapsed, from your server log
Price decay	Falls 9x–900x/yr, task-dependent	Fixed at purchase; capex is sunk	Re-price the API side quarterly
Rework	Per token, retries billed	Per token, retries billed in idle capacity too	Accepted outputs ÷ total generations

Three rows, all measurable from data you already have. None of them appear in a per-token comparison.

Where this breaks

My 1.7% is a single-operator homelab, and it is the weakest possible case for owned hardware. A shared inference cluster behind a real queue, serving fifty engineers across time zones, lands far higher. Above roughly 40% sustained utilization the capacity argument inverts and hardware wins on cost alone. If your box is genuinely saturated, line item 1 is not your problem.

Two other cases where none of this decides anything. Data residency and compliance can make local inference the only lawful option, at which point the cost comparison is a formality. And batch or overnight work reshapes the math, because idle hours you can schedule into are not idle.

There is also a cost to my own advice. Measuring utilization honestly makes the case against a machine I own and use daily. I kept it, for reasons that never appear in a TCO model: no rate limits, no per-request thinking, models that stay put, and a place to run things I would not send to an API. Those are real. They are just not "cheaper," and calling them cheaper is how a $3,349 box turns into a line item somebody has to defend at renewal.

Go pull your own number first. If you run Ollama, your utilization is sitting in ~/.ollama/logs/server.log right now, and it takes one awk to find out whether you bought a workhorse or a very expensive uptime-monitor endpoint.

Your Local LLM Bill Is Per Token. Your Real Cost Is Per Accepted Answer.

Michael Tuszynski — Wed, 05 Aug 2026 20:26:19 +0000

Finance will hand you a per-token comparison. Local inference at some fraction of a cent per thousand tokens, the frontier API at some multiple of that, and a delta that looks like a budget line worth defending. The arithmetic is fine. The denominator is wrong.

Tokens are a throughput unit. Nobody buys tokens. You buy answers a human is willing to ship, and the ratio between those two things is not a constant — it moves with model size, task class, and how many attempts it takes to land one. The moment you divide by accepted answers instead of tokens, most local-versus-cloud spreadsheets stop saying what they said.

I priced the numerator side of this last year — datacenter GPUs, power, cooling, and the DevOps tax of running your own inference. That post assumed the denominator. This one goes after it.

Five things the per-token quote silently drops

Retries. A 32B model that gets it right 70% of the time on the first pass costs you 1.43 generations per accepted answer, not 1. Every failed attempt burns the same tokens as a successful one and produces nothing.

Escalation. Some fraction of tasks the local model cannot finish at all. Those tasks pay the local cost and the frontier cost. If you don't measure that rate, you're carrying it as an invisible surcharge on the cheap path.

The review pass. This is the big one and it never appears in a token quote. A weaker model shifts work onto a person, and a person costs three to four orders of magnitude more per minute than the accelerator does.

Idle time. Utilization, not throughput, sets unit cost on owned hardware. The box depreciates whether or not anything is in the queue.

Amortization and refresh. Per-token framing treats hardware as free after purchase. It isn't. It's a fixed cost divided by however many tokens you actually generate before you replace it.

What my own box costs, and how I got there

I run a dedicated inference host: a 64GB M4 Pro Mac mini serving llama3.3:70b, qwen3.5:27b, qwen2.5-coder:32b, and deepseek-r1:32b through Ollama. Everything below is measured on that machine, and every assumption is on the table so you can redo it with yours.

Assumptions: $2,000 acquisition, 36-month refresh window, roughly 10W idle and 45W under sustained generation at the wall, about 8 output tokens per second on the 70B at 4-bit quantization. Electricity at $0.31/kWh — that's the rate used in a good watt-by-watt measurement of Apple silicon inference, which also priced per million output tokens and then re-ran the numbers against 30 days of real traffic rather than benchmark loops. Substitute your utility rate.

At my actual duty cycle — about 90 minutes a day of real generation, 45 hours a month:

Amortization: $2,000 ÷ 36 = $55.56/month
Power: 2.03 kWh generating + 6.75 kWh idling = 8.78 kWh = $2.72/month
Output: 45h × 3600s × 8 tok/s = 1.296M tokens
≈ $45 per million output tokens

Now saturate the same box. 24/7 generation, 720 hours, 20.7M tokens, $10.04 of electricity. Same amortization, same tokens per second, same model weights.

≈ $3.17 per million output tokens.

Fourteen-fold swing in unit cost with zero change in hardware or performance. The variable was utilization. This is why an H100 cluster and a Mac mini can both be "cheap per token" on a slide and neither number survives contact with a real request pattern — the cluster amortizes a far larger fixed cost and needs far higher sustained load to get there.

The formula to bring to the meeting

Cost per accepted answer:

CPAA = (A × t_l × c_l) + (e × t_f × c_f) + (m ÷ 60 × W)

A   = attempts per accepted answer (1 ÷ first-pass acceptance rate)
t_l = output tokens per local attempt
c_l = your local cost per token (amortization + power ÷ tokens actually generated)
e   = escalation rate to a frontier model
t_f = output tokens on the escalated call
c_f = frontier rate per token (your rate card)
m   = human review minutes per accepted answer
W   = loaded hourly cost of the reviewer

Worked, with my $45/M and plausible task numbers. A = 1.4, t_l = 900 tokens, e = 0.15, m = 4 minutes, W = $120/hour loaded. For c_f I'll use a $15/M output placeholder — plug in whatever your contract says.

Local tokens: 1.4 × 900 × $0.000045 = $0.057
Escalation: 0.15 × 900 × $0.000015 = $0.002
Human review: 4 ÷ 60 × $120 = $8.00

$8.06 per accepted answer. The token lines are 0.7% of it.

Run the same task class entirely on the frontier model. Acceptance goes up, so A drops to 1.15, e goes to zero, and review drops to 2.5 minutes: $0.016 in tokens plus $5.00 in review. $5.02.

The local box wins the token comparison by about six cents and loses the decision by three dollars. Ninety seconds of extra review per answer is worth roughly fifty times the entire per-token delta at my volumes.

The honest counterargument: m is not fixed. On narrow, well-bounded work — commit message drafting, log triage, structured extraction against a fixed schema — a 32B coder model produces output a reviewer skims rather than audits, and m is identical on both paths. When that's true the local box wins outright and the math is not close. My point isn't that local always loses. It's that the term deciding the outcome is one nobody measures, and the term everybody argues about rounds to noise.

Measure m. Measure A. Measure e. Then argue.

Keep escalation a config line, not a rewrite

None of this is measurable if the model choice is compiled into your application. You need the router in front, and the routing decision has to be data.

That's the pattern in inference-router — an OpenAI-compatible layer that sits between the app and both backends, so which model serves which task class is a config change and every escalation is a countable event. Point the app at one endpoint. Move a task class from qwen2.5-coder:32b to a frontier model by editing a file, watch CPAA, move it back if the review time didn't drop.

Without that, e is unknowable and you're arguing from vibes. With it, escalation rate is a metric you can put on a dashboard next to acceptance rate, and the local-versus-cloud question becomes a per-task-class answer instead of a religious one.

While you're in there, cap output length. One team traced a large share of their bill to uncontrolled, conversational output tokens — verbose responses cost the same whether the extra tokens help or not, and on owned hardware they cost you queue time too.

Buy local for the reasons that hold up

There are three defensible reasons to run inference on hardware you own: you need the data to never leave your boundary, you need latency a network round trip can't give you, or you need to keep serving when a vendor changes a model version out from under you. All three are real, and none of them require the token math to work out in your favor.

"It's cheaper per token" is not on that list. It's the claim finance will test first and the one most likely to fall apart, because it's the only one that depends on a denominator nobody is tracking.

My mini pays for itself on data residency. At $45 per million output tokens and a 4% duty cycle, it does not pay for itself on price, and I'd rather say that out loud than have someone else find it in a spreadsheet.

The 26% Who Never Rolled Back an Agent Aren't Winning

Michael Tuszynski — Wed, 29 Jul 2026 00:00:48 +0000

Somewhere right now there's a slide in a board deck claiming a clean record: zero agent rollbacks since launch. It reads like a win. It's closer to a smoke detector with the battery pulled out.

New survey data from Sinch puts the number at 74% of enterprises that have rolled back a deployed AI agent after it went live — customer-communications agents specifically, pulled over governance failures. That figure got picked up everywhere as an indictment — three out of four agent deployments blowing up in production. But the interesting number is the one underneath it. Among organizations with the most mature guardrails, the rollback rate goes up, to 81%.

Read that again. Better safety infrastructure correlates with more rollbacks, not fewer.

Rollback rate measures your eyes, not your agent

There's only one clean way to explain that inversion. The teams with better instrumentation aren't shipping worse agents. They're catching things the other teams are shipping past.

Every agent in production is doing something wrong at some rate. Hallucinated policy details, tool calls against stale records, a tone that drifts on the eighth turn of an angry conversation, a refusal loop that pushes a customer to a human queue that's already 40 deep. The question was never whether the failure exists. It's whether anything in your stack notices it before your customer does.

So the 26% split into two very different groups. A small number genuinely nailed scope — narrow task, tight tool surface, a human approving anything consequential. The rest have no detector. Their agent is failing at whatever the base rate is, silently, and the absence of a rollback is being reported upward as quality.

SRE teams learned this lesson two decades ago and it stuck: an incident count is a function of your monitoring, not your reliability. A team that goes from 3 incidents a quarter to 30 after installing real alerting did not get 10x worse. They got 10x more honest. It's the same principle the write-ups on this data keep circling — the model rarely breaks, the infrastructure around it buckles. Which means model choice isn't what separates the 26% from everyone else.

Treating rollback rate as a failure metric creates the exact incentive you don't want. A VP who gets graded on rollbacks will not build better agents. They'll build quieter ones — loosen the eval thresholds, downgrade the alert to a weekly digest, route the complaint channel to a shared inbox nobody owns. The metric improves. The agent doesn't.

The guardrail tax nobody budgeted for

The second number in that survey is the one that should reset your staffing plan. 84% of AI engineering teams spend at least half their time on safety infrastructure rather than on the agent itself. And enterprise investment now skews toward trust, security, and compliance (76%) over AI development proper (63%).

That's not overhead creeping in at the edges. That's the majority of the work.

It also matches what production looks like. The agent is a prompt, a model call, and a tool list — a week of work for a competent engineer. The other eleven weeks go to the eval set, the PII redaction layer, the escalation path, the audit log that survives a compliance review, the shadow-mode comparison, the per-version rollback switch, and the on-call rotation that owns all of it at 2 a.m.

Microsoft's 2026 Work Trend Index frames the organizational side of this as a capacity question — whether organizations are built to capture the agency that agents free up. Most aren't, and the guardrail tax is a good part of why. The headcount that was supposed to move up the value chain is instead maintaining the machinery that keeps the agent honest.

Who paid for this, and what it actually covers

Now the part the LinkedIn reposts skipped.

Sinch is a CPaaS vendor. They sell customer-communications infrastructure. A survey they sponsored concluding that infrastructure quality predicts agent success is a survey concluding that you should buy more of what Sinch sells. That doesn't make the data wrong, but it does mean the framing was chosen before the responses came in.

The scope is narrower than the headline suggests, too. This is customer-communications agents — support, messaging, conversational flows. Not coding agents, not internal RAG assistants, not the finance-ops bot reconciling invoices. A customer-facing agent has an unusually loud failure mode: the customer complains, and the complaint is logged in a system somebody already reads. Detection is close to free. In a domain where the agent's mistakes land quietly in a document nobody re-reads for six weeks, the 26% number would almost certainly be higher — and mean even less.

Hold the core finding anyway. The correlation between guardrail maturity and rollback frequency is directionally strong enough to survive the sponsorship discount, because it points the opposite direction from the sponsor's simplest sales pitch. "Buy our stuff and you'll roll back more often" isn't a slogan anyone reverse-engineers into a survey.

Instrument the rollback, then go trigger one

Stop reporting rollback count. Start reporting rollback provenance. One row per deployed agent version, and the field that matters is who noticed:

release_id      agent-support-v14
model           claude-sonnet-5
prompt_sha      a91f3c2
tools_sha       7de0b41
deployed_at     2026-07-14T09:12Z
rolled_back_at  2026-07-16T23:41Z
detector        eval_regression | slo_breach | abuse_signal
                | agent_self_report | human_complaint
time_to_detect  3410 minutes

Now run the distribution on detector. If 80% of your rollbacks say human_complaint, your detection layer is your customers, and your time_to_detect is however long it takes an annoyed person to find the contact form. Whatever that interval turns out to be in your own data, multiply it by your conversation volume — that's how many exchanges you can't un-send.

Three things follow from that ledger.

Set a target on time-to-detect, not on rollback count. An hour is achievable with automated eval replay on a sampled slice of live traffic. Days is what you get when complaint volume is the detector.

If you've never rolled back, go cause one. Inject a deliberately degraded prompt into a canary slice — drop a rule from the system prompt, or point a tool at a stale index — and time how long your stack takes to flag it. A rollback path that has never been exercised is a hypothesis, not a control. Same logic as restoring from backup: nobody has a backup, they have a restore they've tested or a file they hope is fine.

And when a vendor pitches you an agent, ask how many times they've rolled back their own. A confident zero means they aren't looking. The right answer sounds like "four times last quarter, median detection 22 minutes, here's the ledger."

The 81% cohort isn't failing more often. They're the only ones who can prove what their agent did last Tuesday at 3 a.m.

AI Use Is a 50/50 Coin Flip. Coding Already Tipped 61% Autonomous.

Michael Tuszynski — Thu, 23 Jul 2026 20:08:27 +0000

The Copilot Framing Is Already the Minority Report

Every vendor deck for a coding tool sells you the same picture: a developer in the driver's seat, the AI riding shotgun, a human hand on every merge. Copilot. Human-in-the-loop. Pair programming with a machine. It's the comfortable frame because it keeps the person central and the tool subordinate.

For software development, that frame already describes the smaller half of what's actually happening.

Anthropic's Economic Index publishes real data on how people use Claude, and it tags every conversation one of two ways. Augmentation means the person stays actively in the loop — back and forth, iterating, learning, steering each step. Automation means the person hands off a task and directs Claude to complete it. Across everything, it's nearly a coin flip: 51 percent augmentation, 49 percent automation globally, and a clean 50/50 in the US.

Narrow to software development tasks and the coin lands differently. 39 percent augmentation, 61 percent automation. The pattern everyone quotes in their pitch is the one coding has already moved past.

What the Split Actually Measures

Read the label carefully, because it's easy to turn this into something it isn't. Augmentation versus automation is a conversation style, not a jobs claim. It says nothing about displacement, headcount, or whether a role survives. It measures one thing: in a given exchange, is the human collaborating turn by turn, or delegating the whole task?

That distinction matters because the two styles want different products underneath them. An augmentation session is a dialogue — the tool's job is to be responsive, explain itself, and stay legible while a person drives. An automation session is a delegation — the tool's job is to run to completion and come back with something you can check.

Most coding tools are still built for the first job. The interface assumes you're watching, approving diffs one at a time, keeping a hand on the wheel. When 61 percent of the actual work is delegation, that assumption is backwards for the majority of what your developers are doing.

The Number That Keeps This Honest

Here's the counter-weight, and it's in the same dataset. Software development is only about 11.5 percent of global Claude requests, and 8.1 percent in the US. Coding is not the center of gravity for how people use these models. It's a slice.

That cuts against the temptation to read "61 percent automation in coding" as "AI is taking over programming." It isn't taking over anything. It's one task category, and inside that category the delegation style leads. Both things are true, and the small share is the part that keeps the big number from getting oversold.

One more caveat worth stating plainly: this is a single snapshot from May 2026. No trend line, no trajectory, no "up 12 points from last quarter." One reading. The 61/39 split is a photograph, not a movie. I'd bet the direction of travel is toward more automation as tools get better at running unattended — but that's my read, not something the data shows. Treat it as a fixed point, not a slope.

And note the seam in the categories: Anthropic reports "Computer and Mathematical" as the number one task category overall, yet software development specifically sits at 11.5 percent. Those aren't in conflict — the top category is broad and covers a lot more than writing application code. But if you conflate them, you'll overstate how much of AI usage is engineers shipping features. Most of it isn't.

What a Buyer Should Do With This

If you're choosing or building a coding agent, design for the automation majority. That's the whole recommendation, and it changes what "good" looks like.

The copilot default optimizes for a person watching every step. The automation pattern optimizes for a person checking the result. Those need different architecture. I've written before that a coding agent stack has two layers — the model and the scaffolding around it — and that the scaffolding decides the outcome more than the model choice does. This data is why. When 61 percent of real coding work is delegation, the layer that runs the agent unattended and catches its mistakes is doing the heavy lifting, not the chat window.

Concretely, designing for automation means:

Autonomous execution with verification gates, not approval-per-diff. The agent should run the task end to end, then hand you a result gated by checks it had to pass — tests green, types clean, lint quiet, a diff that touches only what it claimed. You review the outcome and the evidence, not every keystroke. A human approving line-by-line is the augmentation product, and it doesn't scale to 61 percent of your work.

Real gates, not vibes. The gates are the whole safety story once the human steps back from each step. That means a test suite the agent can't skip, a build it can't merge past when red, and provenance on what changed. If you can't articulate what has to be true before an agent's output reaches main, you haven't built for automation — you've built a faster way to generate unreviewed code.

Legibility at the boundary, not throughout. Augmentation needs the tool legible at every turn. Automation needs it legible at one point: the handoff. Invest your explanation budget there — what did it do, what did it check, what should you look at first.

The tools that ship copilot-by-default are optimizing for the 39 percent. That's a real 39 percent, and for exploratory work, learning a new codebase, or high-stakes changes where you genuinely want to drive, augmentation is the right mode. Don't rip it out. But if your default interaction model assumes a human hand on every commit, you've built the minority product and called it the platform.

The Frame Is Behind the Practice

The interesting gap here isn't between humans and machines. It's between how the industry talks about coding agents and how people actually use them. The marketing still runs on the copilot metaphor — reassuring, human-centered, a decade old. The usage data says developers crossed over to delegation a while ago and mostly stopped narrating it.

61 percent. That's the number to bring to your next tool evaluation. Ask the vendor what happens after the human stops watching — what runs, what gets checked, what stops a bad change cold. If the answer is "well, you approve each diff," they built for the half that's shrinking.

The Meter Is Always Running

Michael Tuszynski — Tue, 21 Jul 2026 14:02:11 +0000

The Meter Is Always Running

For forty years, enterprise software rested on a comfortable assumption: that it was a fixed cost. You paid a license, amortized it over a depreciation schedule, and every unit of value you squeezed out afterward felt like it was approaching free. Seats, not usage. Capex, not cost of goods. A CFO could draw a straight line from spend to return and sleep at night.

AI breaks that line. Most organizations adopted it without noticing.

The thing that actually changed

The interesting shift in enterprise AI is not cloud versus on-premises. That is a deployment question, and deployment questions are the kind of thing infrastructure teams have solved a hundred times. The deeper shift is quieter: software stopped being a fixed cost and became a variable one.

A large language model does not bill like a license. It bills like a taxi. The meter runs on every token, upstream and downstream, and input and output are priced per million tokens at rates that differ by model and by direction. The fare depends on how long the conversation ran, how much reasoning it demanded, how many times a user hit retry. Two customers doing the "same" task can cost 10x different amounts because one of them pasted a novel into the prompt. There is no seat count that caps it. There is no version you buy once and freeze. The cost scales with success: the more people use the feature, the more it costs you to have built it.

That is a new financial object, and the mental model most companies applied to it, SaaS licensing or depreciable capital, is the wrong shape. You cannot amortize a variable cost. You can only manage it.

The on-prem mirage

The popular rebuttal goes like this: just run it yourself. Buy the hardware once, own it forever, and your marginal cost per query approaches zero. Convert the scary variable cost back into a familiar fixed one.

It is a seductive move, and it is mostly wrong.

The "own it forever" asset is not the GPU. It is the model, and the model is spoiling in real time. The cluster you justified this year runs a frontier model that will be mid-tier in twelve months. The hardware ages too: how long an AI accelerator stays economically useful is now an open argument on Wall Street, with investors questioning whether the six-year depreciation schedules on the books match a reality where a new generation lands every eighteen months. You did not buy a depreciable asset amortized over seven years. You bought a seat on a refresh treadmill, plus a standing bill for power, cooling, and the engineers who keep it fed.

And unlike the cloud meter, that silicon costs the same whether it runs at 90% load or 5%. Idle capacity is the norm, not the exception: it is common for GPU clusters to sit 70 to 80% idle between bursts of real work. You did not eliminate the variable cost. You traded a usage-based meter for a utilization risk, and for most enterprises, whose demand is spiky rather than steady, that is the worse trade.

Marginal-cost-to-zero is real, but only at high, sustained load on a model you are content to keep running. That describes a narrow band of workloads. It does not describe "our AI strategy."

Inference is cost of goods, not capex

Here is the reframe that survives contact with a spreadsheet: inference is cost of goods sold.

Not a license you buy. Not a machine you depreciate. It is a per-transaction input cost, like the cloud compute behind a web request or the fee on a card payment. Companies already know how to run a business where serving each customer costs money. They have gross-margin discipline, unit economics, dashboards that flag when a product line goes underwater. They simply never had to apply that discipline to software features before, because software features used to be free to serve once written.

File inference under cost of goods instead of "IT capex," and the right behaviors fall out on their own. You start asking the questions a margin-conscious operator always asks.

What that discipline looks like

Know your unit cost per feature. If you cannot say what a single run costs, you are blind on the one number that decides whether the feature is a business or a liability. Instrument tokens the way you already instrument latency.

Then route by need. Most requests do not warrant a frontier model; send the cheap, frequent, predictable work to smaller or local models and reserve the expensive reasoning for the calls that actually earn it. A good router is worth more than a bigger model.

Cache aggressively. Repeated system prompts, retrieved context, common answers: every hit is a fare you skip. Anthropic reports that prompt caching can cut input costs by up to 90% and latency along with it. Caching is not a nice-to-have optimization here. It is margin.

Put a ceiling on it. Variable cost without guardrails is how one looping agent turns into a five-figure surprise. Budget caps, rate limits, and per-tenant quotas are the circuit breaker. Build them before the invoice, not after.

And decide placement per workload, not per company. Some things belong on a local model you own; some belong on a metered API. "Cloud-first" and "on-prem-first" are both the wrong altitude. The unit is the workload, and the answer is a portfolio.

The reckoning is real. It is not the one on the thumbnail.

A reckoning is coming for a lot of AI initiatives, but it will not arrive as the dramatic collapse of anyone's cloud strategy. It will arrive as a quiet quarterly review where someone finally divides the AI line item by the number of customers it served and does not like the answer. The projects that survive that meeting will not be the ones that picked the right deployment location. They will be the ones that treated inference as what it is, a variable operating cost with no natural ceiling, and built the discipline to manage it from the first day.

The meter was always running. The only question is whether you were watching it.

Your AI Coding Pilot Cost $80K and Shipped 9% Faster. Here's the Line Item Finance Missed.

Michael Tuszynski — Mon, 20 Jul 2026 14:07:50 +0000

The slide always looks the same. Pilot spend: $80K. Velocity: up 9%. Recommendation: expand to the full org. I have sat through enough of these readouts, on both sides of the table, to recite the speaker notes from memory. The room approves it, because $80K buying a 9% faster engineering org of any size is obviously a deal.

The math on the slide is fine. The problem is a line item that never made it onto the slide, and it is the most expensive input in the building.

Where the $80K Went

Run the composite pilot. Fifty seats of an AI coding assistant at roughly $20 a head is about $12K a year. Agent and API tokens for the teams that went past autocomplete, call it $3K a month once real workloads land — I've published my own bill, so I know what this curve looks like. Add the integration sprint, the eval work, and the enablement time nobody bills to the pilot, and $80K is a fair all-in number for a mid-size org's first serious year.

Every dollar of that is metered, invoiced, and visible. Seats show up on a purchase order. Tokens show up on a usage dashboard. Finance can read the pilot's cost to the penny, which is exactly why the readout feels rigorous.

Now the return side. A 9% velocity gain across a 12-engineer group is roughly one extra engineer's worth of throughput, and a loaded engineer runs $250K, which is the number that dwarfs every token bill in sight. So the slide implies $80K bought about $270K of capacity. A 3.4x return, approved before the coffee gets cold.

The Line Item That Never Shows Up

Here is what the slide does not carry: the review and verification attention the pilot consumed, priced at senior-engineer rates.

AI-assisted pull requests change shape. Diffs get bigger. Volume goes up. And the failure mode shifts from obviously-broken to plausible-but-wrong, which is the expensive kind, because plausible-but-wrong has to be caught by a human who understands the system. Suppose each engineer ships five AI-assisted PRs a week and each one takes a reviewer just twenty extra minutes. Across 12 engineers that is 20 hours a week of review time, half a full-time engineer, roughly $125K a year. And it does not land evenly: it lands on the two or three senior people qualified to catch the plausible-but-wrong category, the same scarce attention that was already the bottleneck.

Take the $270K gain, subtract $80K in tool spend and $125K in unpriced senior attention, and the triumphant 3.4x collapses to about 1.2x, before counting rework. That might still be worth doing. But nobody in the room approved that deal, because nobody saw it.

The reason nobody saw it is structural, and it is worth saying plainly: finance manages what is metered. Seats and tokens arrive as invoices. Attention arrives as nothing at all. It is smeared invisibly across a salary line that looks identical whether your seniors spent the quarter building or spent it correcting a machine's confident guesses.

The Evidence Says the Slide Flatters Itself

The 9% is usually self-reported or lightly measured, and self-report runs hot. In METR's randomized study of experienced open-source developers, participants estimated AI made them 20% faster while the measured result was 19% slower. The perception gap is the mechanism that hides the missing line item: the tool feels fast because typing is fast, while the slow part — verifying, correcting, re-prompting — doesn't register as "using the tool."

The vendor numbers run hotter still. GitHub's own study reported 55% faster task completion, a first-party figure measured on a greenfield toy task, and it should be discounted accordingly. Meanwhile DORA's research keeps finding the bill on the other side of the ledger: as AI adoption rises, software delivery stability dips. Faster generation, wobblier delivery. That wobble is rework, and rework is more senior attention, unpriced.

To be fair to the other side of the argument: METR's result is one population on one kind of task, not a universal law, and pilots built on scoped tasks with an explicit verification budget genuinely do clear the honest bar. The point is narrower and harder to dodge. A pilot that measured only what was invoiced has not measured its return. It has measured its receipts.

The Five-Line Version

The fix costs nothing but honesty. Here is the pilot P&L with all the lines on it:

AI coding pilot P&L

+ Velocity delta
    measured (cycle time,
    merged scope), never
    surveyed
- Seats + tokens
    the invoice (the only
    line the readout had)
- Review-hours delta
    PR review time before
    vs. during, x reviewer
    cost
- Rework delta
    change-failure/rollback
    rate, before vs. during
- Comprehension debt
    merged code nobody on
    the team can explain, %

The first two lines exist in every pilot. The last three exist in almost none, and they are all measurable with tooling you already run: your git host timestamps every review, your incident tracker already counts rollbacks, and the comprehension question takes one uncomfortable team meeting.

If the pilot still clears the bar with five lines on the page, expand it with confidence — mine does, and that's precisely why I keep publishing the full bill instead of the flattering half. A tool that survives honest accounting is a tool worth scaling.

Ask for the Line

The 9% on the slide is neither a lie nor a result. It is an unfinished calculation, presented with the confidence of a finished one, to a room that is being asked to multiply it by the whole org.

So ask the one question that finishes it: show me the review-hours delta. If the answer is a number, you are looking at a rare, well-run pilot, and you should probably fund it. If the answer is a pause, then the 9% was never a measurement. It was a survey wearing one, and the most expensive people in the building are quietly paying the difference.

The Bubble Popper and the Payoff Are the Same Thing

Michael Tuszynski — Thu, 16 Jul 2026 23:59:00 +0000

Cory Doctorow thinks AI is a bubble, and he argues it better than almost anyone who cheers him on. In his December speech at the University of Washington, the case runs like this: the tech giants stopped growing years ago, and a company priced as a growth stock faces catastrophe the moment the market stops believing. So they pump whatever keeps the multiple alive. Pivot to video, crypto, NFTs, Metaverse, now AI, each pitched with the same conviction, each abandoned when the next vehicle arrives. "Superintelligence" is the pitch this cycle because science fiction makes a better P/E story than enterprise software ever did.

His sharpest tool is the reverse centaur. A centaur is a human assisted by a machine: you drive the car, the car amplifies you. A reverse centaur is a human bolted onto a machine as its peripheral, hired to absorb the failures the machine can't, at machine pace, under machine supervision. Doctorow's thesis is that the money behind AI is betting on reverse centaurs: workers demoted into error-cleanup for systems sold to their bosses as replacements.

Take him seriously. The losses are real, the incentives he names are real, and I have sat through enough board-mandated AI pilots to know the reverse centaur is not a hypothetical. And then notice that his argument, stated carefully, is about two different balance sheets that the doom discourse keeps smearing into one.

Two Balance Sheets, One Shouting Match

Seller economics asks whether the labs make money. Company-level losses are enormous, but look at where they live: training runs, data-center capex, sales, and the free tier. The serving unit underneath is a different story. SemiAnalysis puts Anthropic's gross margins in the mid-60s — an independent analyst number, though The Information reported the same company projecting 40% when inference costs spiked, so treat the exact figure as contested. The direction is not. Serving paid tokens is gross-margin-positive; what bleeds is everything wrapped around it.

Even the famous counterexample proves the shape. When Sam Altman admitted OpenAI loses money on the $200 Pro plan, the reason was that heavy users outran the flat price. Sell unbounded agent workloads at a fixed monthly fee and the top of the usage tail eats you. That is a pricing-model problem, and pricing models get fixed. Physics problems don't. This one is being fixed right now, mostly at the expense of people like me: usage caps, tier splits, metered agents.

Buyer economics asks a different question: does a scoped deployment pay for itself? That is where operators live, and none of the training-run losses show up on this side of the table. I've written down what my own stack costs per month, and why "$20K in tokens against a $250K engineer" is the wrong math even when it flatters the tools. A deployment either produces measured value over measured cost or it doesn't. The lab's income statement has nothing to do with it.

Doctorow is right about the first balance sheet. The mistake is letting the first one answer for the second.

The Question That Does All the Work

Here is my actual thesis, earned from reps rather than from a valuation model: the economics work themselves out, but only for operators disciplined enough to ask "what problem am I solving" before pointing the tool at anything, and rigorous enough to measure the answer afterward.

An efficiency you don't understand is not an efficiency. It is an unmeasured cost wearing a demo. The team that deploys a coding agent with no baseline, no success metric, and no owner has not automated anything; it has hired a very fast intern nobody supervises and booked the salary as savings. That is blind deployment, and blind deployment is exactly the reverse-centaur failure: the humans end up serving the machine's output because nobody defined what the machine was for.

Which means my thesis and Doctorow's frame are the same observation read from opposite ends. Scoping rigor is what makes you a centaur instead of a reverse one. The human who decides what the machine is for stays the head. The human who cleans up after an unscoped machine becomes the peripheral.

The scoping questions I use, written down (writing them down matters, and the last section says why):

1. What is the problem, stated without naming a tool?
2. What does it cost today, in hours or dollars I can point at?
3. What does "working" look like, measured how, by whom?
4. What is the failure mode, and who catches it?
5. What is the kill criterion, decided before the pilot starts?

The Payoff and the Pin Are the Same Force

Now the part I have not seen anyone say plainly: the rigor that makes AI pay off is the same force that pops the bubble.

Walk through what happens if scoping discipline actually spreads. Every FOMO seat license gets audited against question 2, and half of them fail. Every board-mandated pilot meets question 3 and dies for lack of a metric. The all-you-can-eat experimentation budgets get metered. A real slice of today's AI revenue is hype-spend, and disciplined buyers stop paying for hype. Revenue evaporates precisely as deployments get better.

That is not bearish on AI. It is bearish on the bubble, and those are different positions. It is Doctorow's own "some bubbles leave behind something productive" scenario made concrete. He reaches for Worldcom, a fraud whose CEO died in prison while the dark fiber it buried still carries Doctorow's 2-gigabit home connection. Swap the fiber for scoped deployments quietly compounding inside businesses while the FOMO spend burns off. The economics working out and the bubble deflating are the same event, narrated by an optimist and a pessimist.

The Craftsman Clause

There is an investor-side implication buried in this that is more uncomfortable than any crash prediction.

If every deployment that pays needs a human who understands both the domain and the tool's failure modes, then AI is not the frictionless labor replacement the trillion-dollar valuations are priced on. It is a power tool that still needs a craftsman. A nail gun does not fire the carpenter; it makes the carpenter's judgment the binding constraint on every house. Attention was always the bottleneck; the tools just moved it.

Great news for buyer ROI. Quietly fatal for the "human exits the loop" story being sold upstream, because the wholesale-replacement bet only pays if the craftsman requirement goes away, and every honest deployment I have run or reviewed says it doesn't. The augmentation bet keeps cashing small checks. The replacement bet keeps pre-spending checks nobody has figured out how to write.

Two Half-Lives, One Fork

So if the scarce input is the human who can scope, the personal question is what that scarcity is worth and how long it lasts.

The scoping skill is teachable in principle and bottlenecked in practice, and the bottleneck is structural, not a fad. The skill is tacit. It lives on the seam between a domain and a tool, where few people sit. And organizations promote people for shipping, not for the scoping that made the shipping cheap; nobody's OKRs reward the pilot that didn't happen.

But the edge has two halves with opposite half-lives, and confusing them is the trap. Tool mechanics — prompt-craft, knowing this month's model's failure modes, the tricks — depreciates fast, because every model release is the labs folding exactly that knowledge into the product. The agent-loop mechanics I wrote up are already aging out from under that post. Problem formulation — translating a business's real problem out from under its stated one — depreciates slowly, because it is organizational translation, and organizations stay human. The trap is that the fast-depreciating half is the legible, demoable half. It feels like the edge because you can show it off. The durable half looks like "just asking questions."

Which leaves the fork, and I am going to leave it as a fork rather than resolve it for you. Stay the indispensable bottleneck and you extract rent: real money, now, capped at your personal throughput, gone the day the scarcity ends. Or codify the tacit half into artifacts a colleague can run without you — a written scoping method like the checklist above, reference patterns from the deployments that worked — and you dissolve your own bottleneck while capturing the value of having systematized it. Rent pays this quarter. The moat is owning the codified version when everyone else finally needs it.

The bubble being real is what creates the opening; nobody pays a premium for discipline in a calm market. The economics work out for whoever is on the right side of the rent-to-moat conversion. And the thing to be dismantling, deliberately, starting now, is the bottleneck being you personally — before the labs dismantle the cheap half of your edge for free and leave you holding only the part you never bothered to write down.