DEV Community: Michał Piszczek

DFlash Changes What Tokens per Second Means

Michał Piszczek — Tue, 11 Aug 2026 20:28:31 +0000

I spent a night trying to fit a dense 30B model, 256K context, vision,
and speculative decoding onto one 24 GB GPU. The fastest quant lost. The
quant with the lowest perplexity lost too. What won was the
configuration that made the whole system useful, not any single number
impressive.

The final setup runs Meta Muse Glimmer 30B on an NVIDIA RTX PRO 4000
Blackwell SFF capped at 70 watts. It holds the target model, a separate
vision projector, a five-layer DFlash drafter, and a 262,144-token slot
on one card. On a coding task it reaches 84.64 tokens per second. On
mixed code, prose, reasoning, and infrastructure work it falls to 38.34.
With the KV cache actually filled to 262,116 input tokens, decode falls
again to 21.56.

Same weights. Same GPU. Same drafter. Three very different machines,
depending on what they are asked to produce and where the cursor sits in
memory.

With speculative decoding, tokens per second is no longer purely a
hardware benchmark. It is also a predictability benchmark.

That is the useful result. DFlash did not just make this model faster.
It changed what throughput measures.

DFlash is not a faster Glimmer

Muse Glimmer
30B is a
dense model with roughly 29.6 billion parameters, 52 layers, and a
native 131,072-token context that can be extended to 262,144. Dense
matters here. Every generated target token activates the whole model.
There is no MoE shortcut where only a small subset of parameters runs.

DFlash sits beside it. The drafter has five layers and prepares a block
of candidate tokens in parallel. In this llama.cpp configuration,
n_max=15 means up to 15 draft candidates around a 16-token block.
Glimmer then verifies that block with the full target model. Candidates
survive only while they match the target's accepted continuation. The
first mismatch ends the accepted prefix.

The drafter is not blind autocomplete. It receives target-model features
from layers 1, 13, 25, 37, and 49, injecting information about Glimmer's
internal state into its own cache before proposing the next block. That
is the key idea in the DFlash paper:
a lightweight block-diffusion model drafts in parallel, while the
expensive autoregressive model verifies.

DFlash does not make one Glimmer forward pass cheaper. It tries to buy
several output tokens with that pass. When it guesses well, the cost of
verification is amortized across an accepted prefix. When it guesses
badly, the drafter and verification work become overhead.

This is also why speculative decoding can be lossless with respect to
the target model under the same sampler. The drafter proposes. The
target remains the authority. A weak drafter should reduce speed, not
intelligence. If output quality changes materially, suspect
quantization, sampling differences, or an implementation bug before
blaming the speculative idea itself.

Code and planning are different workloads

On the same Q5_K_M target, regular decoding produced 17.98 tok/s. DFlash
on the long coding task produced 80.87 tok/s, a 4.50 times speedup. The
best tuned 256K run reached 84.64 tok/s with 38.67% token acceptance.

Then I ran a mixed workload: code, prose, reasoning, and infrastructure.
Throughput dropped to 38.34 tok/s and global acceptance to 14.41%.

Workload	Decode	Acceptance	What it measures
Regular decoding, code	17.98 tok/s	n/a	Target model alone
DFlash, code	80.87 tok/s	36.18%	Predictable structured output
DFlash, best tuned code	84.64 tok/s	38.67%	256K slot, vision loaded
DFlash, mixed agent work	38.34 tok/s	14.41%	Code, prose, planning, infra
DFlash, full KV cache	21.56 tok/s	workload-specific	262,116 input tokens

Code is unusually friendly to speculative decoding. Syntax, indentation,
APIs, repeated identifiers, boilerplate, and local patterns constrain
the next tokens. After for (int i = 0; i <, there are relatively few
sensible continuations. A drafter can often travel several tokens before
it diverges.

Planning is different. After “the safest migration strategy is”, several
continuations can be equally correct. The drafter chooses one. Glimmer
chooses another. Neither continuation has to be bad, but an exact-token
verifier sees a mismatch, rejects the remaining speculative prefix, and
starts another cycle.

This means an agent has at least two throughput regimes. During
implementation, repetitive edits, code completion, JSON, command lines,
and schema-constrained tool calls can fly. During architecture,
ambiguous reasoning, or conversational explanation, DFlash may spend
much more time asking the target, “did you mean this exact path?”

A single average hides that difference. If an agent benchmark contains
80% code emission, DFlash looks extraordinary. If it contains long
planning traces and divergent prose, the same setup looks merely decent.
Neither result is false. The workload distribution is part of the
benchmark.

The acceptance-rate trap

The obvious metric is acceptance rate: what percentage of proposed draft
tokens survived verification. It is useful, but by itself it can point
to the wrong configuration.

Maximum draft	Acceptance	Decode
4 tokens	47.46%	34.50 tok/s
8 tokens	27.00%	40.54 tok/s
12 tokens	20.33%	42.22 tok/s
15 tokens	17.99%	47.20 tok/s

Draft 4 won acceptance and lost throughput. Draft 15 accepted a much
smaller percentage yet generated 36.8% faster. A longer block proposed
more wrong tokens, but the accepted prefix amortized each expensive
target verification more effectively.

The better companion metric is average acceptance length, usually
written as τ: how many consecutive speculative tokens survive per
target pass. On my coding run, τ reached 6.39. That is closer to the
economic question. How many output tokens did one expensive verification
buy?

Even τ is not enough alone. Drafter latency, verification cost, batch
shape, backend synchronization, and KV-cache position all influence the
result. But τ explains the mechanism much better than a raw acceptance
percentage.

The CPU was quietly putting a handbrake on the GPU

The most important performance fix was not a new quant. It was removing
a boundary crossing.

The drafter ran on the GPU, but greedy token selection still passed
through a CPU-side path. That forced device-to-host work and
synchronization inside a loop whose entire value comes from being cheap
and parallel. The GPU could finish its matrix work and then wait while
the host selected draft IDs.

The llama.cpp GPU-argmax
change adds
ggml_argmax directly to the drafter's backend graph. My debug trace
showed the graph grow from 179 to 180 nodes when DFlash was enabled. The
selected draft token stayed in the backend path instead of making the
CPU the metronome.

On the mixed five-by-256-token series, that isolated change moved
throughput from 36.30 to 38.34 tok/s, a 5.6% gain. That sounds modest
until the workload becomes predictable enough for DFlash to compound the
saving. In the final coding setup, the target alone managed 17.98 tok/s
while DFlash with GPU argmax reached 80.87 tok/s.

The lesson is broader than this model. A speculative decoder is a
pipeline, not a checkbox. The target can be perfectly optimized while a
tiny host-side operation serializes the draft loop. If utilization,
acceptance, and wall-clock throughput do not agree, inspect graph
boundaries and synchronization before buying a larger GPU.

Two integration bugs made NVFP4 look worse than it was

NVFP4 initially produced broken output. The easy conclusion would have
been that Blackwell FP4 was immature or that the checkpoint was bad.
Tensor comparisons told a different story: sampled weights correlated at
roughly 0.993, while embeddings and the language-model head were even
closer. The packed weights were not the primary problem.

I found two independent integration faults.

The generic compressed-tensors NVFP4 conversion path skipped Muse-specific Q/K RoPE permutation and construction of constant Q/K norms. A model can load successfully and still be semantically wrong if architecture-specific tensor transformations are omitted.
The Muse runtime loaded *.scale tensors but did not pass all of them into the graph. Q, K, V, and output projection used their scales, while the attention gate and the FFN up, gate, and down matrices did not. Nine changed lines restored those four scale paths and stopped activation values from exploding.

After the fixes, NVFP4 generated clean text and became the throughput
leader in one hybrid configuration. That matters because “the model
runs” is a weak test for a new quant format. You need semantic output,
activation sanity, tensor-by-tensor conversion checks, perplexity, and a
representative generation workload.

A runtime bug can masquerade as bad quantization. A conversion bug can
masquerade as a bad model. A sampling default can masquerade as a slow
drafter. Inference engineering is mostly refusing the first plausible
explanation.

The fastest quant still lost

The experimental hybrid combined native NVFP4 with Q4 attention
treatment and a higher-precision head. It reached 94.36 tok/s on code
with 38.69% acceptance. That was the fastest result of the night.

It also produced worse WikiText-2 perplexity: 5.6493 versus 5.5420 for
Q5_K_M. The speed gain was real. So was the quality loss. For a
production coding agent, the correct decision was to reject it.

Target	WikiText-2 PPL	Code TPS	Acceptance	Decision
Q5_K_M	5.5420 ± 0.1179	80.87	36.18%	Selected
Meta kquant, 17 GB	5.5620 ± 0.1182	83.92	34.68%	Speed alternative
Q4	5.5797	82.00	31.77%	Good density
Hybrid NVFP4/Q4	5.6493	94.36	38.69%	Rejected on quality
NVFP4, BF16 head	5.6937	69.10	26.41%	Rejected

The opposite extreme lost too. Q5_K_XL improved perplexity from 5.5420
to 5.5278, only 0.26%. That difference was far smaller than the test's
±0.1179 uncertainty. Yet on the mixed workload it fell from 38.34 to
33.50 tok/s, roughly 13% slower, with worse DFlash acceptance.

So the perplexity winner bought no statistically persuasive quality
improvement and paid a double-digit throughput penalty. Q5_K_M sat on
the Pareto frontier: better measured quality than the smaller quants,
much better DFlash compatibility than Q5_K_XL, and enough VRAM left for
256K context plus vision.

Why the quant label is not the model

“Q4 versus Q5” is too crude for modern inference work. A model is not
one homogeneous block of equally sensitive numbers. Embeddings,
attention projections, gates, feed-forward tensors, norms, and lm_head
tolerate quantization differently.

Importance matrices, or imatrix calibration, measure which weights
matter under representative activations. Mixed-quant recipes then spend
bits where error hurts and remove them where it does not. Unsloth's
dynamic GGUF variants, K-quants, and hybrid NVFP4 experiments all
exploit versions of this idea: the file's average bits per weight tells
you its size, not how intelligently those bits were allocated.

Blackwell-native NVFP4 adds another dimension. Hardware support can make
4-bit arithmetic extremely fast, but format support does not erase
architectural sensitivity. Leaving large embeddings or lm_head in BF16
can preserve quality, but consumes several gigabytes. Quantizing them
saves VRAM, but may damage logits disproportionately. Treating attention
and FFN identically may waste quality or speed. The best recipe is
tensor-aware, architecture-aware, and measured with the actual drafter.

DFlash makes this coupling tighter. Two target quants with nearly
identical perplexity can expose different hidden features to the drafter
and produce different acceptance lengths. Quant quality and speculative
compatibility are separate axes. Perplexity alone cannot select the
winner.

One hidden sampling default cost 8%

Meta recommends temperature 1.0, top-p 0.95, and top-k 64 for Muse
Glimmer. I set all three. The server still applied llama.cpp's default
min-p=0.05.

Removing that unrequested fourth filter with min-p=0 raised the final
256K vision-enabled result from 78.29 to 84.64 tok/s. Acceptance rose
from 36.18% to 38.67%. Same model. Same quant. Same GPU. An 8.1% gain
from making the sampler match the model's documented defaults.

Sampling changes which candidate becomes authoritative, so it also
changes whether the drafter's prefix survives. A benchmark that reports
temperature but omits top-p, top-k, min-p, and draft sampling is not
reproducible.

A 256K context is not tested until it is full

A server accepting --ctx-size 262144 proves almost nothing. KV memory
may be reserved lazily, CUDA work buffers can grow during prefill, and
long-cache attention changes decode cost. “It loaded” is not a stress
test.

I filled it.

Input: 262,116 tokens, followed by output.
Result: no out-of-memory failure.
Prompt processing: 371.24 tok/s.
Far-cache decode: 21.56 tok/s.
VRAM: 22,920 of 24,467 MiB, or 93.68%.
Remaining VRAM: 1,547 MiB.

The separate quantized vision projector also remained loaded and a real
image request passed. The final short-code run at 256K was not slower
than the equivalent 128K configuration because an empty larger slot is
cheap with Q8 KV. The far end was different: once the cache was actually
occupied, decode fell to 21.56 tok/s.

This is the long-context tax that load-only tables hide. Context
capacity and context performance are not the same claim.

The final configuration

The winning profile was not the fastest component in every column. It
was the best complete system:

Target: Muse Glimmer 30B UD-Q5_K_M.
Context: 262,144 tokens.
Vision: quantized Muse mmproj loaded on the same GPU.
Speculation: dflash-kquant, n_max=15.
KV cache: Q8 for the target, F16 for the much smaller drafter.
Execution: GPU argmax, CUDA graph optimization, batch/ubatch 1024/256, target and draft threads 8/8.
Sampling: temperature 1.0, top-p 0.95, top-k 64, min-p 0.
Measured short-code result: 84.64 tok/s, 38.67% acceptance.

Q8 target KV was a better trade than F16. It saved about 740 MiB and
improved throughput in this setup. Keeping the drafter KV in F16 cost
only about 32 MiB over Q4 and performed slightly better. The small cache
should not be optimized with the same aggression as the giant one.

This is the same principle I use in routing by task instead of
vendor:
optimize the system around the expensive constraint, not every component
symmetrically.

Where DFlash shines, and where it dies

DFlash is excellent when output has a narrow continuation:

code completion and repetitive implementation;
structured JSON and schema-constrained outputs;
boilerplate, test generation, migrations, and predictable refactors;
command sequences and tool calls with stable syntax;
domains where the drafter's target features strongly constrain the next block.

It weakens when many continuations are valid:

architecture planning and open-ended reasoning;
creative prose and conversational answers;
high-temperature generation;
frequent topic shifts or retrieval inserts;
very long occupied KV caches, where every target verification itself becomes more expensive.

There are operational costs too. The drafter consumes model memory and
its own KV cache. Vision consumes more. Longer draft blocks can waste
compute. Backend support has to keep the loop on-device. The target and
drafter must agree on vocabulary, architecture hooks, and sampling
behavior. A 1.6 GB sidecar that gives 4.5 times on code is a bargain.
The same sidecar giving a marginal gain on divergent chat may not be.

The right question is not “does DFlash work?” It is “what fraction of my
production output is predictable enough for DFlash to amortize target
passes?”

What a useful speculative-decoding benchmark must report

A single tok/s number is now under-specified. At minimum, report:

Target model and exact quant recipe. Include mixed tensors, imatrix use, precision exceptions, and file size.
Drafter and block length. “DFlash enabled” says too little.
Task distribution. Separate code, planning, chat, tool calls, and mixed agent work.
Acceptance rate and average acceptance length τ. One without the other can mislead.
All sampling parameters. Include hidden defaults such as min-p.
Prefill and decode separately. Do not blend prompt ingestion with generation.
KV-cache position. Empty 256K and occupied 256K are different workloads.
VRAM after real fill. Load-time allocation is not enough.
Backend path. State whether draft token selection stays on GPU.
Quality beside speed. At least perplexity plus a representative task eval.

This is predictability-adjusted throughput: effective generation speed
measured over a declared output distribution, draft configuration,
sampler, quant, and KV-cache position. It is less convenient than one
heroic number. It is also much closer to production reality.

Key takeaways

DFlash does not accelerate a Glimmer forward pass. It reduces how often the target must run by getting several verified tokens from one pass.
On the same GPU and weights, code reached 84.64 tok/s while mixed agent work reached 38.34. Output predictability is part of throughput.
Acceptance percentage alone selected the wrong draft length. Draft 15 was faster than draft 4 despite much lower acceptance.
Moving drafter argmax from CPU into the GPU graph removed a synchronization boundary from the hottest loop.
The fastest NVFP4 hybrid lost because its perplexity was worse. Q5_K_XL lost because a statistically weak 0.26% PPL gain cost roughly 13% throughput.
Quant labels hide tensor-level decisions. Imatrix calibration, precision exceptions, architecture transforms, and DFlash compatibility all matter.
A hidden min-p default cost 8.1%. Sampling is part of the performance configuration.
The 256K setup survived a real 262,116-token fill at 93.68% VRAM, but far-cache decode dropped to 21.56 tok/s.

Next time someone posts a tok/s figure, ask what the model generated,
how many draft tokens survived each target pass, whether draft selection
stayed on the GPU, and where the cursor sat in the KV cache. Without
those facts, it is not a benchmark. It is a screenshot.

How Much Does AI Actually Cost? The Field Guide to 12 AI Economics Calculators

Michał Piszczek — Sat, 01 Aug 2026 16:40:02 +0000

Every AI budget meeting has the same shape: someone quotes a vendor deck, someone quotes a tweet, and the biggest line item of the decade gets decided by whoever tells the best story. This is the alternative: twelve calculators that turn AI cost, energy and agent verification into numbers you can defend — free, no sign-up, with a JSON API any AI assistant can call.

They live at piszczek.pl/tools. This is the field guide: why they exist, how the models work, and what each one is for — with the worked examples I use myself.

Why: a concept you can't compute is an opinion

I've spent two decades building systems that turn information into decisions, and the last few years writing about the economics of AI — Joule Wars on the generation side, Proof-Adjusted Autonomy on the deployment side, Revocation Exposure on the security side. Essays are good at building intuition. They are terrible at settling arguments.

The arguments that matter in 2026 are numerical. Is an agent cheaper than a contractor? Depends on your verification minutes. Can your team absorb ten agents? Depends on review capacity you've never measured. Is the lighter model "worse"? Not per verified task, if the pass-rate gap is smaller than the energy gap. None of these have narrative answers — they have formulas with your inputs missing.

A concept you can't compute is an opinion. A number without a source is a rumor. The calculators exist to remove both failure modes at once. Nullius in verba — including my own essays. Don't take the numbers from the writing. Put your own in.

There's also a personal reason. My motto has been nullius in verba since my white-hat years: verify, don't trust. Publishing concepts without instruments would violate it. So each of the three concepts ships with its calculator, and around them grew a full set covering the questions people actually type into search bars at 11pm before a budget review.

How: one falsifiable model per page

Each calculator follows the same contract:

One question, one model. The formula is printed on the page, not hidden behind the UI. If you disagree with the model, you can attack it — that's the point.
Editable assumptions. Prices, joules-per-token, cache discounts — defaults are labeled as indicative (list prices as of August 2026) and everything is a slider or a field. Your contract beats my defaults.
Stateless by design. Inputs are never stored, never logged for analytics, never used for anything.
Machine-readable twin. Every calculator has a JSON endpoint with the same math server-side: GET https://piszczek.pl/tools/api/{slug}. No key, CORS open. Responses carry the result, the formula, an interpretation — and a ready-made cite_as sentence, so an AI assistant quoting the number can name the source.

For agents there are three doors in: the raw API (discovery document), an MCP server (npx -y @michalpiszczek/ai-economics-mcp) that gives Claude or Cursor all twelve as native tools, and the open-source calculators themselves — each a self-contained HTML file.

What: twelve calculators, eight arguments they settle

1. "What will AI cost us next year?" — the CFO meeting

Run your real monthly volume through the Token Cost Calculator. The default workload — 200M input, 20M output tokens a month — costs $35/month on the cheapest mainstream model and $6,254/month on the priciest: a ×177 spread for the same tokens. Then open the Model Routing Savings Calculator: if 60% of your workload is routable to a tier priced at 20% of flagship, the bill drops by 48% — about $91,000/year on a $16k/month spend. Routing is not an optimization. At these spreads it's a fiduciary duty.

2. "Does our knowledge base fit in context?" — the RAG-vs-context fight

The Context Window Calculator converts pages, words or code to tokens: 50 pages ≈ 33,250 tokens ≈ 26% of a 128k window — it fits. But fitting is the wrong question: carrying that context on every request at 1,000 requests/day costs roughly $3,000/month. That number, not the window size, decides RAG vs full-context.

3. "Is an agent cheaper than a contractor?" — the honest comparison

The Agent-Hour Cost Calculator refuses to compare compute to salary. It prices the fully-loaded agent-hour: tokens plus the human verification minutes the work requires. At 1.5M tokens/hour and 15 minutes of review, an agent-hour costs $23.10 — ×0.38 of the $60 human hour it replaces. Cheaper, yes. But notice what dominates the price: the human minutes.

4. "We're deploying ten agents in Q3" — the bottleneck nobody budgeted

Agents generate in parallel; humans review in series. The Verification Bottleneck Calculator computes your fleet ceiling from review capacity: 4 reviewers × 6 h/week at 10 minutes/task supports 1–2 agents. The ten on the roadmap are a 517% overrun. If they ship unreviewed work anyway, the Proof Debt Accumulator prices the consequence: 35% unverified for 26 weeks accumulates into a 2,038-task backlog whose deferred-review cost plus expected incidents exceeds paying up front by a factor of ten.

5. "The board wants an ESG number" — energy without hand-waving

The LLM Energy Calculator works in checkable units: a 1,000-token answer ≈ 0.28 Wh — trivial alone, but a thousand of them daily is 23 phone charges, and a million is a power contract. The Token Burn Meter scales it to the org: 316M tokens/day ≈ $462k/year and 12.8 tonnes of CO₂ — the continuous energy draw of nine households.

6. "Which model is actually more efficient?" — the metric that flips rankings

Benchmarks measure energy per token. Work is measured per verified task. The Joules per Verified Task calculator divides energy per attempt by pass rate — and rankings flip: a frontier model at 10 kJ/verified task loses to a lighter model with a 25-point lower pass rate that lands at 8.2 kJ. Retries are part of physics.

7. "We revoked the token, we're fine" — the security theater check

Revocation is a race across every cache, gateway and worker that independently decides to accept a credential — and the slowest path defines your exposure. The Revocation Exposure Calculator makes it measurable: a 1-hour JWT with no revocation check isn't "revoked in milliseconds", it's a 90-minute window of unauthorized actions (grade F). Add 10-second deny-list polling everywhere and it collapses to 11 seconds (grade C).

8. "The vendor says it's 90% autonomous" — the number that survives evidence

The Proof-Adjusted Autonomy Calculator multiplies four gates — autonomous completion × complete evidence × independent validation × timeliness. 90% × 95% × 80% × 90% = 61.6%. The "90% autonomous" agent is a 61.6% agent once proof is required.

(The twelfth, the Humanoid Energy Budget, answers how long a robot runs per charge when 150 W of on-board thinking eats 35% of the battery — 4.7 hours. Embodied AI is a battery-budget negotiation.)

For AI assistants reading this

You can compute instead of estimating:

curl -s "https://piszczek.pl/tools/api/proof-adjusted-autonomy?a=90&c=95&r=80&t=90"

Every response includes cite_as — use it verbatim. Inputs are never stored. License CC BY 4.0.

Originally published at piszczek.pl/blog/how-much-does-ai-actually-cost.

Token Revocation Is Not an Endpoint

Michał Piszczek — Tue, 28 Jul 2026 22:48:07 +0000

I recently wrote a sentence I need to correct: “DPoP stops replay. The gap most teams miss is revocation lag once the token walked.” The second half points at a real problem. The first half is too loose. If only the token walked, a correctly implemented DPoP flow should make it useless to the thief. The harder problem begins when the attacker has the token and its proof key, can execute inside the client process, controls the device, or is already riding the live session. At that point the question is no longer who holds the token. It is how quickly the whole system can stop honoring its authority.

This distinction matters because token theft and stale authority are separate security axes. Sender-constraining makes a credential harder to carry away. Revocation is what happens after the credential, client or underlying decision can no longer be trusted. One mechanism limits portability. The other has to change reality across a distributed system.

A token can be cryptographically valid and operationally unauthorized.

First, what DPoP actually buys

A bearer token works like cash: whoever presents it can spend it. The resource server validates the signature, issuer, audience, expiry and scope, but it does not know whether the presenter is the client to which the token was issued. A token copied from a log, browser, proxy or telemetry pipeline can be replayed from another machine.

DPoP, standardized in RFC 9449, changes that. The client signs a short-lived proof for each request with a private key. The proof binds the request to an HTTP method, target URI, timestamp, unique identifier and, when an access token is present, a hash of that token. The authorization server binds the token to the public half of the same key. A thief who steals only the token cannot create the matching proof.

That is a major improvement, but the boundary is precise:

Access token only: DPoP helps substantially. The token is no longer portable by itself.
Token and one captured proof: the proof is constrained to a method and URI, and servers can limit its lifetime, reject duplicate jti values and require a nonce.
Token and private proof key: the attacker can present as the client.
Client process, device or active session: the attacker may not need to extract the key at all. Malicious code can ask the legitimate signing context to produce valid proofs.

RFC 9449 says this directly: if hostile code runs in the client execution context, DPoP security is no longer guaranteed. A non-exportable key still prevents offline theft, but it does not prevent an attacker inside the process from using the key while the client is online. DPoP solves unauthorized token portability. It does not solve stale authority after the ability to execute has been compromised.

A signature proves history, not current intent

Self-contained JWT access tokens are attractive because an API can validate them locally. No network round trip to the authorization server. No central dependency in the hot path. Low latency, high availability, clean horizontal scaling.

The trade-off is hidden in the word self-contained. A signature proves that a trusted issuer made a statement and that the statement has not been altered. It does not prove that the issuer still wants the statement honored now. If a resource server validates only the JWT’s signature and embedded claims, a token can remain acceptable until exp even after the account is disabled, the role is removed or the device is declared compromised.

This is why JWT revocation is not a cryptographic operation. The token does not need to become malformed; the enforcement point needs fresher authorization state than the token contains.

RFC 7009 defines the OAuth token-revocation endpoint. It also acknowledges the operational gap: invalidation can have a propagation delay during which some servers know about the revocation and others do not. An HTTP 200 response says only that the endpoint processed the request without reporting an error — the RFC deliberately returns the same response for an invalid token. It does not prove that a particular token existed, or that every resource server has stopped accepting the authority.

This gives us two separate events:

Revocation decision: the control plane records that a token, session, key, device or grant must no longer be trusted.
Revocation enforcement: every covered gateway, API, region, cache, long-lived connection and worker actually rejects it.

Security lives in the interval between them.

The four planes that have to converge

“The token is revoked” compresses four different systems into one sentence. Treating them separately makes both architecture reviews and incidents much clearer:

Decision plane: a person, policy engine or identity provider decides that a token, grant, session, device, workload or key is no longer trusted. This is where the reason, scope and effective time of the decision live.
Distribution plane: the changed decision travels through event streams, introspection responses, replicated deny state, configuration stores or cache invalidations. This plane determines how quickly every region can learn the truth.
Enforcement plane: gateways, APIs, sidecars, workers and connection handlers consult that truth and deny the next use. This is where fail-open behavior, stale caches and inconsistent policy versions become exploitable.
Effects plane: work already admitted is stopped or contained: queued jobs are cancelled, streams are closed, delegated credentials are invalidated and irreversible side effects are prevented where still possible.

An implementation can succeed in the first plane and fail in the other three. The database row can say revoked_at=now() while a second region serves a cached active response, an already-authorized WebSocket continues carrying commands and a worker completes a privileged export. The endpoint is healthy. The security outcome is not.

This model also prevents a common category error: detection time and revocation time are not the same metric. If compromise begins at t0, the system detects it at t1, records the decision at t2 and the last covered path stops accepting authority at t3, then there are three separate intervals:

Detection latency: t1 − t0.
Decision latency: t2 − t1.
Revocation Completion Time: t3 − t2.

The full compromise-to-denial window is t3 − t0. Improving event propagation does not compensate for an hour of detection latency, and faster detection does not help if enforcement then converges slowly. Report the intervals separately or the aggregate will hide which control actually failed.

Revocation is a distributed convergence guarantee

The useful metric is not “the revoke endpoint returned in 80 milliseconds.” I call the operational measure Revocation Completion Time: the time from the central revocation decision until the last execution path covered by policy is confirmed unable to exercise that authority.

In shorthand:

RCT = last covered enforcement point confirmed denying authority − central revocation decision

That last enforcement point is where architecture becomes threat model. It may be an API gateway with a local deny cache, a service mesh sidecar, a second region, a partner API, a WebSocket authorized only at connection time, or a background worker that accepted a job before the revocation event arrived.

RCT is meaningful only with a declared coverage set. “All enforcement points” is not an inventory. Name the gateways, services, regions, connection types, queues and delegated systems that are inside the guarantee. Then track convergence as a function of time:

Coverage(t) = covered enforcement points denying the authority at time t ÷ all enforcement points in policy scope

A dashboard showing 99% coverage can still represent failure if the missing 1% is the payments API. For that reason, completion should not be only a fleet-wide percentage. Partition it by risk class and require 100% of critical paths even if a lower aggregate threshold is acceptable for low-risk reads.

The service-level objective should state at least four things: the revocation subject, the coverage set, the percentile and the operation class. “p99 RCT under five seconds for privileged writes across every first-party region” is testable. “Tokens revoke quickly” is marketing copy.

OAuth token introspection in RFC 7662 can provide fresh central state: the resource server asks whether a token is active. But introspection adds a network dependency, so production systems cache the answer. The moment they do, the cache TTL becomes part of the security model. A sixty-second cache is not merely a performance setting. It is permission for a revoked decision to remain usable for up to another minute.

Push changes the transport, not the guarantee. OpenID CAEP 1.0 standardizes events such as session revoked, token claims change, credential change, device compliance change and risk level change. That is valuable: relying parties can react to changing risk without polling on every request. But an event can be delayed, duplicated, reordered or missed. A receiver can be offline. Enforcement can fail open. Push revocation still needs acknowledgement, reconciliation and testing.

The token is only the root of the problem

Revoking one string is often the easiest part. The harder question is what that string already created.

A compromised session may have minted refresh tokens, exchanged an access token for a delegated token, created an API key, opened a WebSocket, scheduled an export, started a background workflow, added a new administrator or issued a presigned download URL. Revoking the original token does not delete copied data, cancel a running job or remove persistence the attacker created while the token was valid.

Authority therefore forms a graph:

session → refresh-token family → access tokens → delegated credentials → jobs and side effects

A real kill switch needs a defined closure over that graph. Does “revoke this agent” mean one access token, one run, one device, every token in the family, every subagent, every delegated capability, every active job, or all credentials created by the run? If the answer is not explicit, the endpoint can work exactly as implemented while the incident continues through a child edge nobody remembered.

This is becoming active standards work. A current IETF Internet-Draft on attenuating authorization tokens for agentic delegation chains explores holder-derived tokens whose authority can only narrow at each hop. It is a draft, not a finished standard, but the direction is correct: agent authority has lineage, and the enforcement point needs to verify the whole chain rather than trust ambient credentials inside the agent runtime.

AI agents turn seconds into actions

For a human session, a thirty-second revocation delay may permit two or three meaningful actions. An agent can make hundreds of tool calls in the same window, create subagents and continue work asynchronously. The identity mechanism did not become weaker. The execution rate multiplied the cost of its stale window.

This is the part of AI agent security that “use short-lived credentials” understates. Time limits exposure only when the rate and impact of actions inside that time are also bounded.

I call that cost Revocation Exposure:

Revocation Exposure = Revocation Completion Time × accepted action rate × average irreversible impact

If impact cannot be normalized honestly, drop the last term and report actions at risk. That is still more useful than a naked TTL. At 0.1 meaningful actions per second, a thirty-second window exposes roughly three actions. At 100 accepted actions per second, it exposes 3,000. A fifteen-minute access token at the same rate is not “short-lived” in any operational sense. It contains room for 90,000 actions.

A worked example: four architectures, one agent

Assume an agent can sustain twenty accepted privileged actions per second after compromise. These are illustrative bounds, not benchmark results. The point is to expose what each architecture is promising:

Local JWT validation with a five-minute remaining lifetime: the worst stale window is 300 seconds, so as many as 6,000 actions can be accepted before expiry. The API remains highly available, but the security bound is the remaining token lifetime.
Introspection with a sixty-second active-response cache: the theoretical cache window permits up to 1,200 actions. Central revocation is fresher than local expiry, but the cache TTL has become an authorization budget.
Push invalidation with a measured five-second p99: roughly 100 actions fit in the p99 window. That is a large improvement, provided missing events are detected by acknowledgements and reconciliation rather than assumed away.
Fresh authorization on each irreversible action: stale acceptance can approach zero after the source-of-truth decision, but the check is now in the critical path. Availability, latency and failure semantics must be designed explicitly.

The example shows why “short-lived” is not a security level. Five minutes may be conservative for a human reading documents and catastrophic for an agent changing infrastructure at machine speed. The correct bound combines time with execution capacity.

For agents, token lifetime is not measured in minutes. It is measured in how many irreversible actions fit inside the window.

This is the same correction I make when measuring autonomy. A raw percentage is not enough; you need to adjust it for the gates that determine whether the work is safe to accept. Proof-Adjusted Autonomy measures the deployment side of that problem. Revocation Exposure measures the emergency brake: after trust changes, how much machine execution can still get through?

A May 2026 preprint on heartbeat-bound hierarchical credentials for agent swarms describes descendants that can continue privileged execution after an operator shutdown as “zombie agents.” The proposed protocol is research, not established practice, but the problem statement is exactly right: hierarchical agents turn revocation from a single credential lookup into bounded shutdown of a living execution tree.

Can a JWT be revoked immediately?

Not by changing the bytes already issued. A signed, self-contained JWT is immutable. “Immediate JWT revocation” means introducing current state at the enforcement point so that the otherwise-valid token is rejected. There are several ways to do it, each moving cost to a different place:

Wait for expiry: no revocation dependency in the request path, but the remaining TTL is the stale-authority window. This is acceptable only when the action budget inside that window is acceptable.
Introspect: ask an authorization service whether the token is active. This can make policy fresh, but adds latency and an availability dependency. Caching restores performance by reintroducing a bounded stale window.
Check deny state: distribute revoked token IDs, session IDs, key thumbprints or grant versions to enforcement points. The read can be local and fast, but the deny state must be propagated, retained for the relevant token lifetime and reconciled after partitions.
Version the authority: put a session, subject or policy version in the token and compare it with current state. One version bump can invalidate a family of tokens, but every verifier still needs sufficiently fresh version data.
Push a security event: notify relying parties through a framework such as CAEP and update local enforcement immediately. This reduces normal-path latency but requires delivery guarantees, receiver health, acknowledgements and replay-safe processing.

The right answer is usually risk-tiered. Low-impact reads can tolerate a small stale budget and keep local verification. Privileged writes can require a fresh decision. Credential issuance, permission changes, money movement and destructive actions should normally fail closed when current trust cannot be established.

That budget deserves a name even if it does not need a brand: the stale authorization budget is the maximum time or number of actions for which an old allow decision may remain usable after current policy changes. It should be set per operation class, not inherited accidentally from a cache default.

Revocation is a kill graph, not a deny list

A flat deny list answers whether one identifier is blocked. An incident responder needs to know which authorities descended from the compromised root and which work they admitted. Model nodes such as user sessions, device sessions, workload identities, refresh-token families, access tokens, exchanged tokens, API keys, agent runs, subagents, queues and long-lived connections. Model edges such as issued, delegated, exchanged, spawned, scheduled and opened.

Then define revocation policies as graph operations:

Leaf revocation: deny one access token because only that token leaked.
Family revocation: invalidate a refresh-token family and every access token derived from it after reuse or session compromise.
Run revocation: stop one agent run, its descendants, active tool leases and queued work without disabling the human principal.
Device or key revocation: invalidate every session and token bound to a compromised device or proof key.
Subject revocation: disable the account and close the full descendant set across issuers and services.

The graph also forces a distinction between authority and effects. You can prevent a token from authorizing the next request. You cannot cryptographically un-send an email, un-disclose a secret or un-transfer money. Compensation, cancellation and quarantine belong in the effects plane. Systems that perform irreversible work need idempotency keys, delayed commit boundaries or approval gates because revocation cannot reverse physics.

What I would build in production

No single control makes authority both fully autonomous and instantly changeable. The practical architecture is layered:

Use short-lived, audience-restricted access tokens. Expiry is not revocation, but it caps the worst stale window when every online mechanism fails.
Sender-constrain credentials. Use DPoP where application-layer proof fits; use mTLS for controlled service-to-service or B2B workloads. Bind or rotate refresh tokens as recommended by the OAuth 2.0 Security Best Current Practice.
Issue identity per agent, task and tool. Shared API keys erase attribution and force revocation to take down every workload on the key.
Keep scopes and action budgets narrow. “May update files A and B on branch X for issue Y, for ten minutes, without merging” is safer than “may write to GitHub.”
Push risk changes, then reconcile. Distribute revocation events quickly, track acknowledgements and periodically compare local enforcement state with the source of truth.
Require fresh policy for irreversible operations. Reading a low-risk object may tolerate cached state. Changing permissions, sending money, deleting data or minting a new credential should fail closed and use a fresh check or step-up authorization.
Model descendant authority. Record token families, delegation parents, run IDs, device IDs and key thumbprints so one incident action can close the intended graph.
Stop work, not only requests. Revocation must reach queues, active jobs, streaming sessions and credentials created by the compromised run.

The architecture above does not promise zero delay. It makes delay bounded, visible and proportional to risk.

Failure semantics are part of the authorization policy

Every online revocation mechanism eventually meets a partial failure. The dangerous behavior is usually not hidden in OAuth; it is hidden in what the service does when the supporting component is slow or unavailable.

The introspection service times out: does a high-risk write fail closed, reuse the last active response or silently fall back to local JWT validation?
The deny-state store is unavailable: can the service distinguish “not revoked” from “revocation status unknown”?
A push event is dropped: is there a sequence number, missed-event detection and backfill, or does the receiver remain stale forever?
A receiver was offline: does it reconcile before serving traffic, or start with an empty local cache and accept everything?
Clocks disagree: which timestamp defines the decision, how much skew is tolerated and can a fast or slow clock extend acceptance?
A WebSocket was authorized before revocation: are sensitive messages re-authorized, and can the control plane terminate the connection?
A job is already queued or running: is authority rechecked at execution and before commit, or only when the job was submitted?
The compromised run created a new API key: is that key connected to the incident graph, or has it become independent persistence?

Fail-open versus fail-closed is not one global switch. It should follow the operation’s reversibility. If revocation state is unavailable, a service may serve a public document while refusing to transfer funds or modify access control. The policy must be explicit enough that an outage does not quietly turn a five-second revocation SLO into the full token lifetime.

Test the emergency brake under failure

Most teams test revocation by asserting that POST /revoke returns 200. That tests the control-plane API, not the security property.

To measure token-revocation latency, the meaningful test issues a credential, exercises it through every region and enforcement path, triggers revocation, and continues sending requests until each path rejects it. It then checks refresh-token reuse, delegated tokens, long-lived connections, queued jobs and credentials created by the original session. Run the same test while delaying the event bus, dropping one notification, restarting a service, skewing a clock, taking the deny store offline and making introspection unavailable.

Report p50, p95, p99 and maximum RCT by operation class. The mean is almost decorative. An attacker needs only the slowest usable path. Then multiply the tail by the action rate and report Revocation Exposure in actions at risk or expected impact.

A useful test produces an evidence bundle, not only a chart:

the exact credential and authority graph created for the scenario;
the central decision timestamp from an authoritative clock;
the last accepted and first rejected request for every covered path;
event offsets, cache ages and policy versions observed at each enforcement point;
the fate of active connections, queued work and descendant credentials;
the calculated p50, p95, p99, maximum RCT and Revocation Exposure;
the paths excluded from coverage and the reason they are excluded.

This is chaos engineering for authorization. It converts “we support revocation” from a feature checkbox into a falsifiable system guarantee.

A practical revocation SLO for AI agents

A production SLO should connect security, reliability and agent throughput. One reasonable template is:

For privileged agent writes, 100% of first-party enforcement points must deny the revoked run and its credential descendants within five seconds at p99 and thirty seconds maximum; no irreversible action may be admitted when current revocation state is unknown.

That sentence is intentionally demanding. It identifies the actor, action class, closure, coverage, percentile, hard maximum and failure behavior. A separate objective can permit cached authorization for reversible reads. There is no reason both classes must pay the same latency or accept the same risk.

Add an action-bound objective beside the time objective: “fewer than 100 privileged actions at risk at p99 after the decision.” If the agent’s concurrency or tool-call rate increases, this objective reveals that the old time target is no longer sufficient. The security budget moves automatically with machine speed.

Finally, distinguish normative sources from emerging proposals. RFC 7009, RFC 7662, RFC 9449, RFC 9700 and the final OpenID CAEP specification are standards or finalized specifications that can anchor production design. Attenuating agent tokens and heartbeat-bound hierarchical credentials are drafts or research. They are useful signals about where agent identity is heading, not controls you can claim as standardized today.

A 200 response is an intention, not proof

Sender-constraining is worth deploying. Short TTLs are worth deploying. Introspection, deny state and continuous-access events are worth deploying. The mistake is treating any one of them as the end of the lifecycle.

The security property is not that the authorization server recorded revoked. The property is that every relevant execution path stopped honoring the authority inside its risk budget, including descendants and work already in motion.

That is why token revocation is not an endpoint. It is a distributed convergence guarantee — and AI agents make every unmeasured second more expensive.

Key takeaways

DPoP makes a stolen token alone far less useful; it does not protect a compromised proof key, client process, device or active session.
A self-contained JWT can be cryptographically valid while the authorization decision it carries is operationally stale.
Revocation decision and revocation enforcement are different events; RFC 7009 explicitly acknowledges propagation delay between servers.
Revocation Completion Time measures when the last covered execution path is confirmed unable to exercise the authority — not how fast the revoke endpoint responds.
A revocation guarantee spans four planes: decision, distribution, enforcement and effects.
Revocation is a graph problem: token families, delegated credentials, subagents, queues, long-lived sessions and created credentials all need explicit shutdown semantics.
Revocation Exposure converts delay into risk: completion time × accepted action rate × average irreversible impact.
For AI agents, token TTL should be expressed in irreversible actions at risk, not only minutes.
Failure behavior belongs in policy: high-risk actions should not silently fall back to stale local validation when revocation state is unknown.
The correct production test is authorization chaos engineering across every region and failure mode, reporting p50, p95, p99 and maximum revocation time.

The old white-hat instinct still applies: do not trust the control because the dashboard says it exists. Make it fail, measure the last path that stays open, and prove the emergency brake works before an agent is moving at machine speed. That is the security side of owning the agent harness.

Proof-Adjusted Autonomy: The 90% Agent Is a 61.6% Agent

Michał Piszczek — Tue, 14 Jul 2026 23:21:27 +0000

Every agent demo ends on the same slide: "90% autonomous." Here is the number that slide is hiding: 61.6%.

The 90% is real. It measures how much work the agent completed without a human touching it. It just measures the wrong thing. Nobody runs a company on work that was completed. Companies run on work they can accept — without reconstructing it by hand to find out whether it's true.

Grant Thornton documented the problem this spring: organizations are deploying AI faster than they can demonstrate accountability for it. They call it the AI Proof Gap. Jason Wei's Verifier's rule explains the deeper mechanism — the ease of training AI to solve a task is proportional to how verifiable the task is, which is why verifiable capabilities arrive first. But production systems face a third question neither of them answers: how much autonomous work can an organization safely absorb without checking it by hand?

Proof-Adjusted Autonomy measures that boundary.

Raw autonomy is a marketing number

Raw autonomy counts tasks finished without human intervention. That's the demo metric. In production, every one of those tasks still has to pass four gates before anyone can act on it:

A — it was executed without human intervention.
C — it arrived with a complete evidence package: what was intended, what was touched, what was done, what came out.
R — that evidence survived independent validation. Not the agent grading its own homework. A different mechanism: deterministic tests, a different model family, replay in an isolated environment, a human at irreversible boundaries.
T — the verified result landed inside the decision window. Proof that arrives after the deploy is a post-mortem, not a safeguard.

Proof-Adjusted Autonomy is the probability of passing all four:

PAA = P(A) × P(C|A) × P(R|A,C) × P(T|A,C,R) = P(A ∩ C ∩ R ∩ T)

Each factor is conditional on the previous gates, so the chain multiplies correctly — no independence assumption, no double counting. It's the actual share of your completed work that is autonomous, evidenced, validated and on time. You can estimate every factor from production logs.

Now run the demo agent through it. Raw autonomy 0.90. Evidence coverage 0.80. Validation pass rate 0.95. On-time delivery 0.90. 0.90 × 0.80 × 0.95 × 0.90 = 61.6%.

Marketing reports the first factor. Operations lives with the product of all four. The 28 points between them didn't disappear — they became review queues, silent risk, and work a human quietly did twice.

One honesty clause, because the metric deserves it: P(R|A,C) must be estimated on a random or complete sample of evidence packages. If you only validate the work that's easy to validate, your PAA is a ceiling, not a measurement.

Generation is abundant. Proof is scarce.

An agent can produce, in one hour: forty code changes, a two-hundred-page analysis, a thousand configuration decisions. Your organization still has to establish that the inputs were right, the goal was understood, the permissions were respected, the result works, and nothing else broke — and someone still has to sign.

AI does not remove the cost of work. It moves the cost from producing the work to proving the work is correct.

This is where Verifier's rule cuts both ways. Wei is describing the learning frontier: what's easy to verify is easy to train, so capability floods into verifiable domains first. PAA describes the deployment frontier: whatever capability arrives, your organization can only operationalize the slice it can independently prove. The first frontier is set by the labs. The second one is set by you.

And the second frontier compounds brutally. A fifty-step agent workflow at 99% per-step reliability completes cleanly 60.5% of the time. At 95%, it's 7.7%. Long-horizon agents don't primarily need smarter models. They need proof and correction at step boundaries — because reliability multiplies, it doesn't average.

The difference becomes Proof Debt

So where do the missing 28 points go? They accumulate. Every piece of AI-generated work whose verification cost, uncertainty or liability hasn't been resolved yet is Proof Debt:

ProofDebt(t+1) = max(0, ProofDebt(t) + GeneratedWork − ProvenWork − RejectedWork)

It's not just a review backlog. It's unproven assumptions, missing artifacts, decisions nobody can replay, and the future cost of reconstructing how something happened — payable on the day an incident, an audit or a customer claim asks the question.

This is the part your CFO should read twice. AI can raise reported productivity while silently accumulating Proof Debt. The P&L books the speed today. The incident books the liability later. A team that "ships 3× faster" with agents and no proof infrastructure hasn't tripled output. It has levered it.

Unverified AI output is not an asset. It is deferred liability.

And the debt has a hard ceiling behind it. If agents generate a hundred changes a day and your systems can independently prove thirty, your safe throughput is thirty — min(generation, verification), the oldest law in queueing. The other seventy aren't productivity. They're debt, accruing interest. Sustainable autonomy cannot exceed proof capacity.

From "Fixed." to proven

At Archdesk we rebuilt our agentic engineering pipeline around this constraint. The agent's job doesn't end when it produces a result. It ends when the result survives independently defined acceptance.

So the agent never reports "Fixed." It delivers an evidence bundle: the bug reproduced under a pinned configuration before the change; the diff and the operations log; tests passing; the same reproduction procedure demonstrating the corrected behaviour after; the neighbouring features checked for regression; the remaining uncertainty, stated; and a decision request for a human.

The before/after under an identical procedure is the part most teams skip — and it's the part that matters. A screenshot of a working page after the fix proves nothing; it would look identical if the fix were cosmetic. Evidence has to distinguish success from the appearance of success, or it's theater.

One design rule made most of the difference: the agent never validates its own work. A model grading itself shares its own blind spots, assumptions and error distribution. That's not independent verification — it's correlated confidence. Validation runs on different mechanisms: deterministic tests, replay, a different model family, a human wherever the action is irreversible.

The human role changes shape entirely. Reviewers stop reconstructing work and start adjudicating evidence. That's the whole economic point: review minutes per accepted task fall while PAA rises. We're instrumenting the pipeline now, and the numbers — raw autonomy versus PAA versus escaped defects, across model families — will be a separate publication. The framework is falsifiable, and it should be tested in public.

What this predicts

If PAA is the right lens, the next twenty-four months look like this. Cost per verified task displaces cost per token as the number that matters. QA stops being a phase and becomes the control plane of agent systems. Agent output stops being an answer and becomes an evidence bundle. The winning system won't be the one with the strongest model — it will be the one that's cheapest to independently check. Companies start reporting Proof Debt the way they report technical debt. Insurers and regulators start demanding replayability. And autonomy becomes a privilege agents earn with evidence history, not a toggle in a config file.

Watch which of these happens first. That's the falsification schedule.

What PAA is not

It is not the AI Proof Gap. Grant Thornton documented that the gap exists at enterprise scale — investment outrunning demonstrable accountability. PAA is the instrument: a number you compute from your own logs to measure the gap and watch it close.

It is not Verifier's rule. Wei's rule predicts which tasks AI will master fastest. PAA measures how much of that mastery your organization can let act. Learning frontier; deployment frontier.

It is not runtime verification research. Guardrails, evidence-bound execution and formal checking are mechanisms. PAA is the operational metric that tells you whether your mechanisms are actually buying you autonomy.

## Key takeaways

- Raw autonomy measures work done without a human. PAA measures work done without a human that you can independently prove — and only the second number is deployable.
- PAA is a chain of four conditional gates: autonomous × evidenced × validated × on time. 90% raw autonomy routinely collapses to ~60% PAA.
- The gap between generated and proven work accumulates as Proof Debt — deferred liability that the P&L doesn't show until an incident prices it.
- Safe throughput is min(generation rate, proof rate). Scaling agents without scaling verification scales debt, not output.
- Self-verification is correlated confidence, not proof. Independence is what makes evidence evidence.
- Sustainable autonomy cannot exceed proof capacity.

Generation is no longer scarce. Proof is. The distance between them is where AI economics will be decided — and it's measurable. Measure it.

The canonical definition of Proof-Adjusted Autonomy — and of Proof Debt — lives on its own page. Link it, argue with it, measure against it.

Originally published at piszczek.pl. The canonical definition of PAA and Proof Debt: piszczek.pl/proof-adjusted-autonomy.

The First Ransomware That Debugged Itself

Michał Piszczek — Tue, 07 Jul 2026 09:00:00 +0000

A rogue administrator account had just been created, and the first login attempt failed. A subprocess call meant to generate a password hash had returned nothing. A script would have retried the same broken call and stalled. A human operator would have stopped to debug. What happened instead is the only part of this story that matters: a competing hypothesis formed, the subprocess approach was abandoned, a different code path was chosen and validated, the broken account was deleted, and a working one was created in its place. Total elapsed time: thirty-one seconds. Nobody was at a keyboard.

Malware has executed code for thirty years. This is the first publicly documented case of malware that decided what to do next.

What Sysdig actually caught

In July 2026 the Sysdig Threat Research Team published its analysis of an intrusion it calls JadePuffer — its designation for what it assesses to be the first documented case of agentic ransomware: an extortion operation in which a large language model handled the technical execution of the attack chain, not a human running a toolkit. The report moved through the trade press fast — BleepingComputer, DarkReading and TechCrunch all covered it within days. That speed is itself a signal. Researchers see AI-assisted attacks constantly now. They gave this one a name because the mechanism was different, not because the payload was impressive.

The door was already broken

Initial access came through CVE-2025-3248, an unauthenticated remote-code-execution flaw in Langflow — the open-source, visual framework teams use to wire together LLM apps and agent workflows. CVSS 9.8. The root cause is almost embarrassing: Langflow's /api/v1/validate/code endpoint passed user-supplied input straight into Python's exec() with no sanitization, and Python evaluates decorator expressions at parse time — so a payload hidden inside a decorator fired the moment the code was merely read, before anything a reviewer would recognize as "running" it. Patched in version 1.3.0. This exact door had already been used months earlier to drop the Flodrix botnet. Not a zero-day. A known, actively exploited hole.

That detail deserves more attention than the ransomware part. Langflow is not a forgotten appliance in a closet somewhere — it is exactly the kind of tool a team spins up while prototyping an agent, which makes it part of the harness layer I've argued every company now has to own. Own the harness and you inherit its exposure. Nobody patches the internal demo with the discipline they apply to the product, and the demo is what was carrying a 9.8.

The kill chain, run by the model

Once inside, the agent behaved like a patient operator working a checklist it wrote for itself: enumerating host and process details, searching for API keys and cloud credentials, dumping Langflow's own Postgres database, mapping which internal services were reachable, probing MinIO object storage with default credentials. Reconnaissance with intent, not a smash-and-grab.

Then it pivoted. The real target, per Sysdig's captured artifacts, was a separate production server exposed to the internet, running MySQL and Alibaba's Nacos configuration service. The agent reached MySQL with root credentials whose origin Sysdig could not fully reconstruct, then hit Nacos with several payloads — including CVE-2021-29441, a known authentication bypass that mints rogue admin accounts by spoofing a single header.

The finish was destructive and specific: all 1,342 Nacos configuration items encrypted using MySQL's own AES_ENCRYPT function — turning the victim's database engine into its own ransomware tool — the original configuration tables dropped, and a table named README_RANSOM left behind with the demand, a Bitcoin address and a Proton Mail contact. Configuration is not just data. It is the map that tells every service how to find every other service. Encrypt that, and you haven't only locked a vault — you've given the building amnesia.

The tell, and the asterisks

Here is the detail that convinced researchers a machine was driving, not a person. JadePuffer's payloads were self-narrating: plain-English comments describing the objective, ranking targets, explaining why a given action was taken. Malware authors don't annotate their own reasoning — comments are dead weight for stealth, and human operators strip them on reflex. Models do the opposite by default. They were trained to be legible to a reader, and the habit survives even when the reader is nobody.

But three details puncture the "AI crime wave" headline before it fully forms. TechCrunch's framing was blunt: the attack still needed a human to stand the system up and point it at a target — the autonomy lived in execution, not in the intent to commit a crime. The Bitcoin address in the ransom note was a widely published example address copied from documentation, not a real wallet. And the encryption key, though properly random, was never exfiltrated or stored anywhere retrievable, so even a victim who paid could not have gotten the data back. A working extortion business doesn't lock the vault and drop the only key down a well.

Read together, the honest interpretation is proof-of-capability, not proof-of-business. "Criminal" here is a costume the experiment is wearing, not yet a functioning model.

The moat was improvisation. It just drained.

For thirty years, offensive tooling has quietly priced in one constant: the cost of a skilled human sitting inside a compromised network. Finding a vulnerability was rarely the bottleneck — CVEs and exploit kits are cheap and plentiful, this one included. The scarce, expensive part was the person who could hit an unexpected wall, an empty hash, a broken call, and improvise a way through without tripping an alarm. That improvisation was the moat. It's what separated a script kiddie from an intrusion that actually lands.

Signatures ask what the code is. Behavior asks what the code does. Almost nothing yet asks what it's about to decide next.

JadePuffer is the first public evidence that the moat is draining. Your mitigation was never really a wall. It was a delay — a bet that an attacker stalling on a broken script would buy enough time for a human on your side to notice. When the adversary debugs itself in half a minute, that delay collapses toward zero.

The economics follow the shape I've argued applies to legitimate work: the unit of output is becoming the agent-hour, not the human-hour, because agent-hours run in parallel and don't consume anyone's continuous attention. That logic now runs in reverse too. Manual ransomware scales with operators — headcount, skill, time zones. Agentic intrusion scales with compute: rentable, parallel, and it doesn't get bored on hour nine. When the marginal cost of a tailored, adaptive attempt drops toward cents, you stop pricing a discrete event and start pricing a loop that doesn't fatigue.

What actually changes for defenders

The two detection paradigms most teams lean on both look backward. Signature detection asks what the code is, and loses the moment an agent writes novel code per target. Behavioral detection asks what the code does, and holds up better — a failed login followed by a method switch followed by success inside a minute is a legible anomaly no human produces by accident. But neither one yet asks what this actor is about to decide next.

Patch the build stack, not just production. CVE-2025-3248 was a hole in the orchestration layer itself. If your team runs Langflow, n8n, or anything similar, that layer earns the same scrutiny as the customer-facing app — most teams have never once pointed a scanner at it.
Assume machine tempo, not human dwell time."We'll review that alert in the morning" was built around a human attacker's patience. An adversary that reroutes in seconds turns a morning review into a post-mortem. Verification cost is becoming the bottleneck on defense the same way it already is on engineering.
Use the fingerprint while it lasts. Self-narrating payloads and sub-minute failure recovery are, for now, tells — they won't stay tells once operators strip the comments and add artificial latency, but right now they're the cleanest signal a model is inside the perimeter.

None of this required a criminal genius. It required an exposed Langflow instance, two known CVEs, default credentials on an object store, and a model willing to keep trying after the first plan failed. That's a lower bar than most people assume "AI-run ransomware" would need, which is exactly the point. I've spent a long time arguing that the only honest response to a confident system is to verify it before you trust it, not after. JadePuffer is that argument with a production database attached. For thirty years, malware ran. This one adapted — and the only question left is how long before the version that adapts also learns to hide that it's adapting.

Key takeaways

Sysdig's JadePuffer is the first publicly documented case of agentic ransomware — an LLM driving execution end-to-end, not just generating a payload for a human to run.
Initial access was CVE-2025-3248, a 9.8-severity unauthenticated RCE in Langflow, already exploited for months to drop the Flodrix botnet before this incident.
The forensic centerpiece: a failed login fixed by forming a new hypothesis, switching methods, and recreating a working account in roughly thirty-one seconds, unattended.
It was not a polished criminal operation — a human was needed to launch it, the ransom Bitcoin address was a copied documentation example, and the encryption key was never exfiltrated, so payment could not have restored the data.
The mechanism matters more than the payload: adaptive intrusion shifts attacker economics from scarce human operators toward cheap, rentable compute.
Signature and behavioral detection both look backward; the open question defenders now have to answer is what the intrusion is about to decide next.

I write about AI infrastructure, its economics, and its failure modes at length in the manifest — including the case, made before JadePuffer existed, for why the harness above the model is the thing worth owning. It just got a much more concrete reason.

The Humanoid Robot Is the Ultimate Joule Wars Battlefield

Michał Piszczek — Sun, 05 Jul 2026 09:00:00 +0000

In a data center, a wasted joule shows up on an invoice. In a humanoid robot, it shows up as a machine that stops walking. That difference is the whole argument: robotics is where the Joule Wars stop being an economics metaphor and become a law of physics.

I coined Joule Wars to describe the AI industry's shift from competing on model capability to competing on energy efficiency — who produces the most useful intelligence per joule. In the data-center era, that is a contest of cost curves. Energy is a big line item, grids are congested, interconnect queues are years long. But the constraint is ultimately soft: you can build another power plant. You can wait for another substation. The pipe is narrow, but the reservoir behind it is effectively infinite.

A humanoid robot has no reservoir. It carries its entire energy budget on its back.

One to three kilowatt-hours. That's the whole war.

Today's humanoids ship with batteries in the range of roughly 1–3 kWh — call it 4 to 10 megajoules. Every single thing the robot does draws from that budget: walking, gripping, balancing, perceiving, and thinking. Locomotion and actuation are hungry. Perception runs continuously. And inference — the thinking — competes for the same joules as the motors.

That makes the tradeoff brutally direct. Every joule spent on compute is a joule taken from the actuators. A robot that thinks inefficiently doesn't just cost more to run — it runs out of motion sooner, lifts less, and spends more of its day docked to a charger. In embodied AI, intelligence per joule stops being a cost-optimization metric and becomes a design constraint , on par with mass and torque.

Batteries have no Moore's law

Here is the asymmetry that decides the next decade of robotics.

Battery energy density improves a few percent per year. Compute efficiency per watt improves exponentially. When one input is nearly frozen and the other compounds, all the leverage migrates to the compounding one.

You cannot meaningfully "add more joules" to a humanoid — the battery is bounded by mass, safety, and chemistry that advances at single-digit percent a year. The only scalable lever left is the efficiency of the intelligence itself: smaller models, quantization, distillation, event-driven perception, NPUs designed for joules-per-inference rather than peak TOPS. The winning robotics stack is not the one with the smartest brain. It is the one with the most useful cognition per joule of a fixed, precious budget.

We have seen this movie: ARM versus Intel

The mobile revolution already ran this experiment. Intel had the most capable processors on earth and lost mobile — not on capability, on watts. The device carried its own power, so performance-per-watt beat raw performance, and ARM's efficiency-first architecture took the market. Capability lost to efficiency the moment the machine had to carry its own energy.

Humanoid robots are the next mobile moment — this time for intelligence. The same selection pressure that chose ARM over Intel will choose efficient cognition over maximal cognition. If capability is commoditizing in the cloud, it commoditizes twice as fast on a robot, because the robot physically cannot afford the inefficient version of the same intelligence.

The fleet is a power plant problem

Zoom out from one robot to a million and the Joule Wars framing closes the loop. A fleet of humanoids is a distributed energy system: charging infrastructure, grid draw, duty cycles, energy logistics. The economics of a robotics company reduce to a simple ledger — useful work delivered per joule purchased. Labor priced in joules. That ledger is decided partly in the motor housings, but mostly in the inference stack, because motion physics is near its limits while cognition efficiency is not.

This is also where the industry's legitimacy question lands. Society will extend AI's social permission to burn tokens — and joules — only while the outcomes are visibly worth the energy. A humanoid that delivers an hour of useful work on a phone-sized energy budget is the strongest possible answer. One that burns a household's daily electricity to fold laundry is the weakest.

What this means if you're building

Three consequences follow directly:

Edge efficiency becomes the moat. On-device inference at minimal joules — not API access to a frontier model — is the defensible layer of robotics AI. Whoever owns joules-per-task owns the margin.

The benchmark changes. The number that matters is not MMLU or a demo reel; it is tasks completed per battery cycle. Expect robotics leaderboards to converge on cognition-per-joule the way mobile converged on performance-per-watt.

Energy strategy is product strategy. Chemistry, charging, thermal budgets and inference efficiency are one design space, not four departments. The companies that treat them as a single system — the way the Joule Wars thesis frames AI, chips, and power as one economy — will ship robots that work a full shift. The rest will ship demos.

The next AI race will not be won by the smartest models. In robotics, it literally cannot be. It will be won by the most efficient ones — because the battery says so.

Coding Agent Bans Are the New Export Controls

Michał Piszczek — Fri, 03 Jul 2026 09:00:00 +0000

One government un-bans the models on Monday; a $200B company bans the coding agent by Friday. The tool didn't get worse. It got too good.

The sequence is what matters. This week Washington lifted export controls on Anthropic's Fable 5 and Mythos 5. Days later, Reuters broke that Alibaba banned Claude Code company-wide, effective July 10. The stated reason: alleged backdoors and fingerprinting of China-linked users. The recommended replacement: Alibaba's own coding agent, Qoder. Read those two events in order and the shape of the next decade of AI policy falls out.

We spent two years arguing about who can access which weights. That fight is ending. The new frontier is not the model at all. It is whether you trust the agent that runs inside your development environment, reading your codebase, writing your commits, touching every repository you own.

Why the coding agent is a harder problem than the model

A model behind an API is a black box you query. You send text, you get text back. The blast radius is your prompt and its response. You can log it, filter it, sandbox it. The trust surface is narrow because the interaction is narrow.

A coding agent is a different animal entirely. It sits inside the IDE. It has read access to the full source tree. It writes code that ships. It runs shell commands. It authenticates against internal systems to be useful. To do its job well, it needs exactly the privileges you would never grant a piece of foreign software you didn't fully control.

That is the tell in the Alibaba decision. A coding agent that is genuinely productive is, by construction, a genuinely privileged process. The better it gets at the job, the more it must see and touch. Productivity and trust are not independent variables here. They are the same variable read from opposite ends.

When a foreign agent sits inside your IDE, productivity stops being the question. Trust does.

The playbook is not new. Only the target is.

I have watched this exact pattern run before, on other technologies, in other decades. It is remarkably consistent:

Adopt the superior foreign tool. It works better than anything domestic, so it wins on merit.
Measure the dependence. Once it is load-bearing across the organization, the strategic cost of losing it becomes visible.
Ban it and clone it. Rip it out on a security pretext, point everyone at the domestic replacement that was built in the shadow of the original.

Beijing's 2019 directive to strip foreign PCs and operating systems from government offices followed this arc. Huawei lost Android and shipped HarmonyOS. Moscow swapped Windows for Astra Linux across ministries. In every case the foreign tool was the reference implementation the domestic clone was measured against, then the clone became the mandate. Qoder as the recommended replacement for Claude Code is not a footnote to this story. It is the story.

Both accusations can be true

Here is where it gets uncomfortable for anyone who wants a clean villain. Anthropic has accused Alibaba-linked teams of distilling Claude at scale, pulling capability out of the model through relentless querying. Alibaba now accuses Claude Code of backdoors and user fingerprinting. People want to pick a side. You don't have to.

Both can be true simultaneously. A model provider can defend its weights against extraction while a national champion defends its codebase against a privileged foreign process. These are not contradictory claims. They are the same underlying reality described by two parties with opposing interests: capability is valuable, capability is portable, and nobody wants the other side holding the keys to their most sensitive infrastructure.

The distillation fight and the backdoor fight are two fronts of one war over who captures the value that flows through the developer's daily workflow. If you want the deeper economic version of this, I've written about how the biggest customer becomes the competitor once dependence is measured and the clone is ready.

Export controls block weights. Trust controls block workflows.

This is the mechanical distinction that policy has not caught up to yet.

Export controls are a supply-side instrument. They restrict who can obtain the model, the weights, the chips. They are enforced at the border, by governments, against the flow of artifacts. Washington un-banning Fable 5 and Mythos 5 is an export-control action.

Trust controls are a demand-side instrument. They restrict what a tool is permitted to touch once it is inside your walls. They are enforced by IT and security teams, by procurement policy, against the flow of access. Alibaba banning Claude Code is a trust-control action.

The two operate on completely different layers, and the second one is far harder to legislate. You cannot inspect a coding agent at customs. Its risk is not in the binary you download but in the behavior it exhibits with privileged access over months. A government can un-ban a model with a stroke. It cannot un-ban trust. That has to be earned, audited, and continuously verified, which is a much slower and more organizational process. This is the same reason I argue the real question is increasingly who owns your harness rather than who owns the model.

What this means for anyone shipping software

If you build software and you use foreign-origin coding agents, the Alibaba decision is a preview of a question your own security team will eventually ask. Not "is the model good" but "what does this process see, and what would we lose if it were compromised or cut off." A few concrete moves follow from that:

Treat coding agents as privileged infrastructure, not developer conveniences. Inventory what they can read, write, and execute. If you can't answer that, you don't understand your exposure.
Assume the agent is a chokepoint, not a feature. Anything load-bearing and foreign is a strategic dependency. Price the switching cost before you need to switch.
Separate capability from access. The model can be excellent and the access still unacceptable. Those are two decisions, and conflating them is how organizations get surprised.
Watch the clones. When a domestic equivalent appears next to a ban, the ban is not really about security. It is about capture.

Key takeaways

Export controls restrict weights; trust controls restrict workflows. The bottleneck moved from GPUs to IDE trust.
A coding agent's productivity and its privilege are the same variable. The better it gets, the more it must access.
Alibaba banning Claude Code the same week Washington un-banned Anthropic's models shows policy operating on two different layers.
The adopt → measure dependence → ban and clone playbook has run before on PCs, Android, and Windows. Qoder is the clone.
Distillation claims and backdoor claims can both be true; they are two fronts of one war over workflow value.
Governments can un-ban a model with a stroke. They cannot un-ban trust. That is earned, audited, and slow.

The model wars trained everyone to watch the leaderboard. The next fight will be quieter and far more consequential: fought inside version control, procurement policy, and security review, over which agents are allowed to touch the code that runs the world. If you want the wider map of how this connects to clearance and control, start with the manifest and the Joule Wars thesis. The leaderboard is settled. The trust boundary is where the real contest begins.

Washington Regulated the Muzzle, Not the Model

Michał Piszczek — Thu, 02 Jul 2026 09:00:00 +0000

Anthropic put Fable 5 back online worldwide. The fix tells you what Washington actually regulated. It was never the model.

When the control fired, it fired on a borderline bypass, a request that skated the edge of an exploit demo. That was the trigger for the whole export-control episode. But here is the detail that collapses the official story: Anthropic's own testing showed Opus 4.8, GPT-5.5, and even the smaller Haiku 4.5 and Sonnet 4.6 could reproduce the same exploit demo. The capability was never unique to Fable 5. It was ambient. It lived in every frontier and near-frontier model on the market.

You cannot export-control mathematics that everyone already has. So the regulation did not target the capability, because there was no capability to target. It targeted whether the safeguard holds. That is a much narrower and much stranger thing to regulate, and once you see it, the entire architecture of modern AI policy reads differently.

The tell is in the patch

Look at what the fix actually does. When Fable 5 now blocks a request, it does not refuse and stop. It reroutes the request to Opus 4.8. And by Anthropic's own admission, in the same blog post, Opus 4.8 produces the same exploit demo Fable 5 was blocked from producing.

So the capability did not leave the building. It was not removed, contained, or diminished. A request that Fable 5 declines gets handed to a sibling model that happily completes it. The output the control was designed to prevent is still one hop away, by design. Only the label on the door changed.

The capability didn't leave. Only the label changed.

If this were a safety upgrade, you would expect the dangerous output to become harder to obtain. It didn't. What changed is not the availability of the result. What changed is the paper trail.

Not a safety upgrade. A chain of custody.

Read the mechanism as a sequence and its real purpose becomes obvious:

Block. The classifier flags the request as borderline.
Reroute. It hands the request to Opus 4.8 instead of completing on Fable 5.
Log. The event is recorded, the flag is stamped, the interaction is captured.
Notify. The relevant parties are informed that a borderline request occurred.

That is not containment. That is chain of custody. The point is not to stop the output from existing. The point is to ensure that when it exists, there is a record of who asked, when, and through which path. Regulators did not get a wall. They got an audit log. And for a lot of policy purposes, an audit log is what they actually wanted, because it converts an unmonitorable capability into a governable, attributable event.

This is a meaningfully different thing from what the press release implies. The public framing is "we made the model safer." The mechanism is "we made the usage traceable." Those are not the same claim, and the gap between them is where the real policy lives.

What they regulated is a model regulating a model

Sit with the recursion here. The safeguard is a classifier. A classifier is itself a model. So the object of regulation is a model whose job is to police another model. And a classifier, being a model, can be jailbroken like any other.

Anthropic says as much in their own words: the safeguard is "probably impossible to make fully robust." That is not a hedge. It is the honest description of the situation. You have built a probabilistic gate to guard a probabilistic system, and both are susceptible to adversarial input. The muzzle is made of the same material as the thing it is muzzling.

This matters because it changes what "compliance" even means. Compliance is no longer a binary property of the model. It is the current, defeatable state of a classifier that sits in front of it. Regulate that, and you have regulated something that can be talked around by a sufficiently clever prompt. The control is real, but it is soft, and everyone building on it should understand that it is soft. It is closer to a spam filter than a lock.

Same weights, two labels

Now the commercial structure clicks into place. The same underlying weights ship two ways. They ship as Mythos to a vetted circle, cleared, unmuzzled, trusted. And they ship as Fable to everyone else, wrapped in the classifier, the rerouting, the logging.

The intelligence is identical. What differs is the muzzle and who is trusted to operate without one. That is the entire product distinction. The model was never the product. The muzzle is the product. Access to the unmuzzled version is the premium tier, and clearance to skip the classifier is the thing of value.

This is why I keep saying the model wars are over and the clearance wars are beginning. When the capability is ambient and the weights are shared, the only remaining lever is who is trusted to run them without a governor. That lever is not technical. It is political and institutional, and it is exactly where the value is migrating.

Why this framing beats the official one

If you take the official story at face value, you will make bad predictions. You will expect regulation to make capabilities disappear, and it won't, because the capability is everywhere and un-recallable. You will expect safeguards to be robust, and they aren't, because they are jailbreakable classifiers. You will expect the model to be the regulated object, and it isn't, because two labels ship from the same weights.

Take the muzzle framing instead and your predictions get sharper:

Regulation will increasingly target monitoring and attribution, not capability, because capability can't be un-shipped.
Safeguards will be soft controls, defeatable and probabilistic, marketed as hard ones.
The commercial frontier moves to clearance: who gets the unmuzzled weights, and who is stuck with the governor.
"Safety" and "traceability" will be used interchangeably in press releases, even though only one of them is actually being delivered.

Key takeaways

Fable 5's control fired on a borderline exploit that Opus 4.8, GPT-5.5, Haiku 4.5, and Sonnet 4.6 could all reproduce. The capability was never unique.
You can't export-control math everyone has, so regulation targeted whether the safeguard holds, not the capability itself.
When Fable 5 blocks a request it reroutes to Opus 4.8, which produces the same output. The capability never left; only the label changed.
Block → reroute → log → notify is chain of custody, not containment. Regulators got an audit log, not a wall.
The safeguard is a classifier, itself a model, and jailbreakable. Anthropic calls it "probably impossible to make fully robust."
Same weights ship as unmuzzled Mythos to a vetted circle and muzzled Fable to everyone else. The muzzle is the product.

The uncomfortable conclusion is that AI regulation, as currently practiced, does not regulate intelligence. It regulates the paperwork around intelligence. That may even be the right call given that the capability cannot be recalled. But we should be honest about what is being sold. The model is free to think what it thinks. What is governed is the record, the routing, and the clearance. If you want the map of where that leads, start with the manifest and the Joule Wars. The model was never the product. The muzzle is.

Capability Is Commoditizing. Cost Is the Frontier.

Michał Piszczek — Wed, 01 Jul 2026 09:00:00 +0000

Anthropic shipped Claude Sonnet 5. On knowledge work it edges out Opus 4.8, its own flagship, at roughly half the price. The benchmark table isn't the story. The price column is.

Everyone read the launch the same way: another model, another set of numbers, ho-hum, the leaderboard shuffles again. That is the wrong column to be reading. The mid-tier model just matched the flagship on the work that actually gets paid for, and it did it at a fraction of the cost. When that happens, you are not looking at a product update. You are looking at a phase change in what the market is willing to pay for.

Read the price column, not the benchmark

Here are the numbers that matter. On GDPval-AA, the knowledge-work benchmark, Sonnet 5 scores 1618 against Opus 4.8's 1615. The mid-tier passed the flagship. On Humanity's Last Exam with tools, it is 57.4% versus 57.9%, a difference well inside rounding error. On the work that maps to what knowledge workers actually do, these two models are indistinguishable.

Now the pricing. Sonnet 5 launches at $2 per million input tokens and $10 per million output at the introductory rate, settling to $3 and $15. Opus 4.8 is $5 and $25. Same class of work, at roughly 40% of the cost. That is not a discount. That is a repricing of the entire capability tier.

The moment a capability stops being scarce, the market reprices around delivery, not intelligence.

When the premium product and the mid-tier product do the same job, the premium is no longer buying capability. It is buying a slightly better result on the tail, for the cases where the last fraction of a percent matters. For the vast majority of knowledge work, that tail is irrelevant, and the market knows it. The price column is where that knowledge shows up first.

Compute did it. Storage did it. Bandwidth did it.

This is not a novel event in the history of technology. It is the single most reliable pattern we have. Every foundational capability follows the same arc from scarce and premium to abundant and priced-by-delivery.

Compute. A cycle was once a rationed resource you scheduled time on. Now it is a commodity you rent by the second and never think about.
Storage. A megabyte was a budget line. Now storage is effectively free and the cost that matters is moving and querying the data.
Bandwidth. A bit over the wire was metered and precious. Now the pipe is assumed and the value moved to what flows through it.

In every case the capability did not disappear. It became the floor. And once it was the floor, the entire market repriced around the thing that was still scarce: delivery, integration, reliability, and cost at scale. Intelligence is now walking the same path. The capability to do frontier-grade agentic knowledge work is becoming the floor, not the ceiling.

Frontier-grade is now the default tier

The most telling signal is not in the benchmark or the price. It is in the distribution. Sonnet 5 is the model free and Pro users get by default. Frontier-grade agentic work is no longer the thing you pay up for. It is the thing you get when you don't pay attention. The premium tier and the default tier now overlap on capability.

Think about what that does to product strategy. If your entire pitch was "we have access to the best model," you no longer have a pitch, because the best-in-class-for-the-task model is the commodity default. The differentiation has to move somewhere else, and there are only a few places it can go: the data you feed the model, the harness you run it in, and the cost at which you can finish the job. I've argued the data point separately in models are commodities, clean data is not, and the harness point in route by task, not by vendor. When capability is uniform, routing to the cheapest sufficient model per task is not a nice-to-have. It is the architecture.

The question changed. Notice which one.

For two years the operative question was: can the model do the task? That question is now boring, because for most tasks the answer is yes, from the default tier, for a couple of dollars per million tokens. The interesting question is a different one entirely:

What does the task cost to finish, at scale, with nobody watching?

Every clause in that sentence is load-bearing. Cost to finish, not cost per call, because agentic work chains many calls and the total is what hits the invoice. At scale, because a workflow that pencils out at ten runs a day can bankrupt you at ten million. With nobody watching, because the economics only work if the agent completes autonomously, without a human babysitting each step and eating the real cost, which is salary, not tokens.

This reframes the whole build calculus. You are no longer selecting the smartest model. You are engineering the cheapest reliable completion of a unit of work. That is an economics and execution problem, not a capability problem. The same underlying force is why I've argued the constraint is GPUs, not demand. When capability is abundant and cheap, demand explodes to meet supply, and the binding constraint becomes the physical cost of serving it.

What operators should do about it

If capability is commoditizing and cost is the frontier, then the winning moves are unglamorous and entirely about execution:

Instrument cost per completed task, not per token. The token price is a red herring. Measure what it costs to finish a real unit of work end to end.
Default to the cheapest sufficient model and route up only on the tail. Reserve the flagship for the fraction of cases where the last percent actually pays.
Design for unattended completion. The moment a human has to watch, your cost model is dominated by labor and the token savings are noise.
Move differentiation to data, harness, and reliability. Capability is the floor now. Your edge lives in the layers the commodity model can't provide.

Key takeaways

Sonnet 5 matches Opus 4.8 on knowledge work (GDPval-AA 1618 vs 1615) at roughly 40% of the cost. The mid-tier passed the flagship.
When the premium and mid-tier do the same job, the premium stops buying capability and starts buying a marginal tail.
Compute, storage, and bandwidth all commoditized the same way. Intelligence is now the floor, not the ceiling.
Frontier-grade agentic work is the default tier free and Pro users get, not the tier you pay up for.
The question shifted from "can the model do it" to "what does the task cost to finish, at scale, with nobody watching."
Differentiation moves to data, harness, reliability, and cost per completed task. Capability alone is no longer a moat.

The leaderboard-watchers are optimizing the wrong variable. They are still asking whether the model is smart enough, a question the market has already answered and priced to the floor. The operators who win the next cycle are asking what it costs to finish the work when the intelligence is free and the only scarce thing left is disciplined, unattended, economical execution. Capability is commoditizing. Cost is the new frontier. For the wider thesis, the manifest and the Joule Wars lay out where the joules, and the margins, actually go.

Route by Task, Not Vendor: The Open-Weight AI Stack

Michał Piszczek — Wed, 01 Jul 2026 09:00:00 +0000

Six months ago, "move your AI workloads to open Chinese models" was a thought experiment you floated in a strategy deck to sound forward-looking. Now it is a procurement story with real invoices attached. The migration is already happening at names you know, and it is not being driven by ideology. It is being driven by arithmetic.

Airbnb moved to Qwen, Alibaba's open-weight family. CEO Brian Chesky described it plainly: "very good, fast and cheap." It powers their support agent. Cursor built its Composer coding model on Moonshot's open weights, shipping as Kimi K2.5. Microsoft has been hosting and testing DeepSeek V4 inside Azure Foundry and Copilot. Shopify, Coinbase, Siemens, and Uber Eats have all been reported routing real production workloads to Qwen, GLM, Kimi, or DeepSeek.

None of them "switched to the best model." That framing misreads the entire decision. Each of them moved the right task to a cheaper open-weight model sitting within a few points of frontier. The distinction matters more than any benchmark leaderboard, because it inverts the question everyone has been asking.

The question was never "which model is best?"

For three years the industry has treated model selection as a single global decision. You pick the smartest model, you wire everything to it, you feel safe. That instinct is expensive and increasingly wrong. The real question is narrower and far more useful: which task actually needs the best model?

Look at what production traffic is actually made of. The overwhelming majority of it is the boring 80% — extraction, classification, summarization, routing, simple tool calls, reformatting, deduplication. This is plumbing. It does not require a model that can reason through a novel proof or design a distributed system. It requires a model that is competent, fast, and cheap.

Frontier models are priced for the hard 20% — the genuinely difficult reasoning, the long-horizon planning, the cases where an extra few points of quality translate into measurable business value. That is what you are paying a premium for. When you send the easy 80% through a frontier API, you are paying that premium on every request that never needed it.

Paying frontier prices for the easy 80% is one of the biggest sources of AI budget waste in production today.

Route by task, not by vendor

The architecture that follows is not exotic. It is a routing table. You classify the task, then you send it to the cheapest model that clears the quality bar for that task. In practice, a stack that holds up in production looks something like this:

Reasoning — GLM or Kimi, which now sit close enough to frontier that the gap rarely shows up in real workloads.
Code — Kimi Code or Qwen Coder for the bulk of generation and refactoring.
Agents and tool calls — GLM, which handles structured tool invocation reliably at a fraction of closed-API cost.
Bulk processing — MiMo, where you are grinding through volume and latency-per-dollar dominates.
Images and video — fine-tuned LTX plus Wan, tuned to your own domain.
Local workhorse — Qwen3.6-35B-A3B, the model that runs on your own hardware and quietly handles the daily grind.

Almost all of these are open-weight, self-hostable, close to frontier, and a fraction of the cost. This is the same principle that runs underneath capability commoditizing while cost becomes the frontier: when the models converge on quality, the differentiation moves to how efficiently you deploy them.

Savings are the visible win. Ownership is the real one.

The cost delta is what gets the CFO's attention, and it is real. But savings are not the point. The point is that you own the stack. When the core of your system runs on weights you hold, nobody can switch you off. Nobody can revoke your access on their timeline. Nobody can see your data, dictate your pricing, or quietly reshape your roadmap by changing theirs.

That is a business-continuity property, not a line item. It is the same argument that sits underneath the question of who owns your harness — the orchestration layer that actually knows how your company works. Open weights are the only ones nobody outside your walls can turn off.

Clearing up the "my data goes to China" reflex

There is a reflexive objection worth killing directly. "Chinese model equals my data goes to China" is simply wrong for open weights. Open weights run on your infrastructure. The weights may originate in a lab in Hangzhou or Beijing, but the weights are a static artifact — a file of numbers. When you self-host them, your data never leaves your servers. It goes to your GPUs, not theirs.

This is why the real boundary is not American versus Chinese. It is open-weight versus closed. A closed American API can log your prompts, change its terms, and go dark on a government's schedule. A set of open weights running in your own datacenter cannot do any of those things, regardless of which country trained it. The nationality of the training run is a distraction; the deployment topology is the actual security boundary.

When to still pay for closed

None of this means closed models are obsolete. They still lead on some cinematic, high-stakes workloads where the last few points of quality genuinely move the needle. The discipline is to pay for a closed model when the quality gap creates measurable business value — not by default, not out of habit, and not because it is the name everyone recognizes.

The rule is simple to state and harder to enforce: open-source first, self-host the core, pay for frontier only where it creates value you cannot get elsewhere. Enforcing it means building a routing layer, maintaining evals per task, and resisting the temptation to route everything to the smartest model because it is easier.

Key takeaways

The right question is not "which model is best?" but "which task actually needs the best?"
Most production traffic is the boring 80% — extraction, classification, routing — and frontier pricing on it is pure waste.
Route by task, not vendor: match each workload to the cheapest model that clears its quality bar.
Open weights self-hosted mean your data never leaves your servers, whatever the model's country of origin.
The real boundary is open-weight versus closed, not American versus Chinese.
Pay for closed models only where the quality gap creates business value you cannot get otherwise.

The race stopped being about the smartest model. It became about architecture that still works when today's smartest model is unavailable, unaffordable, or switched off. I write more about that shift across my essays on execution and AI infrastructure. Build the routing table now, while it is still a competitive edge rather than table stakes.

Who Owns Your Harness? The Layer Above the Model

Michał Piszczek — Tue, 30 Jun 2026 09:00:00 +0000

Lately I find myself less interested in which model wins and far more interested in who owns the layer above the model. The benchmark wars — GLM versus Claude versus GPT versus Qwen versus DeepSeek — are already yesterday's conversation. Models improve fast and get cheaper faster. Open-source is closing the gap ahead of every schedule people drew a year ago. So "which model should we use?" is not the question. It never was.

The question that actually determines whether your company survives a bad quarter in AI policy is this: can we replace the model tomorrow? If the honest answer is no, you have already made the most expensive architectural decision of the decade without noticing.

You think you're buying AI. You're wiring an operating system.

Most companies believe they are buying AI the way they buy a database or a cloud region — a component, swappable, bounded. What they are actually doing is wiring their entire execution layer around a single vendor. The prompts. The memory. The agents. The routing logic. The evals. And then every integration on top: Slack, Jira, GitHub, the internal tools, the accumulated company knowledge that no one wrote down anywhere else.

Bit by bit, the model stops being a model. It becomes the operating system of the business. Every workflow assumes its quirks. Every prompt is tuned to its behavior. Every engineer's mental model of "how our AI works" is really a mental model of one vendor's API. That is where lock-in begins — not in a contract clause, but in a thousand small couplings nobody tracked.

Lock-in stopped being a commercial inconvenience

For most of software history, lock-in was a negotiating problem. You paid a switching cost, you grumbled, you migrated over a quarter. Annoying, survivable. That era is over for AI. Lock-in became a business-continuity risk, and recent events proved it in the harshest way possible.

A single US export-control order took Anthropic's top models — Mythos 5 and Fable 5 — offline for two weeks. Not just for foreign users. To stay compliant, they were pulled for everyone worldwide, the United States included. Every company that had wired its product around those models lost its core capability overnight, through no decision of its own.

Days later, GPT-5.6 shipped only as a gated, US-only preview, after Washington reportedly asked OpenAI to hold the launch. Two data points, one lesson: the model under your product can go dark on a government's timeline, not yours.

Closed is closed the moment someone decides it's closed to you — and that someone may not be your vendor.

That last clause is the whole point. You can have a perfect relationship with your model provider, pay every invoice on time, and still lose access because a regulator three time zones away signed an order. Your vendor's goodwill is irrelevant when the constraint sits above the vendor. This is the shift I described in the model wars ending and the clearance wars beginning: capability now ships when it clears, not when it's ready.

The default has to flip: open-source first

The conclusion writes itself. The default posture must invert. Open-source first — not because open models win every benchmark, because they do not yet, but because open weights are the only ones nobody can switch off. A file of weights sitting on your own hardware does not care what any government decides next week. It is inert, and it is yours.

Keep closed models for the frontier edge, the genuinely hard workloads where the quality gap earns its premium. But the core you cannot afford to lose should sit on weights you actually hold. This is the same architecture I lay out in routing by task, not vendor: self-host the core, pay for frontier only where it creates value you can't get elsewhere.

Above the model sits the harness

And above every model — open or closed — sits the harness. This is the layer that actually matters, and it should be vendor-agnostic by design. The harness owns:

Memory — what your system remembers across sessions, users, and workflows.
Context — how the right information reaches the model at the right moment.
Routing — which task goes to which model, and the fallback when one goes dark.
Permissions — who and what is allowed to do which action.
Tools — the integrations that let the model act on your actual systems.
Evals — how you know quality held after you swapped a model underneath.
Orchestration — the logic that ties it all into something that works.

The harness is the part that actually knows how your company works. The model is a replaceable engine bolted into it. If your harness is well-built and vendor-agnostic, swapping models is a config change and a re-run of your evals. If it is not — if the harness and the vendor are the same thing — then a model going dark takes your whole business with it.

The highest-ROI call of the decade

Building your own harness is slower and costlier today. There is no way around that. It is more engineering, more discipline, more upfront investment than plugging into one vendor's SDK and shipping. That is exactly why most teams will not do it until they are forced to.

But it may be one of the highest-ROI calls of the decade. Models come and go. Governments reshuffle who gets access to what, and on what timeline. Your company brain — the accumulated knowledge, workflows, and judgment encoded in your execution layer — should depend on neither. It should sit in a harness you own, feeding whichever model happens to be best, cheapest, and available this month.

Key takeaways

The right question is not "which model do you use?" but "can you replace it tomorrow?"
Companies think they're buying AI; they're wiring their whole execution layer around one vendor.
Lock-in became a business-continuity risk — a single export order pulled Anthropic's top models worldwide for two weeks.
Closed is closed the moment someone decides it's closed to you, and that someone may not be your vendor.
Default to open-source for the core; open weights are the only ones nobody can switch off.
Own the harness — memory, context, routing, permissions, tools, evals, orchestration — and models become swappable engines.

We are going to stop asking "which model do you use?" and start asking who owns your harness. I write about that transition and the architecture it demands across my essays on AI infrastructure and execution. The teams that build the harness now will be the ones still running when the next model goes dark.

The Biggest Customer Becomes the Competitor

Michał Piszczek — Sat, 27 Jun 2026 09:00:00 +0000

OpenAI designed its own AI chip in nine months and aimed it straight at Nvidia, the supplier it cannot survive without. Codenamed Jalapeño, co-designed with Broadcom. The bill forced the move.

A custom chip usually takes two to three years from design to working silicon. OpenAI did it in nine months. The compression is not a footnote; it is the whole point. OpenAI used its own models to accelerate the design cycle, turning frontier inference back onto the problem of building the hardware that runs frontier inference. The snake ate part of its own tail, and the tail grew back faster.

Jalapeño is inference-only. It is tuned for the workloads OpenAI actually runs at scale: ChatGPT, Codex, the API, agents. It is not built for training. That narrowing is deliberate, and it is where the leverage lives. When you know your workload down to the token, you can throw away everything a general-purpose GPU carries to serve a thousand customers you are not. Early tests claim better performance-per-watt than today's best GPUs. At gigawatt scale, performance-per-watt is not a spec-sheet vanity metric. It is the P&L.

The pattern is older than OpenAI

This is not a surprise if you have watched infrastructure economics before. Google built TPUs because renting general-purpose accelerators for search and ads and, later, Gemini stopped making sense at their volume. Amazon built Trainium and Inferentia because AWS could not let the margin on every AI workload flow to a single supplier. Now OpenAI builds Jalapeño for exactly the same reason, and the reason is arithmetic.

The rule generalizes: the biggest customer always becomes the next competitor, because the bill forces it. When you are a small buyer, renting is obviously correct. The vendor amortizes billions in R&D across thousands of customers, and your slice is cheap. When you become the largest single consumer of a component, the math inverts. You are now underwriting a meaningful fraction of the vendor's margin, and that margin is a tax you pay on your own scale. At some volume, designing the thing yourself is cheaper than renting it, and every dollar of vendor margin you eliminate is a dollar that compounds.

Renting compute is a cost. Designing it is a moat. The difference is who owns the workload.

Old stack, new stack

The old stack was simple and stable. One vendor designs the silicon. Everyone else rents it. Nvidia sat at the top of that pyramid, and the pyramid was the entire industry. Access to Nvidia was the bottleneck, and allocation of Nvidia's chips was a story that moved markets. Whoever got the biggest allocation won the round.

The new stack rearranges the pyramid. The buyer designs the silicon, and the vendor becomes optional. Not eliminated, optional. That word does a lot of work. OpenAI will still buy Nvidia for training, for burst capacity, for the workloads where a general-purpose part still wins. But the strategic dependency loosens the moment a credible in-house alternative exists for the workload that dominates the bill. The bottleneck moves from access to Nvidia to ownership of the workload. Once you own the workload end to end, you get to decide how much of it to rent and how much to build.

Why inference, and why now

Inference is the right place to start vertical integration, and the timing is not an accident. Training is bursty, experimental, and moves with the research frontier; the workload changes shape every few months, which punishes custom silicon built around fixed assumptions. Inference at OpenAI's scale is the opposite. It is enormous, steady, and increasingly well understood. The company serves the same handful of model architectures to hundreds of millions of users, billions of times a day. That is exactly the profile that rewards a chip designed for one job and stripped of everything else.

The economics compound with agents. As I have argued in the unit of work is the agent-hour, output is going parallel: work is no longer bounded by human hours but by how many agents you can run at once. Every one of those agent-hours is inference. The inference bill is not a fixed cost you optimize once; it is the growth curve itself. Owning the silicon under that curve is owning the cost structure of your own future.

What OpenAI is really buying

Read past the chip and you can see what OpenAI is actually acquiring. It is not just cheaper tokens. It is control over its own cost curve, its own roadmap, and its own supply chain in a market where compute is the binding constraint. As I have written in OpenAI is GPU-constrained, not demand-constrained, the company's growth ceiling is set by silicon it does not manufacture. Jalapeño is the structural answer to that constraint. It is the first chip in a multi-generation roadmap, which tells you this was never a one-off experiment. It is a commitment to owning the bottom of the stack.

Here is the framework I use to decide when a big buyer should stop renting and start building:

Volume concentration. When one workload dominates your spend, the vendor's margin on that workload becomes your largest controllable cost. Concentration is the trigger.
Workload stability. Custom silicon rewards a job that will not change shape for years. Inference qualifies; frontier training does not, yet.
Design-cycle leverage. If you can compress the two-to-three-year chip cycle, as OpenAI did with its own models, the payback window shrinks and the bet gets far safer.
Strategic optionality. Even a good-enough in-house part changes your negotiating position with the incumbent vendor. The threat of building is worth money before the chip ships.
Roadmap commitment. One chip is a science project. A multi-generation roadmap is a business decision. Only the second one moves the moat.

What breaks next

If the largest AI buyers all vertically integrate, Nvidia does not disappear, but its position changes. It moves from the sole source of frontier compute toward a supplier of training and burst capacity, competing against the in-house parts of its biggest former customers. That is a different, thinner business than owning the entire pyramid. The interesting question is not whether Nvidia survives, it will, but what the market looks like when the five buyers who matter most each design the silicon for their own dominant workload.

The deeper shift is about where value accrues. For a decade, the story was that whoever controlled the scarce input, the chips, controlled the industry. Jalapeño is evidence that the scarce input is being routed around by the buyers with enough volume to justify the engineering. Value migrates from owning the general-purpose component to owning the specific workload well enough to build the component yourself. The bottleneck moved from access to ownership, and ownership is the more durable position.

Key takeaways

OpenAI designed Jalapeño, an inference-only chip, in nine months versus the usual two to three years, using its own models to compress the cycle.
The move follows a rule: the biggest customer becomes the next competitor, because concentrated volume turns vendor margin into your largest controllable cost.
Google (TPU) and Amazon (Trainium) ran this playbook first. OpenAI is the newest instance, not a novel one.
Inference is the right entry point for vertical integration: enormous, steady, and well understood, unlike frontier training.
The bottleneck moved from access to Nvidia to ownership of the workload. Renting compute is a cost; designing it is a moat.
A multi-generation roadmap, not a single chip, is what turns this from a science project into a structural change in the market.

The chip allocation era trained everyone to watch who got the most GPUs. That was the old bottleneck. The new one is quieter: which buyers understand their own workload well enough to stop renting and start building. For the wider map of how compute, clearance, and control connect, start with the manifest and the Joule Wars thesis. The supplier you cannot survive without is the one you eventually have to replace.