A week ago I published a fairly absurd number:
612.9 million tokens for $28.35.
That was DeepSeek Harness (DSH) + DeepSeek V4 Pro working on LovelaceSharp, a real C# numerical computing project, over a visible four-day development window.
The repository received 45 commits across 193 files.
The point of that article was not that 612 million tokens are somehow inherently productive.
It was that the economics of a persistent coding agent look very different from the economics suggested by raw token counts.
DSH had a 98.94% input cache-hit rate.
It was processing enormous amounts of context, but very little of that context was novel expensive input.
Then I kept running the experiment.
And DeepSeek released V4.1 Flash.
The numbers got considerably more stupid.
From September 10 through September 12, my actual provider export for deepseek-flash shows:
| Metric | V4.1 Flash |
|---|---|
| API requests | 12,253 |
| Cache-hit input | 2,123,399,808 |
| Cache-miss input | 14,976,005 |
| Output | 16,114,788 |
| Total tokens | 2,154,490,601 |
| Input cache-hit rate | 99.30% |
| API cost | $19.22 |
| Blended cost / 1M processed tokens | $0.00892 |
Yes.
2.15 billion tokens. Nineteen dollars.
And before anyone starts typing "tokens are not productivity" again:
I agree.
So let's look at what happened to the software.
Where the Previous Article Stopped
The previous benchmark ended around September 5.
At that point LovelaceSharp already had arbitrary-precision arithmetic, optimized big-integer algorithms, typed N-dimensional arrays, DSP, a scripting language, a browser IDE, Native AOT support, benchmarks and a Lean reference formalization.
The project was already well beyond a toy.
But most of the work was still fundamentally numerical infrastructure.
What happened afterward was a much larger expansion of scope.
From the last September 5 commit to the current HEAD, the repository moved forward by:
166 commits.
During that period LovelaceSharp acquired an actual symbolic mathematics stack.
Not a SymPy wrapper.
A native one.
The work included:
- immutable hash-consed symbolic expression DAGs;
- canonical addition, multiplication and powers;
- exact rational arithmetic;
- assumptions with three-valued logic;
- conditional rewriting with explicit side conditions;
- symbolic differentiation;
- limits and series, including Laurent behavior;
- symbolic integration;
- polynomial arithmetic;
- factoring and square-free decomposition;
- real-root isolation;
- equation solving;
- parametric solution families;
- Gröbner bases;
- polynomial-system solving;
- symbolic matrices;
- conditional matrix inversion and linear solving;
- CSE and Horner optimization;
- complex symbolic/numeric evaluation;
- a typed computational IR called MathIR;
- scalar and vectorized compiled evaluation;
- structured result types instead of prose results;
- symbolic inspection in Studio;
- an agent-facing structured execution protocol;
- differential comparison against SymPy;
- and a significantly larger correctness infrastructure around all of it.
This is the part I think matters when discussing agent economics.
The $19 did not buy "2.1 billion tokens."
Tokens are an implementation detail.
It bought access to a software production process that could keep expanding the product while simultaneously expanding the machinery used to verify the product.
That distinction is important.
Features Without Tests Are a Terrible Agent Benchmark
One of the easiest ways to make an AI coding demo look impressive is to count features.
LLMs are very good at producing code.
If I ask an agent to maximize visible feature count, it will happily oblige.
That does not mean I have created valuable software.
So I care much more about the relationship between feature growth and validation growth.
When the first symbolic kernel landed on September 8, it already came with 100 Symbolics tests plus executable documentation examples.
Then the API expanded.
Relations, solution families, polynomials, matrices, compilation and Gröbner bases landed.
The Symbolics suite moved past 200 tests.
Then came structured APIs, assumption hardening, semantic fixes, solver work, differential oracles, property tests, metamorphic tests, protocol tests and adversarial cases.
By September 10, one of the convergence commits reported:
1,971 passing tests across 14 suites.
The project kept going.
The most recent full local validation recorded:
5,655 passed.
0 failed.
0 skipped.
This is a much more interesting development curve than simply saying "the agent added a CAS."
The product surface grew, but the test surface grew with it.
And the tests are not all repetitions of ordinary unit assertions either.
The repository now contains things such as:
property tests
differential tests
metamorphic tests
golden protocol fixtures
cross-surface consistency tests
precision-boundary tests
Native AOT smoke tests
SymPy oracle comparisons
pre-fix regression controls
determinism tests
concurrency/isolation tests
budget/cancellation tests
That changes how I think about the cost of software development.
Testing Used to Compete Economically With Features
There is an uncomfortable reality in normal software development.
Every hour spent writing tests is an hour someone is not spending implementing the next feature.
Every alternative architecture you prototype and discard costs engineering time.
Every benchmark takes time to construct.
Every edge-case corpus has to be written.
Every independent implementation used as an oracle costs more time.
Every documentation example that is kept executable costs more time.
Every time you say:
let's test another 50 weird cases before shipping this
you are spending part of a finite engineering budget.
That doesn't mean those things aren't worth doing.
It means they have a real opportunity cost.
Agentic engineering changes this equation.
Suppose a feature takes an agent another 30 million tokens to implement, test, benchmark, revise and document.
At the effective blended price I observed with V4.1 Flash, 30 million processed tokens would cost roughly:
$0.27.
Obviously token cost is not the only cost.
There is compute outside the model.
There is CI.
There is human review.
There is specification.
There is the cost of accepting a bad implementation.
But the inference component has become almost comically small.
So the economically rational behavior changes.
Instead of asking:
Do we have time to build a differential oracle for this?
the answer increasingly becomes:
Why wouldn't we?
That is a much bigger change than cheaper autocomplete.
V4.1 Flash Changed the Curve Again
The previous article already showed strange economics with V4 Pro.
After publishing it, I have a useful second window.
From September 5 through September 9, the new export contains this V4 Pro usage:
| Metric | V4 Pro |
|---|---|
| Requests | 2,026 |
| Cache-hit input | 588,642,048 |
| Cache-miss input | 3,595,338 |
| Output | 2,847,970 |
| Total tokens | 595,085,356 |
| Input cache-hit rate | 99.39% |
| Cost | $20.96 |
| Blended cost / 1M tokens | $0.0352 |
Then V4.1 Flash:
| Metric | V4 Pro | V4.1 Flash |
|---|---|---|
| Total tokens | 595.1M | 2.154B |
| Requests | 2,026 | 12,253 |
| Cache-hit rate | 99.39% | 99.30% |
| Cost | $20.96 | $19.22 |
| Cost / 1M processed tokens | $0.0352 | $0.00892 |
The workloads are not identical, so this is not a controlled intelligence benchmark.
I cannot conclude:
V4.1 Flash is 3.62× more productive than V4 Pro.
That would be nonsense.
What I can measure is the economic side.
Flash processed 3.62× more tokens for 8.3% less total money.
Its effective blended cost per processed token in this workload was approximately:
3.95× lower.
And the cache behavior barely changed.
V4 Pro: 99.39%.
V4.1 Flash: 99.30%.
So this isn't a story about DSH suddenly discovering caching.
It was already doing that.
The underlying inference economics changed.
One Billion Tokens in a Day Is Now Boringly Affordable
September 11 is the funniest row in the export.
That single day:
| Metric | September 11 |
|---|---|
| Requests | 6,463 |
| Cache-hit input | 1,002,941,952 |
| Cache-miss input | 7,388,973 |
| Output | 8,712,583 |
| Total tokens | 1,019,043,508 |
| Cost | $9.81 |
One billion tokens.
One day.
Less than ten dollars.
I don't think "billions of tokens" should become a goal.
If anything, that would be another terrible optimization target.
But it tells us something about capacity.
During the original benchmark, 612.9 million tokens over four days felt large enough to deserve an article.
V4.1 Flash processed around 1.66× that entire benchmark's token volume in one day.
For roughly one third of its inference cost.
This means the number of agent iterations you can economically throw at a software problem has changed dramatically.
This Is Where Software Engineering Economics Get Weird
Traditional software economics are dominated by expensive human time.
Imagine a feature requiring:
architecture
implementation
unit tests
integration tests
benchmarking
documentation
review
bug fixing
Those are competing allocations of engineering hours.
Now consider the same work under an agentic harness.
An implementation agent can write the feature.
Another execution can build the tests.
Another can benchmark competing implementations.
Another can compare results against an external oracle.
Another can examine the public API.
Another can search for boundary cases.
The original agent can then revise the implementation.
The harness retains the work that happened before.
And because the context is highly reusable, most of those repeated passes are being served as cache hits.
In my V4.1 Flash window, the API processed more than:
2.12 billion cached input tokens.
Against only:
14.98 million cache-miss input tokens.
That is the economic mechanism.
The agent doesn't have to repeatedly forget the repository in order to be cheap.
It can accumulate architecture, requirements, test results, previous failures and implementation decisions as reusable context.
Then revisit that context extremely cheaply.
This is almost the inverse of how we traditionally think about context efficiency.
The Marginal Cost of Engineering Work Is Collapsing Unevenly
This doesn't mean software development becomes free.
It means different parts of software development are becoming cheap at radically different rates.
Code generation is becoming cheap.
Test generation is becoming cheap.
Repository exploration is becoming cheap.
Documentation is becoming cheap.
Trying a second implementation is becoming cheap.
Throwing away a failed implementation is becoming cheap.
Running another agent over the same repository looking for problems is becoming cheap.
The human parts aren't falling at the same rate.
Understanding what the product should do is still expensive.
Making architectural decisions is still valuable.
Recognizing when a requirement itself is wrong still matters.
Reviewing the highest-risk assumptions still matters.
Taking responsibility for what ships still matters.
But this changes what human engineering time should be spent on.
I don't need to spend my time manually typing every test case.
I need to decide which properties actually matter.
I don't need to manually implement three algorithms just to compare them.
I need to understand whether the benchmark used to select between them means anything.
I don't need to manually write 100 edge cases.
I need to notice the class of edge case the agent forgot to generate.
That is a different allocation of engineering labor.
The Feature/Test Ratio Can Move in the Right Direction
There is a common fear around AI-generated code that I think is completely justified:
If generating code becomes 10× easier, won't we just create technical garbage 10× faster?
Yes.
If the only thing we make cheaper is feature generation.
But inference does not know that production code is special.
The same cheap intelligence that generates a feature can generate:
- tests for it;
- property checks;
- fixtures;
- benchmark harnesses;
- differential comparisons;
- migration tools;
- documentation;
- fuzz inputs;
- failure reproductions.
This is where I think agentic SWE becomes much more interesting than "developer autocomplete."
The economically optimal feature can start carrying much more validation around it.
The first LovelaceSharp symbolic implementation didn't just add symbolic differentiation.
It added tests.
When Gröbner bases arrived, they came with invariant tests.
When MathIR gained vectorized execution, scalar-vs-batch equivalence was tested.
When arbitrary precision behavior expanded, results were compared against mpmath and SymPy.
When the runner became a machine interface rather than only a human CLI, its structured envelopes acquired their own contract tests.
When correctness bugs were found, regression cases were added.
The point is not that 5,655 tests prove LovelaceSharp is correct.
They don't.
The point is that test production is scaling with feature production instead of being economically squeezed out by it.
That is exactly what I want from agentic development.
The $40 Continuation Window
There is another way to look at the whole period after my previous article.
From September 5 through September 12, the provider export contains:
2.759 billion processed tokens.
14,304 requests.
$40.44 total cost.
During the corresponding repository continuation, LovelaceSharp moved 166 commits beyond the September 5 boundary.
I don't think cost-per-commit is a serious productivity metric.
Commits are arbitrary.
An agent can trivially game that number by creating smaller commits.
But just to establish the scale, if I pessimistically charge every dollar in that provider window to LovelaceSharp, including usage that may not belong to it, that comes to:
about $0.24 of inference per commit.
Again:
That does not mean a software commit is "worth 24 cents."
It means inference has become such a small component of the economic equation that counting dollars per patch starts becoming almost silly.
The expensive question is whether the patches form useful software.
This Also Changes the Value of Failed Work
One of my favorite observations from the original benchmark was Newton division.
DSH implemented it because asymptotically it should beat Knuth division.
Then it benchmarked the result.
And discovered that the crossover was so far out that Knuth should remain the normal production path.
From a traditional project-management perspective, you could describe that implementation as partially wasted effort.
An engineer spent time implementing something primarily to discover that it should barely be used.
Under agentic economics, that interpretation makes less sense.
The failed hypothesis produced knowledge.
And the inference cost of getting that knowledge was tiny.
This effect gets stronger as inference gets cheaper.
You can afford to explore more branches.
You can implement competing designs.
You can benchmark them.
You can discard two and keep one.
You can ask an agent to prove its own optimization is useful before accepting the complexity.
That is closer to search than traditional implementation.
And search becomes much more attractive when each branch costs cents instead of engineer-days.
So What Is the Unit of Productivity?
Definitely not tokens.
Probably not commits.
Definitely not lines of code.
I suggested this in the previous article:
validated engineering work
──────────────────────────
inference cost
I still like that.
But this second run makes me think the interesting economic unit may eventually become something closer to:
useful product capability × confidence
──────────────────────────────────────
total engineering cost
Because an agent that implements ten features with weak validation may be economically worse than an agent that implements six features and surrounds them with enough tests, benchmarks and executable contracts that I can safely continue building on them.
Cheap generation increases the numerator.
Cheap validation increases the confidence multiplier.
Persistent cached context reduces the inference component of the denominator.
That combination is much more important than token efficiency in isolation.
I Am Not Claiming V4.1 Flash Is 4× Better
To be clear, this is still a case study.
I changed the model.
But the project also changed.
The tasks changed.
The harness evolved.
The later workload contains much more symbolic mathematics, validation and hardening.
The request distribution is different.
So the 3.95× reduction in blended token cost is an economic observation, not a model-quality benchmark.
DeepSeek claims V4.1 Flash exceeds V4 Pro on its agentic benchmarks, and the architecture is especially interesting for this workload: a 552B MoE with asymmetric activation, using only 8B active parameters on input and 16B on output. DeepSeek also says its KV-cache requirements are substantially smaller than the previous generation.
That architecture is almost comically aligned with DSH.
DSH is extremely input-heavy.
The Flash window contained:
2.138 billion input tokens.
Only:
16.1 million output tokens.
The vast majority of the workload is the model repeatedly reading accumulated engineering context.
Making input and cache reuse radically cheaper directly attacks the dominant term in the workload.
That matters much more to me than shaving a few tokens from a prompt.
What I Think Agentic SWE Actually Changes
The easy prediction about AI coding is:
software will be cheaper because programmers will write code faster.
I think that undersells the change.
The more interesting possibility is:
we will economically justify doing engineering work that was previously skipped.
More tests.
More alternative implementations.
More benchmarks.
More compatibility checks.
More executable documentation.
More migration tooling.
More differential validation.
More experiments that are expected to be thrown away.
More maintenance of internal developer tools that were previously "not worth the sprint."
More exploration before committing to an architecture.
Some teams will absolutely use AI to produce the same amount of validation and ten times more code.
That will be terrible.
But that isn't the only equilibrium available.
If the marginal cost of both features and verification collapses, then the feature/test economics can improve rather than deteriorate.
That is the part I am interested in testing.
Conclusion
The first article was:
612.9 million tokens.
45 commits.
$28.35.
It convinced me that cache locality could make persistent agent workloads much cheaper than raw token counts suggest.
The continuation is:
2.759 billion additional processed tokens for $40.44.
166 commits beyond the September 5 repository boundary.
And, inside that period, the first three complete V4.1 Flash days alone were:
2.154 billion tokens.
12,253 requests.
99.30% input cache hit.
$19.22.
What came out the other end was not 2.1 billion tokens worth of chat.
It was a substantial expansion of a real software system: symbolic mathematics, exact rationals, Gröbner bases, system solving, MathIR compilation, structured machine-facing APIs, more numerical capabilities and a test/validation surface that eventually reached 5,655 passing tests.
That is the benchmark artifact.
The code is public.
The Git history is public.
The usage export is the bill.
And I think the economics are starting to point somewhere important.
For most of software history, implementation effort forced us to ration engineering ideas.
We rationed features.
We rationed tests.
We rationed prototypes.
We rationed benchmarks.
We rationed cleanup work.
We rationed experiments.
Agentic engineering does not remove the need for good engineers.
But it may remove a large part of the economic pressure that forced engineers to choose between many of those activities in the first place.
At $0.0089 per million processed tokens, I don't need my engineering agent to be token-efficient in the traditional sense.
I need it to turn cheap inference into useful, tested software.
That is a much more interesting optimization problem.
Top comments (7)
The 99.30% cache-hit rate is the real story here, not the token count. Once an agent works one repo for days, nearly all of its input is repeated context, and pricing that looks absurd per million tokens becomes boringly cheap. I think this quietly changes how you design the harness too: you can afford to be wasteful with reads as long as writes stay deliberate, which flips the old instinct to minimize context. Curious what the hit rate does if you add a second project to the same setup - whether the economics survive context switching.
Yeah, I think that's probably the next experiment worth running.
I don't have a clean two-project A/B in this export, so I don't want to claim the economics survive context switching yet.
My suspicion is that it depends heavily on what we mean by "switching."
If DSH keeps project A and project B as separate stable context prefixes, then switching between them shouldn't necessarily destroy the economics. Each project pays a cold/warm-up cost, but once its prefix exists you should be able to return to it and resume getting cheap reads.
If instead the harness keeps mutating one giant prefix with unrelated repositories, I'd expect the hit rate to degrade pretty badly. At that point I'd consider it a harness design problem rather than an unavoidable property of agents.
One encouraging signal from this run is that the hit rate barely moved despite a lot of repository evolution:
V4 Pro: 99.39%
V4.1 Flash: 99.30%
So change within a project doesn't appear to be particularly hostile to cache locality. But an independent second repository is a different test.
I'd like to measure that explicitly: cold-start cost per project, hit-rate recovery after switching, and steady-state hit rate while alternating A → B → A → B.
And I think your "wasteful reads, deliberate writes" framing is exactly right. If cached reads become nearly free, minimizing context may actually be the wrong optimization target. The scarce operation becomes changing the state of the software correctly, not rereading enough context to reason about it.
The number I'd pull out of your export is the cache-hit rate, not the cost per million: 588M of 595M input tokens hitting cache is what makes $19.22 for 2.1B look wrong. Do you have the miss-only figure? That's the number deciding whether the economics survive a workload with cold starts and no repetition, and it's the one a skeptic will ask for first.
The feature/test coupling is the more defensible claim either way — an agent that lands a Symbolics test with each new API surface turns "more tokens" into "more confidence", and the Knuth-vs-asymptotic division case is exactly the situation where a barely-used implementation was worth cents. One thing I'd be careful about generalising: a numerical computing library has a dense, checkable oracle sitting in SymPy, so verification is cheap there. Most product code doesn't, and in that case the expensive part is the definition of correct, not the inference. Does your differential-oracle approach reach the parts of the project where "correct" is a product decision rather than an identity?
Yep — the miss-only number is 14,976,005 input tokens.
Total input was 2,138,375,813 tokens, so that's about 0.70% miss / 99.30% hit. And I agree that's the sensitivity variable here.
A useful counterfactual: if I take exactly the same input volume and make all of it cold, at V4.1 Flash's current rates the input alone would be about $321 off-peak or $642 peak. Keeping the observed 16.1M output tokens puts the whole workload at roughly $330–$661, depending on the pricing window.
So the $19.22 result absolutely does not generalize to a workload with cold starts and no repetition. My claim is narrower: persistent agentic SWE appears to have enormous context locality, and DSH is exploiting that locality extremely well. Without the 99%+ hit rate, the economics are very different.
And I agree with the oracle caveat too. LovelaceSharp is unusually friendly to differential validation because mathematics gives me things like SymPy/mpmath plus identities and invariants.
I don't think differential testing solves product correctness.
Once "correct" becomes a product decision, the oracle has to become something more like an executable contract: accepted examples, invariants, E2E scenarios, golden behavior, compatibility constraints, or—in migration work—the existing system's behavior where that behavior is intentionally being preserved.
The agent can make producing and exercising those contracts extremely cheap. It cannot tell me what the contract should be.
That's actually one of the boundaries I'm increasingly interested in: inference cost is collapsing much faster than the cost of defining correctness.
This resonates hard — the economics shift you're describing is exactly what we
saw when we moved from "one frontier model for everything" to cascades. Our
setup: cheapest-first routing with an independent critic before escalation.
Result on 10k requests: cascade + critic cost $2.6 vs $10.0 for a single
frontier model, but with 93-94% quality vs 92%. That's not a compromise, that's
a Pareto improvement — exactly your "validated engineering work / inference
cost" framing.
Your 99.3% cache-hit rate is the hidden multiplier most teams miss. We
discovered this when comparing local Ollama (zero marginal cost, but weaker on
complex reasoning) vs cloud APIs. The winning architecture turned out to be
hybrid: local models for 70% of requests (boilerplate, simple completions),
cloud for the 30% that need frontier reasoning, with a lightweight router
deciding the tier. Router cost: ~$0.001 per decision. Net effect: 80% cost
reduction with ~95% quality retention.
The interesting question is: what's your threshold for when cache locality
stops helping? We found that for exploratory/creative work (low repetition),
cache-hit drops to ~40% and the economics invert — local models win again
because they have zero marginal cost regardless of cache performance.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.