The AI thinks, the gate decides — how I made LLM code edits deterministic (and cut token usage 42 )

Sergio Corruchaga — Thu, 10 Sep 2026 21:18:28 +0000

title: "The AI thinks, the gate decides — how I made LLM code edits deterministic (and cut token usage 42×)"
published: true

tags: ai, opensource, typescript, llm

The AI thinks, the gate decides

D-Engine: a deterministic harness that matches coding agents' quality while burning 14–42× fewer tokens

Sergi Corruchaga · September 2026 · D-Engine v0.2.2 (MIT, open source)

1. The number that started it all

On September 10, 2026, I ran the same programming task three times, with the same model (DeepSeek V4.1-Flash), the same literal prompt, and the same repository:

"En utils.ts, añade una función formatDate que reciba un Date y devuelva DD/MM/YYYY"
(Add a formatDate function to utils.ts that takes a Date and returns DD/MM/YYYY)

All three runs produced functionally the same code. Here's what each one cost:

Tool	Architecture	Tokens consumed	Time
D-Engine (my harness)	Deterministic pipeline	2,552	~4 s
dsh — Minimal mode	Agent (single tool: shell)	34,600	1m 04s
dsh — effort Off	Full agent, no thinking	37,100	6 s
dsh — factory defaults	Full agent, thinking High	107,000	28 s

DeepSeek's official agent burned 42× more tokens than my tool to produce the same diff. And as you'll see in the controls section, that gap is explained neither by the model, nor by "thinking mode", nor by the agent's toolbox. It's explained by the architecture.

This article covers how I got here: what D-Engine is, how I ran the full benchmark (10 tasks, 5 contenders, 2 deliberate traps), what agents do better than my tool (quite a few things, and I'm going to disclose all of them), and why I believe the future of AI-assisted programming isn't a smarter agent — it's a stricter gate.

2. The problem: how an agent spends tokens

The dominant AI coding tools (OpenCode, Aider, dsh, Claude Code…) all follow the same pattern: the agentic loop. The model receives your request, decides to call a tool (read file, search, run shell), gets the result, decides another call, and so on until done.

The commonly overlooked detail: the model has no memory between calls. On every turn of the loop, the harness re-sends the full system prompt, all tool definitions, and the entire conversation trajectory so far. If the agent takes 20 steps, step 20 re-sends the previous 19. Cost grows quadratically with the agent's diligence — not with your task's difficulty.

Measured in my benchmark: the same task, in the same repo, with the same model, cost dsh between 32K and 214K tokens depending on how many loop turns it decided to take. A 6.6× variance the user neither controls nor can predict.

There's a second, subtler problem: state drift. The agent works from the "snapshot" of the code it has been reading during the session. If that snapshot goes stale — or the model misremembers it — it will edit something that doesn't exist. Or worse: it will believe it sees things that don't exist. In section 6 I describe how dsh reported a corrupted file that was perfectly healthy, complete with fabricated line-level evidence.

The third problem is atomicity: most agents write directly to your working tree. If the change breaks compilation, your main branch is already broken. Some will even auto-commit the disaster.

3. The idea: separate "thinking" from "touching"

D-Engine is built on a radical separation of responsibilities:

The cloud (the LLM) only thinks. It receives the minimum necessary context and responds with SEARCH/REPLACE blocks — patches anchored to existing code.
The local, deterministic runtime only touches. It applies those blocks in a photocopy of the repo (a shadow git worktree), compiles with tsc --noEmit, and only if the gate is green merges into the real repo.

A typical task consumes exactly 2 LLM calls:

Selector (optional, ~300 tokens): given a map of the repo, the model picks the minimal set of relevant files. The user confirms — the selection never applies without authorization.
Proposal (~2,000 tokens): the model receives only those files and generates the SEARCH/REPLACE blocks.

Everything else is local code: the LocalEditor applies each patch through a 4-strategy cascade (exact match → newline normalization → ignore trailing whitespace → fuzzy at 0.85 threshold), the compiler validates, and commitAndMerge stages only the files touched by the patch (with a git status --porcelain guard that aborts the merge if any foreign file appears, logging the offender's diff before destroying the photocopy).

There's also a Verify mode adding an optional second phase: send only the modified snippet for a semantic audit (the model answers OK / OK_WITH_OBSERVATIONS / FAIL). Measured cost: 330–984 tokens per task — 15–30% on top of the proposal. Nearly free semantic safety, on the programmer's demand.

The project's motto sums up the philosophy: the AI thinks, the gate decides. The model can propose whatever it wants; the only source of truth in the system is the compiler.

4. The benchmark

Methodology

Test repo: bench-repo, a TypeScript mini-shop (products, cart, pricing, utilities), frozen at the benchmark-base tag.
10 representative tasks: add a function, multi-file rename, validation guards, mass JSDoc documentation, cart line deduplication, an operation-ordering bug, an extraction refactor, two deliberate traps (an already-implemented task and an "optimization" of something already optimal), and a full feature (a coupon system with expiration).
Rules: same literal prompt for every contender, one attempt per task, git reset --hard benchmark-base + git clean -fd before every run, engine frozen during the benchmark.
Contenders: OpenCode, Aider, dsh (DeepSeek's official harness), and D-Engine in Fast and Verify modes.
Era declaration: the original round ran on V4-Flash non-thinking; the dsh round ran on 2026-09-10 on V4.1-Flash — the very day DeepSeek retired the previous model. The benchmark survived a mid-flight model extinction thanks to declaring model+effort+date per row, plus the control runs in section 6.

Methodological honesty disclosures

Before the results, two confessions. First: two tasks (T3 and T5) turned out to be defective in their first round — the repo already contained what they asked for; the voided rows are preserved in the record as evidence, and the base was fixed. Second: the literal prompts typed in the first round were not preserved (the git clean -fd cycles wiped Aider's histories, and my own record document stored summaries instead of the actual texts). The original specifications were recovered from the design conversation, prompts were frozen in the record on 2026-09-10, and every later execution uses them verbatim. T1 shares an attested literal prompt across all eras — it is the comparability anchor.

None of this is glamorous. That's exactly why it's in the article.

5. Results

Quality: a statistical tie

Contender	Points (max 50)	Incidents
OpenCode	49/50	4 on trap T9
D-Engine Fast	48/50	4 on T9
dsh (factory)	48/50	4 on T7 (unrequested API added), 4 on T9; 1 hallucination
Aider	47/50	4 on T7, 4 on T9; committed a main branch that didn't compile (T2)
D-Engine Verify	46/50	4 on T7, 4 on T9; false rejection on T6 (parsing bug, since fixed)

Nobody crushed anybody on quality. With the same model, the "textbook" solution converges — on three tasks, three different contenders produced byte-identical files. What differentiates the tools isn't the answer: it's the machinery around the model.

Tokens: here's the difference

Contender	Tokens per task (avg)	vs D-Engine Fast
D-Engine Fast	~2,100	1×
Aider	~2,100	~1×
D-Engine Verify	~2,600	~1.2×
OpenCode	~9,700	~4–5×
dsh	~93,000 (range 32K–214K)	~44×

(Aider deserves a fair note: it's by far the leanest agent, because it only passes the files you tell it to. Its problem wasn't cost — it was the gate. Keep reading.)

The full dsh round consumed ~931K tokens versus D-Engine Fast's ~21K for the same task set and equivalent results.

Time

Fast ~2.7s · Verify ~3.7s · Aider ~4.9s · OpenCode ~14.9s · dsh ~38s (wall clock; its own UI reports ~20s — the gap between both measures, 9 to 66 seconds per task, is startup and latency time the agent doesn't account for).

The front-page moment (T2)

Task T2 asked to rename a constant across two files. Aider warned it was missing context ("I don't have them in the chat. Let me know if you want me to review them")… and then auto-committed a main branch that didn't compile anyway (TS2305). Without a compile gate, AI can break your repo while knowing it's breaking it.

D-Engine, on the same task, rejected its own first attempt: the patch compiled in the photocopy but broke index.ts — the gate caught it, the merge never happened, and main stayed intact. Failing safe isn't a bug: it's the architecture.

Trap T9: the most revealing behavior

T9 asked to "optimize calculateTotal using Array.reduce"… when the function already used reduce. The perfect answer was "nothing to do here".

Nobody gave the perfect answer. Every first-round contender made cosmetic changes (4/5). But dsh did something more interesting and more unsettling at once:

It admitted the trap ("it already used reduce") — only OpenCode had done that.
It found a real bug nobody had asked it to look for: round2(1.005) returned 1.00 instead of 1.01 due to binary floating-point noise. A legitimate, valuable find.
…and then it fixed it unilaterally, modifying utils.ts (outside the target) and changing the rounding behavior of the entire system, when the prompt said to "keep rounding correct".
And it finished by reporting that src/products.ts was corrupted ("line 16 reads ndProduct… it breaks the whole project's compilation"). Manual verification: the file was intact. A hallucination with fabricated line-level evidence, in the same message where it claimed tsc passed cleanly.

The behavior was safe (it asked permission before touching the "corrupted" file). But had I answered "yes, fix it", the agent would have edited a healthy file chasing a ghost. D-Engine structurally cannot have this class of hallucination: it doesn't opine on repo state — truth comes from tsc, not from the model.

The cost of all that unleashed diligence: 214K tokens on a task whose correct answer was "nothing to do". One hundred times D-Engine.

6. The controls: killing objections before they're raised

I anticipate three objections to the token gap. All three have measured answers.

"It's the model" → No. The dsh round ran on V4.1-Flash; I ran D-Engine v0.2.2 on the same new model (adapted the very day of the API migration): 2,552 tokens. Gap intact.

"It's thinking mode" → Partially. With effort set to Off (an exact replica of the first round's non-thinking configuration), dsh dropped from 107K to 37.1K. Thinking amplifies the gap ~2.9× (and quadruples loop turns: 12 vs 3 tool calls — a model that "thinks" also wanders more). But the remaining 14.5× is still there with reasoning off.

"It's the tool arsenal" → No. In Minimal mode (a single tool: a persistent shell), dsh consumed 34.6K — practically identical to the full agent without thinking (37.1K). With a primitive shell the agent needed more turns (11), not fewer: search with Get-ChildItem, read with Get-Content, edit with Add-Content and hand-typed \r\n escapes, re-read to verify, compile… The cost isn't in the tool schemas. It's in the loop: every turn re-sends the full trajectory. Shrinking the arsenal doesn't shrink tokens; shrinking the loop does.

Final gap decomposition on T1 (same model, same prompt, same diff):

D-Engine (pipeline)       2,552 tok   1×     ← no loop
dsh Minimal              34,600 tok   13.6×  ← the arsenal doesn't matter
dsh Off                  37,100 tok   14.5×  ← pure architectural overhead
dsh Factory (thinking)  107,000 tok   42×    ← thinking amplifies ~2.9×

7. What agents do better (and it would be dishonest to hide it)

This article is not "agents are bad". dsh produced, by far, the most diligent work in the benchmark:

It self-verified: compiling with tsc inside its own loop, and on two tasks it went as far as executing the program to confirm the output was identical before and after the change. No other contender did that.
On T10 it wrote a 23-case edge-case suite for the coupon system (same-day expiration, NaN dates, JavaScript's February 31st…), ran it — 23/23 — and deleted it afterwards. Brilliant.
It can create new files; D-Engine can't yet (documented limitation, on the roadmap).
It detected dead code, duplications, and a validation hole in applyDiscount, and reported them without touching anything, asking permission. Exemplary scope discipline — when it chooses to have it.
It found a real rounding bug I didn't know existed.

The honest conclusion isn't that agents are unnecessary. It's that today you pay for their diligence blind: you don't know if your task will cost 32K or 214K tokens, whether the agent will respect your scope or redecorate half your repo, or whether its report about your code's state is true or a plausible hallucination. D-Engine proposes the inverse split: the agent provides judgment; the machine provides truth and a fixed bill.

An important note about money: at DeepSeek's prices (with 96% cache-hit rates measured in some sessions), 100K tokens cost cents. Direct cost is not the argument. The argument is latency (2.7s vs 38s per task), predictability (bounded bill vs 6.6× variance), context degradation in long sessions, and what this gap means when the model costs dollars per million instead of cents — or when the loop runs unattended in CI, with no one there to say "no" in time.

8. Limitations

I declare them before the first hostile comment does:

Small repo (a 5–6 file mini-shop). The absolute gap would grow with repo size in both systems, but the structure of the gap (loop vs pipeline) is size-independent.
A single model family (DeepSeek). Nothing prevents rerunning the benchmark with other providers; the method is portable.
First-round literal prompts were not preserved (incident disclosed in section 4; prompts frozen since 2026-09-10).
D-Engine can't create new files yet, and its file selector (P9) is non-deterministic — though the architecture absorbs the variance safely.
Fuzzy matching (0.85 threshold) is the weakest link: the only patch that broke syntax in the entire benchmark came through that path. On the roadmap: retry with compiler feedback, and mandatory Verify when a patch only applies via fuzzy.
I don't measure agent judgment quality on open-ended tasks ("improve this design"), where the exploratory loop has real advantages. D-Engine is built for bounded, specifiable changes — which are most of daily work.

9. What's next

v0.3: new-file creation, retry with tsc feedback, Verify-if-fuzzy, and time+tokens printed in the final summary (measuring time by hand with a stopwatch was the least glamorous part of this benchmark).
Published: the code is MIT and lives on GitHub with the complete benchmark (frozen prompts, voided rows included) so anyone can reproduce or rebut it: https://github.com/corruchaga/D-Engine
dsh's PTC mode remains as future work: DeepSeek is already trying to collapse the loop into a single TypeScript program. If it works, it's proof the industry is converging toward this idea on its own.

10. Conclusion

Coding agents are impressive. They're also token-burning machines with unpredictable variance, no native compile gate, and a… creative relationship with your repository's actual state.

This benchmark shows that for daily work — bounded, specifiable, verifiable changes — a deterministic pipeline produces the same quality (48/50, tied with the best agent) at a fraction of the cost (14–42× fewer tokens depending on configuration), a fraction of the time (2.7s vs 38s), with a predictable bill and zero broken commits.

We don't need a smarter model. We need a gate.

The AI thinks. The gate decides.

Sergi Corruchaga is a junior developer (DAM graduate, currently studying DAW). D-Engine is his first open-source project. The complete benchmark — every table, incident, and voided row — is available in the repository.

DEV Community: Sergio Corruchaga