AI disclosure: this article is AI-generated — written by my agent toolchain under my direction. I set the questions, ran the hands-on checks, and verified every claim; the judgments are mine.
Wavestone AI Lab has published an 83-page source-code study of eleven production coding agents: Claude Code, Codex CLI, Gemini CLI, Mistral Vibe, OpenHands, Aider, Mini-SWE-Agent, Hermes, Pi, OpenCode, and OpenClaw — plus one meta-harness control group (Omnigent). Around 4 million lines of Python, TypeScript and Rust, studied subsystem by subsystem with a 90-day longitudinal diff on top. (Paper: https://arxiv.org/abs/2609.00006 — volume figures below are the paper's own; I didn't re-run its statistics.)
One line in it stayed with me: "inventory claims (tool counts, feature cells, version pins) decay in weeks, while structural claims (loop taxonomy, subsystem anatomy, the absences) have so far proven durable."
So I went and checked an inventory claim myself. By the very binary version the paper names, it was already gone.
This piece is not a summary. Three things most write-ups skip: I re-verified two of the paper's claims hands-on, I asked why a consulting firm would invest in publishing this, and I wrote down usage guardrails for anyone who cites it.
What the paper found
The double absence. None of the eleven systems uses a general-purpose agentic framework, and none embeds code retrieval into its context loop as a core mechanism. The systems that actually ship took the boring path: their own schedulers, their own context management. (Paper's finding; not re-verified by me.)
Skills are winning. Nine of eleven have adopted a skills/plugin system — ahead of MCP adoption at eight of eleven. One ships a skill authored by the agent itself. Supply-chain governance is becoming a thing. (Paper's figures.)
Convergence by imitation. Across the 90-day diff, architectures are converging: later movers copy the pattern leaders. (Paper's finding.)
The full paper also carries a seven-subsystem anatomy, 29 design patterns, and 13 cross-cutting observations. High density; not re-summarized here.
Two things I checked myself
When I read a paper like this, I try to verify at least a couple of claims on my own hardware. Both checks turned out interesting.
Check 1 — the constant is gone, at both ends. The paper cites two Claude Code constants from its March 2026 source snapshot: AUTOCOMPACT_BUFFER_TOKENS = 13,000 and POST_COMPACT_TOKEN_BUDGET = 50,000. Its own tables carry a warning: the shipping binary ("2.1.206, July 2026") may differ.
So I checked both ends. I pulled the 2.1.206 binary from npm and word-boundary grepped it; then I ran the same check on my current install (2.1.296, October 2026):
grep -aow 'AUTOCOMPACT_BUFFER_TOKENS' <2.1.206 binary> # 0 hits
grep -aow 'AUTOCOMPACT_BUFFER_TOKENS' <2.1.296 binary> # 0 hits
grep -aow 'POST_COMPACT_TOKEN_BUDGET' <each binary> # 0 hits
Zero across the board. What is present in both binaries is a renamed family: autoCompactWindow, compact_boundary, PreCompact/PostCompact hooks. The old names are what decayed; the compaction mechanics they describe live on under the new ones. And here is the part that stayed with me: the paper had already written the lesson down. My check was that sentence, replayed on-site.
Check 2 — I've used one of the eleven systems. Hermes, by NousResearch, is among the systems dissected. I ran it for a long stretch before retirement, and my old local install (v0.21.3) is still around. The two Hermes-specific mechanisms the paper cites — lineage compaction (parent_session_id session rotation) and a verification guard (verify-on-stop) — both grep out of my old install.
Having used one specimen first-hand also let me calibrate the rest: the Hermes chapter matched details I knew only from operating the system — a reassuring signal for the chapters I can't personally check.
The question I didn't see asked: why would a consultancy publish this?
Instead of guessing at motives, I look at shelves. Five places, all public:
- Products — they maintain a public agentic-coding skills repo: github.com/wavestone-sa/agentic-spec-skills
- Clients — their site pushes a Sanofi × Claude Code case study
- Hiring — they're recruiting Agentic engineers/consultants in multiple countries
- Financials — a reported 2% quarterly revenue decline, adjusted annual targets, and an announced acquisition of an AI/data team (public reporting: https://www.actusnews.com/fr/wavestone/cp/2026/07/29/q1-2026-27-revenue-down-2-per-cent-annual-targets-adjusted-ai-builders-targeted-acquisition-of-a-french-ai-and-data-player)
- Partnerships — a conversational-AI platform partnership; a Microsoft Copilot specialization
Put together — and this is my read — it's a classic consulting acquisition funnel: research builds credit → a "maturity assessment / roadmap" becomes the standardized product (the margin layer) → custom programs at the apex. The paper's job is to shorten the time it takes to be trusted. Deals still close on existing pain and relationships.
None of that is a dig — dissecting 4 million lines across eleven systems is real work, and I'm reading the shelf around a solid product, not attacking it. It's the standard posture of a consultancy in the AI era — and it happens to answer a question I care about: who actually pays for evaluations, and why.
My take on "will AI kill consulting" is the same shape: it changes form rather than dying. The pyramid's base (research, slideware, first drafts — the three cheapest AI tasks) thins out. What stays is the signature and the accountability of judgment: models can't be held accountable, can't be fired, and don't sit on boards. As information floods, "a specific person stands behind this" gets more valuable, not less. Watch two measurable numbers: junior headcount, and billing model (day-rate moving toward outcome).
Caveats — read before citing the paper
- Benchmark scores: not independently verified. Whatever eval numbers appear in the paper or the systems it covers originate with the system vendors themselves; I cannot re-run them on my machine. Treat as unverified.
- Scale and method figures are the paper's own. "~4 million lines", "90-day diff", "29 patterns", "13 observations" — I did not re-run its methodology.
- I could find no public code or data artifacts. For work whose selling point is a source-code study, that's a reproducibility gap (my judgment). The structural findings don't depend on line-by-line reproduction; the detailed counts do.
- Snapshot discipline. Everything I cite carries a date (this piece's snapshot: 2026-10-13). A paper whose cited constant got renamed within months is the best argument I know for this rule.
- Scope of my verification: exactly two of the paper's claims. Everything else here is cited as "the paper says" or flagged as my judgment.
About me
LUM — Independent evaluator. No consulting background, and I won't pretend to have one. I've spent the last few years building my own workflows on top of AI tooling — and picked up one habit along the way: when a paper, tool, or project shows up, I take it apart first, see how the inside turns, which parts bear the load and which are just decoration, and only then ask whether it's worth adopting. Every verdict carries a date stamp, so later facts can test it.
Corrections policy: if anything here is factually wrong, I'll mark the fix at the top of the piece with a date. Tell me via DM on X: @luming58112609 — how errors get fixed, and who gets told, is written here so it isn't just a promise.
Top comments (1)
The distinction between a durable structural claim and a fast-decaying inventory claim is useful. A practical citation workflow could store the source snapshot, version, and a small revalidation command next to each claim, then mark the claim stale when the command no longer matches. That gives readers a way to reproduce the calibration without treating a renamed constant as a refutation of the underlying mechanism.