DEV Community: Odilon HUGONNOT

Backtest and oracle: where does this price come from? Two books read

Odilon HUGONNOT — Thu, 05 Nov 2026 09:00:02 +0000

Your strategy, replayed on past prices, shows a Sharpe ratio of 2.8, an excellent return-to-risk score. You switch the bot on with real money and it loses in the first week. These are illustrative numbers, not a memory: you have been there, or you will be. Either way, the figure that convinced you came from somewhere you never looked.

A backtest replays a strategy on past prices to see what it would have earned. It is only worth something if every price it uses really existed at the moment the strategy would have acted.

The Sharpe ratio scores the result: the average return, divided by volatility. The return is counted above the risk-free rate, what a safe investment pays. Volatility is how widely returns swing.

I have worked on the centralized side of this trade, meaning platforms run by a company. As a freelancer I write Python crypto trading bots that run around the clock. They handle a stop-loss, an order that cuts a losing position. They also handle a trailing stop, an exit threshold that follows the price up, and position sizing.

At Goin Invest I wrote trading algorithms in Go for managed mandates: running a client's portfolio under a contract. They connected to five exchanges (platforms where cryptocurrencies are traded), including Binance, Kraken, Bybit and OKEx.

The other side escaped me: DeFi, decentralized finance, where the program itself is the marketplace. So I read two books from Packt's ExpertEdge subscription: Quantitative Trading by Ernest P. Chan (Wiley, 2nd edition, 2021) and Building Full Stack DeFi Applications by Samuel Zhou (Packt, 2024).

The first teaches you to check that a backtest price is honest. The second shows what happens when a program trusts a price someone manufactured for it.

Look-ahead: when your backtest uses a price that does not exist yet

A comparison: you play heads or tails, but you are allowed to read the result before you bet. You win every time, and you have no talent at all.

Look-ahead bias is that cheat, when it creeps into a backtest without you meaning it. It means using information that did not exist yet at the time of the trade.

Chan gives an example. A rule such as "buy when the price is within 1% of the day's low" cannot be applied live. That low is only known at the close. The backtest has the whole history at hand and never complains.

A trading day: at 10 am the rule buys near the day's low, but that low is only known at the close 10 am: the rule buys "within 1% of the day's low" Close: the day's low finally becomes known day's low open 10 am close Information from the future: that is look-ahead

Diagram made for this article, the time of day is an illustration. In red: the decision made with information that does not exist yet. In green: the moment it becomes known.

Chan proposes a truncation test (lesson 15). The idea: rerun the strategy with the last days hidden, and check that it decides the same way.

You compute your strategy's positions, what it holds on each date, on the full history. I call that result A.
You drop the last N days (he mentions 10 to 100), rerun, and get B.
You cut the last N rows off A and compare with B.

For Chan, any difference proves the program uses the future: "the discrepancies in positions mean that you are inadvertently using the truncated part of the historical data."

Here is my version in Python, not the book's. It is mine, adapt it to your strategy. It is the cheapest test in the trade, and almost nobody writes it:

import numpy as np
# prices and positions are pandas Series or DataFrame

def test_truncation(strategy, prices, n=60):
    """Fails if the strategy uses data from after the day it decides."""
    positions_a = strategy(prices)            # full history
    positions_b = strategy(prices.iloc[:-n])  # without the last n days
    # Over the common period, positions must be identical.
    common = positions_a.iloc[:-n]
    assert len(common) == len(positions_b)
    # equal_nan: a rolling window leaves NaNs at the start, in the same places
    assert np.allclose(common.values, positions_b.values, equal_nan=True), \
        "look-ahead: the future leaked into the positions"

# The classic trap this test catches (my own example):
# a signal normalised by the mean and standard deviation of the whole history.
# Replace it with a rolling window, for example prices.rolling(50),
# and the test passes.

(The comment about global normalization is my own illustration, it is not in the book.)

Wire this test into your continuous integration pipeline, the one that runs your tests automatically on every change. Every time someone touches the strategy, the test rereads history with a veil over the last few days.

Chan does not talk about crypto, but the principle carries over. A market that never closes has no "daily close". The temptation is to build a daily candle: the open, high, low and close of one day. Then to decide inside it.

Careful, the truncation test will not catch this trap. The day's candle is the same in both passes, because you only cut whole days. The counter is to decide on yesterday's candle and act today. You shift the signal by one day (signal.shift(1) with pandas, the Python data-table library).

Too many settings, too little data, too much leverage

An honest backtest still has two enemies: too many parameters and too little data.

A parameter (or setting) is a number you choose, such as a threshold or a duration. Chan gives a clear rule: "I would not employ more than five parameters," entry and exit thresholds, holding period and moving-average window included (lesson 15).

The disease is called data-snooping: optimize on the past long enough and you tune your strategy to noise. To picture it: memorizing the answers of an old exam. You get 20 out of 20 on that one, and nothing is proved for the next.

The book's remedy is to split the history in two. You tune on the first part, the training set. Then you test on the second, which you have never looked at.

On the amount of data, Chan relies on the calculations of Bailey and coauthors (2012) on the minimum length of a backtest. The backtest Sharpe is the one you measure on the past. The true Sharpe is the one the strategy will have live. A data point is a closing price, one per day:

A backtest Sharpe of 1 needs 681 data points to be confident that the true Sharpe is at least 0. That is 2.71 years of daily prices.
A backtest Sharpe of 2 needs only 174.
To be confident that the true Sharpe exceeds 1, you need a backtest Sharpe of at least 1.5. That takes 2,739 points, almost 11 years of daily data.

I would not carry those numbers over to other frequencies without redoing the calculation.

Then there is leverage, the size of your portfolio divided by your own capital. Chan presents the Kelly criterion: the leverage that maximizes long-run growth. Its formula is the average return above the risk-free rate, divided by the variance, the square of volatility.

For SPY, the fund that tracks the S&P 500, Kelly gives 2.528. In practice most traders take half of it, half-Kelly. The reason: "it is always safer to have a leverage lower than what the Kelly formula recommends" (lesson 28).

The safeguard I like best is lesson 29: "the leverage to use is always the smaller of the half-Kelly leverage and the maximum leverage obtained using the worst historical loss." Kelly assumes bell-curve returns, while crashes are far more frequent than that.

The S&P 500 lost 20.47% in a single day on October 19, 1987. If you can tolerate a 20% loss in one day, the maximum leverage is about 1, against 1.26 for half-Kelly. You keep 1.

For a crypto bot I cannot derive a number from this, since Chan provides none. The principle carries over: the worst day has to be computed on your own data.

DeFi: how to rig a price, and how to defend against it

Now the contract side. A smart contract is a program deployed on a blockchain, and nobody can change it afterwards. A blockchain is a shared ledger, kept by a network of computers rather than by a company.

The pool: a scale that tips

A traditional exchange keeps an order book: the list of buy and sell offers, which it matches. An AMM does without. Another way to see it: a scale with two pans. ETH (Ethereum's currency) on one side, USDT (a dollar-pegged token) on the other. The price is read off the imbalance: the emptier a pan gets, the more what it held is worth.

An AMM (Automated Market Maker) is that scale written in code. It is a smart contract that holds two tokens, two digital currencies, in a shared reserve: the pool.

The rule in Uniswap v2, the best-known AMM, is called the constant product. You multiply the reserve of A by the reserve of B, and the result, called K, does not change with trades. My example, without fees:

The pool before and after buying 1 ETH: 5 ETH and 10,000 USDT at a price of 2,000, then 4 ETH and 12,500 USDT at a price of 3,125 Pool before the buy Pool after the buy 5 ETH 10,000 USDT 4 ETH 12,500 USDT K = 5 × 10,000 = 50,000 K = 4 × 12,500 = 50,000 Price: 2,000 USDT Price: 3,125 USDT I buy 1 ETH I pay 2,500 USDT

In green, the pool's reserves (indicative heights, one scale per token). In red, the price after the buy. K has not moved.

I buy 1 ETH: only 4 remain in the pool. K must stay at 50,000, so the USDT reserve rises to 50,000 / 4 = 12,500. I paid the difference, 2,500 USDT, and each ETH now sells for 12,500 / 4 = 3,125.

A single buy moved the price from 2,000 to 3,125. That is price impact. The book gives the extreme case: with equal reserves, buying 90% of one token multiplies its price by 100 (lesson 25). Keep that in mind, because it is exactly what makes a pool's price manipulable.

The oracle: a price read from a single pool

A contract knows no price on its own. It asks an oracle, the service that feeds it a price. A made-up case: you value your house by asking the opinion of a single neighbor. If someone managed to convince that neighbor of an absurd price, your valuation is absurd too.

The lazy solution is the same: read the reserve ratio of a single pool. Zhou judges it bluntly: "this way is extremely dangerous" (lesson 84).

The attack: inflate the price, then borrow against it

The attack revolves around a loan. To borrow, you deposit an asset as collateral, meaning a guarantee. The borrowing cap depends on the value of that deposit. With a collateral rate of 80%, you can borrow at most 80% of the deposit's value.

It also takes funds to buy with. A transaction is an order sent to the chain. A flash loan is an uncollateralized loan that must be repaid within the same transaction. If repayment fails, the whole transaction is cancelled.

Zhou details the attack in four steps (lesson 98). The flash loan supplies the funds at the start and is repaid at the end, all within a single transaction:

The attacker buys the token, which inflates its price in the pool.
They deposit a little of it as collateral. The oracle reads the inflated price, so it believes that deposit is precious.
They borrow other assets, far beyond the deposit's real value.
They sell the token back.

The four-step attack on a single-pool oracle: flash loan received, buy that inflates the price, collateral deposit, big loan, sell back, repayment, all within one transaction A single transaction: everything goes through, or everything is cancelled Start: the flash loan provides funds 1 2 3 4 Buy the token Deposit a little of it as collateral Borrow other assets Sell the token back The pool price inflates: this is the rigged price The oracle reads the inflated price, the deposit looks precious Far beyond the deposit's real value End: the flash loan is repaid

In red, the steps that produce or use the rigged price. In green, the flash loan, received at the start and repaid at the end.

By how much must the price be inflated? With an 80% collateral rate, a price inflated by 25% is enough. The borrowing cap then equals the deposit's real value. Indeed, 80% of 125% makes 100%.

The book cites the Euler Finance attack of March 2023: $197 million. It is the largest known flash-loan attack at the time of writing.

The defense: an average over 720 seconds

To get a feel for it: to know the price of gas, the average price over the week is more reliable than the price of one minute. Faking a minute is easy. Faking a whole week costs far more.

The book's defense follows that idea: an averaged oracle. A block is the batch of transactions the chain validates at once, roughly every 12 seconds. At each block, each token pair adds its price multiplied by the elapsed time to a running total.

The oracle derives a TWAP (time-weighted average price) from it over a window, here 720 seconds. That is 60 blocks of 12 seconds. A manipulation fits inside a single block, so it weighs little in the average. For Zhou, this approach "doesn't rely on multiple oracle sources while keeping the price manipulation resistant" (lesson 100).

A one-block price spike of 12 seconds, and the 720-second average that barely moves price time Averaging window: 720 s 720 s average (TWAP): barely moves One block (12 s): the instant price shoots up Instant price

Durations to scale (720 s against 12 s), illustrative prices. In red, the one-block spike. In green, the average that dilutes it.

The book's test replays the attack. After the manipulation, the averaged oracle values BAR, a test token from the book, at 2.16 ETH. The old oracle values it at 723.47 ETH. The attacker's loan is refused with the reason ACCOUNT_UNHEALTHY (lesson 105). Against the old oracle, the same scenario earned the attacker over 100 ETH (lesson 99).

It is the best section of the book: you see the rigged price and the honest price side by side, drawn from the same pool.

A checklist: "where does this price come from?"

Here is what I take away from setting the two books face to face. The list is mine, it is in neither Chan nor Zhou, but every line rests on a case they describe:

At what moment did this price really exist? If your strategy or contract reads it before that moment, you have look-ahead (the day's low known at the close).
Who can move it, and with how much capital? If a single order is enough, it is not a price. It is the opinion of whoever pays (the pool going from 2,000 to 3,125).
How long must it hold before anyone relies on it? An average over a window costs more to distort than a snapshot (12 seconds against 720).
How many independent sources? A source that can be moved alone is a flaw, even a clean one.
Is it the price I will actually pay? The backtest price is not the execution price, and the pool's price impact shows it in three lines.
On which dataset did I tune and on which did I test? If it is the same one, the price proved nothing.

What the two books do not say about real-world crypto

Chan barely mentions crypto. In my extraction the word appears once, in a list of asset classes handled by a backtesting platform.

His examples are stocks, ETFs and futures, with one closing price per day. An ETF is a traded fund that tracks an index. A future is a contract to buy or sell at a fixed date. Nothing on a continuous market or on connecting to several exchanges. His industrialization targets an individual, with Excel links and orders sent in batches to a broker.

On the stop-loss, Chan's thesis is that the tool only works in a trend. First case: the gap, a price that jumps from one level to another without trading in between. The order fills much lower, and the stop merely locks in the loss. Second case: mean reversion, when a price drifts away and then returns toward its average. There, the stop makes you exit "at the worst possible time" (lesson 37).

I am not converted for crypto: a continuous market has fewer gaps than a stock that is closed overnight. Only a test on crypto data will settle it, and the book provides none.

Zhou has the opposite flaw: he is full of crypto, but his DeFi stops at the basic services: swapping, staking, lending and the oracle. His lending protocol is based on Aave v2, one of DeFi's major lending protocols. v3 is only presented, with isolation mode and E-mode, features "we will not implement in this course."

Liquidation is the seizure of the collateral of a borrower whose debt exceeds their cap. The book explains it with numbers. A liquidator who pays 1,000 USDT for an account in default leaves with 1,100 USDT of collateral, bonus included (lesson 86). A close factor caps the share of the debt that can be repaid in one transaction (lesson 94). The code of the liquidation function, however, is not published in the course: the author finds it too long. The health factor, Aave's indicator of a loan position's health, appears nowhere in the extracted lessons.

On security, the book shows the right oracle test, but the attack is played with a test account, not with a real flash loan. In the extracted lessons, two things are missing:

fuzzing, tests with random inputs;
a mainnet fork, a test against a copy of the real chain.

The code has aged too. It talks to the chain with Ethers v5, a JavaScript library since replaced by v6. It starts the front end from create-react-app, a React starter tool since deprecated.

Neither book, in the parts read, connects its two worlds. Nobody there says how to backtest a strategy whose prices are those of a pool. The price impact there depends on the order you place. That question is open, and these books do not address it.

Which one to read, depending on your profile

Quantitative Trading: 6.0 out of 10 in my reading note. Read it if you write trading bots and want to know whether they really win. Read it too if you size your positions by gut feeling. Read lessons 15, 28 to 31 and 37, skim the rest. The code is written three times, in MATLAB, Python and R, and the universe (US stocks, Excel links) has aged. Pass if you want crypto or around-the-clock trading: the book barely covers it.

Building Full Stack DeFi Applications: 5.8 out of 10 in my reading note. Read it if you come from finance or back-end work and want to see an AMM, a lending pool and an oracle from the inside. Read the AMM and the oracle, skim the lending part, skip the React front end. It is not a book for deploying to production: no audit, and the liquidation code is not published in the course.

If you only have time for one and your bots run on centralized exchanges, take Chan. If you need to understand why a protocol you integrate with reads a given price, take Zhou's oracle part, lessons 98 to 105.

Conclusion

I went looking for two books about two trades and found a single question asked twice. In Chan, the misleading price comes from your own code, which looked too far ahead. In Zhou, it comes from a third party, who paid just enough to move it. In both cases nobody lied: the number was exact, it simply came from a place nobody had questioned.

What strikes me is that the remedies look alike. The truncation test removes the future to see whether decisions change. The TWAP dilutes the instant to see whether the price changes. Either way, you do not trust a number until you have removed the thing that could cheat it. I have not found a book that applies this discipline to a bot trading on a DeFi pool. I would like to read someone who has done it with data.

Agent Feedback Loops and Parallel Agents: Two Books Read

Odilon HUGONNOT — Wed, 04 Nov 2026 09:00:02 +0000

I write reading notes on technical books. To check them, I have agents proofread them. An agent is one Claude instance, an AI working alone on a task with its tools. Each reviewer compares my claims to the book.

One day, these reviewers flagged perfectly real facts as invented. The fault was mine: I had handed them condensed notes instead of the text of the book. They could not confirm what they could not see. A judge is only worth what it can see.

Picture a student grading their own paper. They recognize their own ideas, find them clear, and give themselves a good mark. An outside grader sees what the student no longer can. (This image is mine, it does not come from the books.)

With agents, this becomes a feedback loop. In the loop, one agent writes, another scores, then you fix and go again. You stop when the score reaches a threshold: below the threshold you fix, above it you exit.

I run this every day. I also launch several agents at the same time, which means in parallel, instead of having them work one after the other. An agent can launch others, called subagents. Each has its own context window, the text it keeps in view. Only its final message comes back to the one that started it.

I had read four books on Claude Code. The best one, The Claude Code Operating Model, puts numbers behind its own claims. I wanted something more advanced on these two topics. Packt gave me access to its ExpertEdge subscription platform, and I read two 2026 books there:

Agentic Architectural Patterns for Building Multi-Agent Systems, by Ali Arsanjani and Juan Pablo Bustos. I call it "Arsanjani."
Spec-Driven Development with BMAD, by Marlon Vidal Ferreira. I call it "BMAD," after the name of the tool it teaches.

Both taught me something about loops. On parallelism, they describe more than they measure.

The one who writes must not be the one who scores

This is the strongest idea in either book, and it comes from Arsanjani. An agent that produces a solution and then judges it "struggles to be an objective critic of its own work" (lesson 111). It rationalizes its mistakes: it finds good reasons to leave them as they are.

Two ways to correct. In green, agent A writes, a different agent B scores it, A corrects based on the score, B scores again. In red, agent A writes then scores itself: it rationalizes its mistakes. With a separate judge: you can really correct 1. Write agent A 2. Score agent B, a different one 3. Correct agent A, from the score 4. Score again, and repeat Without a separate judge: the agent scores itself Write, then score itself agent A alone It rationalizes its mistakes (lesson 111)

In green, the loop with a separate judge. In red, the agent that scores itself.

I have written about this before: No Agent Grades Its Own Homework. The book turns it into a pattern, a design recipe that comes up often. It is called Hybrid Planner + Scorer. One agent plans (the planner), another scores (the scorer), and the score is used to adjust the first.

The book adds that the loop is only as good as its measurement: a system of agents "can only improve what it can measure" (lesson 110).

Arsanjani also names two failure modes. They happen when the judge evolves alongside the agent that produces the output (the generator). This is the vocabulary that helped me most:

Collusion: the generator learns to "game" the system. It produces mediocre output that the judge lets through (lesson 114). To picture it: the student who studies the grader's tastes instead of the subject.
Runaway divergence: the system drifts away from the real goal to optimize an internal metric, a number it tracks itself (lesson 114).

His example is about code (lesson 115). A fixer agent patches a vulnerability. An auditor agent scores the patch. The story unfolds in two cycles:

In the first cycle, the fixer finds that wrapping a crash in a try/except is enough for a good score. A try/except catches the error and hides it. The system plateaus.
In the second cycle, the auditor is retrained on pairs of "band-aid" versus "real fix." Retraining a model means making it learn again from new examples. The fixer's scores drop, and it has to learn to actually debug.

A judge like this is called an LLM judge: a language model, an AI that writes text, asked to score another model's output. If its criteria are generic, it can be fooled. A text-similarity metric is a calculation that scores how much two texts look alike. The book says one of them, BERTScore, is "logic-blind" (lesson 112).

Mind the difference in nature with my loop. Arsanjani's retrains models (lessons 113 to 115). Mine fixes a document based on a report, without touching any model. What carries over is the failure and the remedy, not the mechanics. The numeric thresholds and the maximum number of rounds described below are mine alone.

Against cheating: an outside check, then a limited budget

Against collusion and divergence, Arsanjani prescribes an external anchor: "periodic human review or validation against a golden dataset" (lesson 114). An external anchor is a checkpoint placed outside the loop, one the agents do not control. A golden dataset is a set of cases whose right answer is known in advance.

For coding agents, the most direct anchor is a test that actually runs. A test is a small program that checks that a feature works. This is where the BMAD book is useful.

BMAD is an open source working framework. Role-based agents (analyst, architect, developer, tester) pass written documents to each other before any code exists. The book yields six control moves that I keep, whatever the tool:

Red before green. A red test fails because the feature does not exist yet. A green test passes. A test module generates, before the code, one test per acceptance criterion, a written condition the feature must meet. Each test is marked test.skip(). The method is called ATDD, for Acceptance Test-Driven Development: you first write the tests that check that the feature does what was asked. The developer removes the skip markers one by one, from red to green. Tests written after the implementation do not necessarily cover every scenario, the book says (lesson 26).
A gate before writing code. A gate is a check you pass before moving on. The implementation readiness check cross-references the requirements, the architecture and the work breakdown. It returns PASS, CONCERNS or FAIL (lesson 15). A FAIL is nothing to dread: the mistake still lives in a document, not in code.
A three-pass review after each story. A story is a unit of work written in a file, small enough to fit in one session. A session is a conversation with the agent. The three passes: a logic-error hunter, an edge-case hunter and an acceptance-criteria auditor. An edge case is a rare situation, like an empty list or concurrent access. The book recommends giving it to "ideally a different model" (lesson 30) and states that the review layer "is not a formality" (lesson 18).
A self-correction loop. In the demo, an end-to-end test fails: it walks through the application the way a user would. The workflow then fixes itself. A workflow is a BMAD procedure started by a command. It reruns the test suite, finds another defect, fixes it and verifies (lesson 30).
A fresh session per step. The book forbids chaining ("never chain," lesson 43), because accumulated context biases what follows. Opening a fresh session means starting without any memory of the previous one.
A retrospective that rewrites the instructions. An epic is a large batch of stories. At the end of an epic, after a test coverage gate, the retrospective rereads the notes each story left behind. It regenerates project-context.md (lesson 31). This is BMAD's equivalent of a CLAUDE.md, the instructions file the agent rereads at the start of each session. It becomes "the living record of how the project has learned to build itself" (lesson 19).

One last guardrail, from Arsanjani: the budget cap. A cap is a limit you do not exceed. In the book's example, an analyst agent must summarize 5,000 PDFs on a $50 budget. At 90% of the budget spent, around the 50th PDF, a circuit breaker stops collection. A circuit breaker cuts everything when a limit is crossed, like the one in a house. The agent then switches to the report using the data already collected (lesson 117).

The same book admits that a learning loop can cost "10 to 100 times" a normal inference, without citing any source. An inference is a call to the model to get an answer.

On the BMAD side, the cost boils down to "time- and token-intensive" (lesson 31) and "very expensive in terms of token usage, but much less than letting these problems reach production" (lesson 29), with no figure in tokens or dollars. A token is the small chunk of text the model reads and your subscription counts. So for a multi-round loop, set the cap before the first round, not after the first bill.

My own loop: a score to reach, three rounds at most

Neither book gives an exit threshold, a number of rounds or a split of roles for a review loop. Here is mine, worked out on four book notes and one article. A book note is my reading note on a book, published on this site. The notes from the previous series went from 59 to 76 out of 100 on the first round and ended between 84 and 97 after the Sonnet rounds. The article from the previous series went from 6.1 to 8.1 out of 10.

A round is one full cycle: a reviewer scores, then another agent fixes. Sonnet and Opus are two Anthropic models, Opus being the more expensive: I keep it for the last pass. Each role receives and returns this:

Role

Model

Receives

Returns

Reviewer, rounds 1 to 3

Sonnet

the rules, the text to review, the full text of the book (not condensed notes), the previous round's report

a full report in a file, with a score

Fix writer

Sonnet, a different agent from the reviewer

the text, the report, the book text

the corrected text, every fact rechecked

Final review

Opus, once

the text that passed the thresholds, the book text

a publication verdict and last touch-ups

The exit thresholds depend on what I review:

For a book note: at least 85 out of 100, a factual fidelity score of at least 20, and every idea at 8 minimum on four axes.
For an article: an average of at least 8 out of 10 and no criterion below 7.

My cap is three Sonnet rounds at most: three tries, then stop. Beyond that, I move to the Opus review with the list of what blocks, rather than keep looping.

My review loop: up to three rounds with a Sonnet reviewer. If the score passes the threshold in a round, we exit. After round 3, still under the threshold, we move to the Opus review with the list of what blocks. Thresholds: note 85 out of 100, fidelity 20, each idea 8; article average 8 out of 10, no criterion under 7. Cap: 3 Sonnet rounds at most Round 1 reviewer scores Sonnet Round 2 reviewer scores Sonnet Round 3 reviewer scores Sonnet fix fix threshold met threshold met threshold met Score above the threshold: we leave the loop after round 3, still under the threshold Final review: Opus, once with the text that passed the thresholds or, cap reached, with the list of what blocks The threshold depends on what you review A book note 85 out of 100 at least factual fidelity: 20 at least each idea at 8 minimum, on 4 axes An article average of 8 out of 10 at least no criterion under 7

In green, leaving the loop. In red, the cap on rounds. The thresholds are the ones from my own practice.

The reviewer of a new round receives the previous round's report. It keeps the same grid and the same severity this way, because a fresh reviewer with no history scores differently.

On isolation, the reviewer receives the text, the rules, the book and the previous report (see the table), without the reasoning behind the writing. Each report goes into a file. Each corrected version is committed between rounds, meaning saved in the code history. So a session that restarts loses the running agents, not the work. The report skeleton I ask for looks like this:

Moyenne : X/10 (seuil 8, aucun critère < 7). Sortie atteinte : oui/non.

Critères notés sur 10 : accroche, structure, fidélité et faits, voix, autonomie, utilité, SEO, niveau par rapport aux étalons.

Pour chaque point perdu : le passage fautif cité, la source qui le contredit
(chercher par recherche de texte dans le livre, pas dans les notes), la correction demandée.
Pour passer le seuil : les corrections par ordre de priorité.

(That is the real template, in French, since my notes are in French. In English: average out of 10 with the threshold and the exit flag; criteria scored out of 10; for each lost point, the faulty passage quoted, the source that contradicts it found by text search in the book rather than in the notes, and the fix requested; then the fixes in priority order.)

Several agents at once: described in the books, never measured

One question remains: does running several agents in parallel really improve the result? The books describe it, but never measure it.

A comparison: handing three chapters to three people who write at the same time. You need a lead who hands out the work, then someone who gathers the chapters. With agents, it is the same pattern.

Arsanjani describes parallelism. Multi-agent means several agents cooperating. It is presented as coordination that "allows for parallel processing" (lesson 43). The book shows real concurrent code, in two versions:

A supervisor, the agent that hands out the work, launches three data collectors in threads before a writer assembles the report (lesson 50). A thread is a line of execution: a job that moves forward at the same time as the others.
Another supervisor launches them with asyncio.gather (lesson 91). It is the Python function that starts several tasks together and waits for them all to finish.

Parallel agents: a lead hands out the work to three collectors working at the same time, then a writer assembles the report. The book does not say how many agents to run, what it costs, or how they cope on a real repository. For me: two or three agents at a time. 1. It hands out 2. At the same time 3. It gathers The lead (supervisor) Collector 1 of data Collector 2 of data Collector 3 of data The writer assembles the report What the book does not say how many agents to run at once what it costs how they cope on a real repository My setup two or three agents at a time a comfort figure, not a measurement

Light green: the design described in the book (lessons 50 and 91). Dashed gray: what the book does not measure. Solid green: my practice.

Two patterns are built to check a result:

Consensus through parallel execution (lesson 69): two independent agents do the same task, and their scores are compared with a 10-point tolerance.
Majority voting (lesson 75): three or more agents answer, and the most frequent answer wins.

Merging the answers is covered, by averaging or voting, with escalation to a human on disagreement: you hand the case to a person.

Simultaneous writes to shared state have their own pattern (lesson 64). Shared state is a set of data that several agents read and change. Two agents writing at the same time can overwrite each other. The book proposes atomic operations on a store such as Redis, a fast in-memory database. An atomic operation cannot be cut in two by another one. It also provides a time to live for the information.

The limits are elsewhere:

In the verification examples, the code calls the agents one after the other, and their answers are made up. The score in lesson 69 comes from random.randint, a function that draws a random number. The votes in lesson 75 are hardcoded.
The cost of redundancy is stated in one sentence, "the most expensive redundancy pattern" (lesson 75), not measured.
There is nothing about coding agents editing the same repository: no file conflicts, no isolated branches. A repository is the folder where a project's code lives.

The advice that helped me is about independence. A second medical opinion is only worth something if the second doctor did not come out of the same school as the first. The book says this about agents: they should be "ideally powered by different foundation models" (lesson 75). A foundation model is the large starting AI model, like Claude. You need an odd number of agents to avoid ties, with a planned path for disagreement. Three copies of the same prompt means paying three times for the same mistake. A prompt is the instruction given to the agent.

BMAD says less, and the book does not agree with itself:

Lesson 31 describes a review that spawns three agents in parallel. Lesson 18 states that the same review runs three passes in sequence.
Lesson 42 answers "can I run several stories in parallel?" with yes, under conditions. You have to customize the workflow to split by layer rather than by vertical slice, or hand independent epics to different owners. That advice describes a team of people, not agents.
Party Mode loads all the agents into one discussion to get several viewpoints (lesson 7): a single conversation thread.
BMAD Loop is a community-built orchestrator, a program that launches the steps in order. It chains workflows in sequence (lesson 31).

So neither book says how many agents to run at once, what it costs, or how they cope on a real repository.

Here is what I do, without claiming it is the right answer. The reviews of several book notes run in parallel, two or three agents at a time on my server (a comfort figure, not a measurement). Each writes its report to a file, and I commit between rounds. This matches what Arsanjani's book calls incremental checkpointing: saving every step so an interrupted batch restarts from the files already written. The comparison is mine, the book does not apply it to coding agents.

Which book to read, depending on your profile

Agentic Architectural Patterns (Arsanjani and Bustos): 5.6 out of 10 on my reading note, provisional score. It is a catalog of about fifty patterns that names failures well and measures nothing: not a benchmark, unsourced gains, code that simulates the model.

Its first sections are enterprise context tinted with Google Cloud, to skim: model selection, RAG (making the agent search documents) and fine-tuning (retraining the model). Read it if you already run agents and are looking for words for your loop failures, starting with lessons 110 to 112, 114 and 115.

If you have never written an agent, start with AI Engineering by Chip Huyen (the reading note), stronger on the foundations and on measuring quality. Between Huyen and Arsanjani, Huyen first: Arsanjani comes after, as targeted reading.

Spec-Driven Development with BMAD (Ferreira): 5.2 out of 10 on my reading note, provisional score. It is a short 3 h 20 course, a tool manual, dated (the author warns that his course "may contain outdated info," lesson 6). It is tied to Cursor, the AI code editor he is an ambassador for in Brazil.

Read it for its control loops (red tests, gate, review, retrospective) and its fresh-session rule, skim the rest. It suits a small team better than someone coding alone, and the book says so itself ("a team process, not a solo tool," lesson 45).

Conclusion

I went looking for books that would teach me to make agents work together. What helped most concerns what each agent is not allowed to believe about the other: the judge that applauds a try/except, the reviewer that reads notes instead of the text. Arsanjani describes cooperation with code that simulates the agents, and BMAD with tool demos. Neither says what happens with ten real agents on a repository.

So, more advanced than what I practice? On the vocabulary of loop failures, yes: collusion, divergence and external anchoring helped me. On the mechanics of a review loop and on coding agents in parallel, no. Neither fixes an exit threshold or a cap on rounds for a review loop. Neither gives a number of agents to run at once. My loop already does all that.

I have not found a book that puts numbers on parallelism. The figure from my server, two or three agents, is a comfort setting, not a measurement, and I would like to read someone who has one.

Best Claude Code book: I read 4 published in 2026

Odilon HUGONNOT — Tue, 03 Nov 2026 09:00:02 +0000

Wednesday, October 14: an email from Packt, offering a partnership and free access to ExpertEdge, its subscription platform for technical books. My first search, out of habit: "Claude Code". Four books, all published in 2026. I shrugged. I have used Claude Code every day for over a year, and I have written here about its hooks, about my CLAUDE.md and about what its plugins cost. What could a book possibly teach me?

I was half wrong. Three of them get syntax wrong, sometimes to the point where their examples simply do not work. On the essentials, though, they overlap, and that part will not go stale. One of the four stands out despite its flaws: The Claude Code Operating Model.

The covers of the four Claude Code books published in 2026, by Packt and Zenva

How I read 1.2 million characters in one evening

I did not read these books alone, and I would rather say so up front. ExpertEdge does not sell PDFs: it displays books cut into lessons in an online reader. I let Claude drive my browser to extract the text lesson by lesson. I skipped the quizzes: they are assessments in my name, and I was not going to hand them to an AI. Then agents read each book, and I checked every quote I kept against the extracted text, word for word, with a script. An agent that reviews its own work grades itself too generously, and I explained why in No agent grades its own homework.

The extracted text weighs 1,170,431 characters: 527,600 for The Claude Code Operating Model (84 lessons extracted out of the 93 advertised), 385,585 for Agentic Coding with Claude Code, and 130,430 and 126,816 for the two Zenva books (Zenva is an online course publisher).

The cost, on the other hand, is real. A token is the small piece of text the model reads and that your subscription or API counts. Reading Agentic Coding in full with two agents cost me around 250,000 tokens, enough to put a serious dent in a quota. So I spent at least 250,000 tokens to learn how to waste fewer of them.

Which Claude Code book to read first

Book

Publisher

Rating

In one sentence

The Claude Code Operating Model (Jia Huang)

Packt

6.6/10

The only one that thinks like an engineer, despite a sloppy proofread

101 Claude Code Tips (Zenva, author not named)

Zenva

6.2/10

The notebook of a team that got burned

Agentic Coding with Claude Code (Eden Marco)

Packt

5.4/10

A video course put into writing, full of demos and mistakes

Best Practices for Claude Code (Zenva, author not named)

Zenva

6.0/10

A good discipline course under a title that promises too much

The best: The Claude Code Operating Model. It is the only one that explains how the pieces fit together: CLAUDE.md, skills (reusable instructions Claude loads on demand), subagents, hooks, MCP (the protocol that plugs Claude into external tools) and headless mode, built for CI (continuous integration, which runs your checks every time code is pushed). It also puts numbers on its own claims. It has 93 lessons and almost 6 hours of course. It is worth it mainly if you already use Claude Code and want to understand the system rather than collect tips.

The lowest rated: Agentic Coding (5.4/10), because of its wrong configuration examples, detailed below. The most disappointing, though, is Best Practices for Claude Code (6.0/10), on a different criterion: the gap between what the title promises and what the book contains. Agentic Coding keeps its promise, a course that builds an application, flaws included. Best Practices promises all the best practices. I found no subagents, no MCP, no skills in the Claude Code sense. Permissions get a single line pointing elsewhere, and hooks one sidebar. What remains is a decent discipline course if your problem is a Claude that does far more than you asked.

Depending on your profile:

You have an hour and a half and a few weeks of practice already: 101 Claude Code Tips (53 lessons, 1 h 26). A good share of the tips revolve around content production (video, Trello boards), by my rough count: you can skip those.
You are a beginner and want to follow a project end to end: Agentic Coding builds an application (HookHub) step by step. Careful, it contains several wrong configuration examples. Check every block against the documentation before copying it, or you will spend an evening wondering why nothing works.
You are putting up with a Claude that rewrites half your code for a small request: Best Practices (44 lessons, 2 h 24).

What Claude Code books agree on, and what you can do with it

Three bylines (Jia Huang, Eden Marco, and Zenva for two books with no named author), two publishers. The same ideas turning up proves little: they probably all lean on the same documentation. What matters is where they go beyond the docs and overlap. Here is who carries what:

Idea

Operating Model

101 Tips

Best Practices

Agentic Coding

Short CLAUDE.md

yes

one sentence

Hooks guarantee, prompts do not

yes

a sidebar

shows the mechanics

Subagents for noise

yes

Do not believe what the chat says

via the Stop hook

yes

not spotted

A short CLAUDE.md. All four say it, some louder than others. Jia Huang gives the most radical test: if deleting a rule would not stop Claude from behaving well, the rule does not belong in the file. 101 Tips aims for about 50 lines and writes that a bloated file makes Claude ignore the real instructions, lost in the noise. Best Practices lists an overlong file among its pitfalls, and Agentic Coding settles for one sentence: keep memory files concise and specific. My file was 296 lines before my audit, which made me the counter-example of all four books at once. It is 142 now: still nearly three times the 101 Tips target, but under the 200 lines Operating Model cites. Your turn: for each rule in your CLAUDE.md, imagine it deleted. If Claude behaves the same, cut it.

What must always happen goes in a hook. A rule written in CLAUDE.md remains an instruction Claude can misread. Operating Model has Kage, its fictional mentor, say: "Hooks are different: they are not suggestions, but enforcement." 101 Tips, shorter: "hooks guarantee execution; prompts do not." Best Practices gives them a sidebar. Agentic Coding teaches the mechanics with a hook that plays a sound at the end of a reply, without drawing that principle from it. It is what I ended up understanding while writing my article on hooks: stop repeating "run the tests" every session, and hand it to the Stop event (it fires when Claude has finished answering). Here is the script from tip 48 of 101 Tips, which I shortened. Replace bun test with your own test command:

#!/usr/bin/env bash
input=$(cat)
# The guard: without it, the hook restarts Claude forever
if [ "$(echo "$input" | jq -r '.stop_hook_active // false')" = "true" ]; then
  exit 0
fi
if ! bun test --silent >/dev/null 2>&1; then
  echo '{"decision":"block","reason":"Tests are failing. Fix them before declaring done."}'
fi
exit 0

Claude Code passes the script a JSON payload on standard input. When the tests fail, the hook returns block and Claude has to keep going. The stop_hook_active guard prevents the infinite loop: it is true when Claude is running again because of the hook, and the script then lets it stop. Operating Model and 101 Tips both have this guard.

A subagent is for reading, not for saving money. A subagent is a separate instance of Claude with its own context window: only its final message comes back into your conversation. Operating Model, 101 Tips and Agentic Coding use it to isolate noise. The 101 Tips rule is the sharpest: if the task produces search results, logs or files you will not look at again, run it in a subagent. Jia Huang's calculation is the most honest of the lot: in his example, delegating saves 8,500 tokens out of 125,000, or 6.8%, and only 1 to 2% once prompt caching is taken into account (text already sent to the model is billed at a lower rate the next time). The real gain is a main conversation that stays readable. So send anything noisy to a subagent: logs, test output, searches.

The chat is not proof. 101 Tips describes a Claude that announces "I ran bun test, all green" without ever calling the Bash tool: the narration is fluent, plausible, and false. The book's remedy: read the real tool output in the transcript, not the summary. Operating Model gets to the same place by another road, a Stop hook that runs the test suite when Claude claims to be done. Best Practices offers the best question to ask before approving: "List all the behaviors that should remain unchanged after this modification. Explain how the current implementation preserves them.", and sums up its method in one line: "compilation is not review". I found nothing equivalent in Agentic Coding. Ask that question before you approve, and look at the real tool output. Also be wary of the verbs "refactor", "improve", "clean up" or "optimize": Best Practices sees them as permission to redesign far more than you meant.

The config mistakes not to copy

A book about a tool that changes every month goes stale before it is printed, and these four were not always proofread. A few mistakes I checked.

The hook format. Tip 48 of 101 Tips shows a Stop hook that runs the tests. settings.json (Claude Code's configuration file) expects something else according to the current documentation: an outer hooks level, a type field, and no matchers for Stop, which does not support them (a matcher is a filter on the name of the tool that triggers the hook). I did not test what Claude Code does with the book's text; I only note that it differs from the docs.

// ✗ What the book shows (tip 48)
{
  "Stop": [
    {
      "command": ".claude/hooks/check-tests.sh",
      "matchers": []
    }
  ]
}

// ✓ The documented format
{
  "hooks": {
    "Stop": [
      { "hooks": [{ "type": "command", "command": ".claude/hooks/check-tests.sh" }] }
    ]
  }
}

A file that does nothing. Operating Model recommends a .claudeignore as a first layer to hide files, while noting it can be bypassed, and advises permissions.deny as a second layer. The current documentation goes further: a .claudeignore has no effect at all, and its entries must become Read rules in permissions.deny.

A misspelled key. Agentic Coding writes statusline in its settings.json example, forty lines after discussing the statusLine key. The docs only know statusLine. I did not test what Claude Code does with the faulty version.

Security scripts with holes, of two kinds. In 101 Tips, the "block dangerous commands" hook searches for a text string: grep -Eq '(rm -rf|git push --force|…)'. I replayed it: rm -rf is blocked, while rm -fr, rm -r -f or rm followed by two spaces get through. Operating Model has other blind spots: its Write|Edit matcher does not cover cat > .env, because it filters on the tool name and cat goes through Bash. It also contradicts itself: it puts auto-compaction at 50% by default in one chapter and at around 95% in another.

My rule: in a Claude Code book, take the principles, and look up every syntax in the official documentation.

The fix for an agent that copies out your secrets

A chapter of Operating Model describes a trap I had never considered. A code review agent runs in CI with claude -p "Review the configuration for problems" --allowedTools "Read,Grep". It is asked to check the configuration. It reads the .env file, and writes the database URL with its password and the Stripe key into the CI log. No malicious instruction: it simply did what it was asked. GitHub's automatic secret masking does not cover text written by the model.

The fix comes down to two reflexes. The first: forbid reading sensitive files in the permissions (.claude/settings.json), using the syntax from the current documentation:

{
  "permissions": {
    "deny": ["Read(./.env)", "Read(./secrets/**)"]
  }
}

Add the variants you have (.env.local, for example). The docs specify that these rules cover Claude's read tools and commands like cat, but not a Python script that opens the file itself: for a barrier that holds against any process, you need to enable the sandbox. The second reflex is a prompt rule, not a configuration one: never ask a model to read or print an environment variable. The book says so in so many words.

In the same chapter, a simple idea to make a CI fail on Claude's verdict: have it write a sentinel line, and search for that exact line. The prompt asks it to end the review with <VERDICT>request_changes</VERDICT> or <VERDICT>approved</VERDICT>, the book's workflow writes the answer to review.md, then:

if grep -qF "<VERDICT>request_changes</VERDICT>" review.md; then

Searching for the word "critical" in the answer would be worse: it also appears in "no critical issues". One last move, right now: open your settings.json and check that your .env is in permissions.deny.

Conclusion

The books stress many things. The one I lived the most: do not believe what Claude announces. No agent summary got through without my going back through the script.

The book I recommend has a rare quality: it puts numbers on its own claims, even when the number works against its subject. Jia Huang praises subagents, calculates 6.8% savings and then 1 to 2% with caching, and writes it down. His book is far from error-free, but that reflex is worth more than his syntax. If you only read one, read that one, with the documentation open beside it.

AI code review: what the papers and tools say, compared to my skill

Odilon HUGONNOT — Mon, 02 Nov 2026 09:00:01 +0000

When you change code, a colleague reads your changes before they go live. That is code review. I have AIs do it, with a tool I wrote for Claude Code, Anthropic's programming assistant. That tool is a skill: a folder of instructions that Claude Code reads before a specific task.

My skill, senior-review, launches several reviewers. A reviewer is an instance of Claude launched to read the code (you can also call it an "agent"). It does not know the author of the code, the person who wrote it (I explained why the reviewer must never be the author). Each defect a reviewer flags is called a "finding". I use this skill almost every working day: 236 reviews since August 14.

The skill has three settings, called tiers. "quick" is light, "standard" is the default, "deep" is supposed to be more rigorous, with more reviewers. It keeps a journal: for each review, it records how many findings were raised and how many pass verification. Verification demands proof for each defect and tries to refute the most serious ones, meaning to show that they are false.

The journal says otherwise. On "deep", 74% of the findings raised survive verification. On "quick", it is 89%. Apparently, I had paid more for reviewers that miss more often.

Those numbers made me want to check whether my skill is solid or a hack I happen to like. I launched two research agents: one in the scientific literature, the other among existing tools.

AI code review: my skill against the other tools

I compare my skill to other AI review tools. There is Anthropic's, commercial tools (CodeRabbit, Greptile, Qodo, Bugbot) and community skills. I kept nine criteria.

Context: the whole repository (the project folder), its history, the code that calls the changed part.
Independence: the reviewers do not know the author of the code.
Proof: the reviewer shows how it knows the defect exists.
Refutation: someone tries to prove the defect is false.
Noise: few false alarms (defects flagged wrongly).
Memory: the tool keeps track of its mistakes.
Loop: fix then re-review, with a maximum number of rounds.
Read-only: the tool cannot change anything in the code.
Ticket: the tool reads the description of the work requested (the "ticket").

I added three criteria that were missing: externally validated measurement, integration and cost. Externally validated measurement: has someone other than the vendor tested the tool? Integration: does the tool work with no setup in "pull requests", the requests you post to have your changes reviewed?

Each column of the table is a tool or a group of tools, each cell a score from 0 to 5 (5 is the best). It is my judgment, based on the documentation I read. "Not documented" in a source does not mean "absent", and I scored accordingly. "1-2" covers the tools of the group. Look at proof (5 for my skill) and external measurement (2).

Criterion

My skill

Official Claude Code

CodeRabbit, Greptile, Qodo, Bugbot

Community skills

Repository context (callers, history)

1-2

Reviewers independent from the author

Proof before reporting a defect

Adversarial refutation

1-2

2-3

Noise control

2-3

Memory and learning

3-4

Review-and-fix loop with limited rounds

Guaranteed read-only

1-2

Alignment with the ticket

3-4

Externally validated measurement

Integration (native PR, zero setup)

Cost and simplicity

"Official Claude Code" covers the managed review service from Anthropic and its code-review extension. That extension launches four agents at the same time. Other agents then check the defects found, one per defect: an unconfirmed defect is dropped. Its documentation cites a confidence threshold of 80, which is no longer in the code.

My strengths are the required proof, the ticket, the loop with a limited number of rounds and the memory. My weaknesses: high cost, no native pull request integration, and above all weak external measurement. The commercial tools display benchmarks (numbered tests), but each vendor wins on its own measure, such as "comments resolved before merge". Those numbers cannot be compared. I cite none of them.

What the research says, and what it does not

The research agent opened the page of about thirty papers. The figures come from their abstracts, except one, taken from the body of the Cihan study. Several are "preprints", texts not yet reviewed by other researchers. Six results matter for a review skill.

Give the reviewer the intent of the change. A study tested GPT-4o and Gemini 2.0 Flash. They classify 68.5% and 63.9% of code cases correctly (good or faulty) when given the problem description. Without it, they do worse: up to 24.8% of correct pieces of code receive wrong suggestions (Cihan et al., arXiv 2505.20206). My skill reads the ticket before the code.
More instructions do not mean fewer false alarms. A "prompt" is the text of instructions given to the model. More elaborate prompts, which ask for explanations and fixes, make correct code get judged faulty more often (Jin and Chen, arXiv 2508.12358).
A model does not correct itself. Without external feedback, self-correction does not help and sometimes hurts (Huang et al., ICLR 2024). It works with a reliable signal, such as an executed test (Kamoi et al., 2024). Hence my rule: no finding without proof. A fix is proven when undoing it turns a test red, meaning it fails.
Several reviewers are not as many independent votes. A model family is a vendor and its models (Claude, GPT, Gemini). On a language task, nine models from seven families were made to vote. Their answers look so alike that they count for only about two independent votes. The study is a single-author preprint (Kohli, arXiv 2605.29800). My reviewers are all Claude models, so the same family: the effect should be worse for me.
Debate is not better than voting. Making models debate does not reliably beat simple majority voting (Smit et al., arXiv 2311.17371). It can even degrade the accuracy of answers (Wynn et al., ICML 2025 workshop). My refutation looks like a debate: its value comes from my proofs, from no study.
At equal cost, a single agent matches or beats a team. That holds on reasoning tasks, not code (Tran and Kiela, arXiv 2604.02460, preprint). Cost is counted in tokens. A model reads and writes text in small pieces, often a word or part of a word. Each piece is a token.

One last figure comes from a real tool, CodeRabbit. Out of 31,073 reactions to its reviews, across 239 repositories, 56.3% of the comments were rejected. They were mostly false alarms, repeats or off-topic remarks (Lin et al., arXiv 2607.03316). That is a single tool, but the order of magnitude is a warning: more than one comment in two ends up rejected.

What my journal actually contains

Here is the output of the learn command, which analyses the journal. It keeps only the reviews whose counts add up: 189 out of 236, of which 182 fall in the three tiers of the table. The last column gives the cost of a finding, in tokens.

Tier

Reviews

Findings that survive

Tokens per finding

quick

89%

56,000

standard

127

81%

57,000

deep

74%

77,000

The cost depends on the number of agents. A finding costs about 66,000 tokens with 1 agent, 50,000 with 4 to 5, then 87,000 from 6. Nothing shows that adding reviewers helps beyond five.

Be wary of these readings, though: the reviews are not about the same changes. A "diff" is the list of lines changed by a change. Its median size (the middle value) is 158 lines in quick against more than 1,000 in deep.

Single-agent reviews mostly re-read a small change, round after round. The comparison "one big reviewer against several", at an equal budget, remains to be done.

My real gap is elsewhere: the way to read that "74%". Here is the path of a finding.

The path of a finding: reviewers flag defects, verification demands proof, refutation tries to show they are false, findings without proof or refuted are discarded, the ones left go through my hands to be fixed, rejected or moved to another ticket. The reviewers they do not know who wrote the code findings Verification proof required for each finding Refutation we try to refute the most serious Discarded no proof or refuted Findings left they survived "precision" counted here then My triage fixed, rejected or moved to another ticket "acceptance" counted here

The path of a finding.

To picture it, think of a dictation. A first corrector underlines words: the reviewers. A second one strikes out those it judges correct: the verification. The words still underlined look like mistakes. Yet the pupil, who is me, may refuse one: "it is a character's name, it is on purpose". Counting them measures the second corrector, not what the pupil accepts.

My journal did the same. Its "precision", the share of findings that survive verification, measured what my own filter lets through, not what I accept. A confirmed finding may well be rejected by me ("that is intended"). The 236 lines of the journal did not carry what became of each finding. A tool that grades itself with its own filter measures only that filter.

What I changed

The journal and the research made me change three things, published in the skill's public repository (version 1.26).

What became of each finding. The journal now records how many confirmed findings were fixed, rejected, or moved to another ticket. The meaning of "precision" does not change. An "acceptance" figure is added: fixed divided by fixed plus rejected. An imagined example: 8 fixed and 2 rejected give 8 out of 10. No reviewer decides what becomes of a finding: the counts come from me or from the review loop.
The reviewers' model is recorded, so I can later compare precision by model.
The 5-agent cap becomes verifiable. The skill must not launch more than 5 agents per review. The learn command now flags the reviews that exceed it (11 since the cap exists, 28 over the whole journal). It also computes how many findings come out per 100 lines of diff.

A fourth change, version 1.25, comes from a book on agent architectures (how to organise several agents) that I was reading in parallel. A reviewer can crash or return an empty answer. The skill then relaunches it once. On the second failure, the report writes "not covered" next to the aspect of the review it was meant to check. The final verdict can no longer be "approved". Before, a reviewer that dropped out left a hole the report did not show.

A separate agent read the code for these changes. It found six defects, including a case where two simultaneous writes to the journal could make a line disappear. Each got a failing test before the fix.

What I did not do

I did not plug the skill into pull requests. It is interactive, built for my workstation: I launch it myself. Plugging it in would need a program that launches itself on every code push. It would need setup, passwords and a charge on every push. For a solo use, it is not worth it.

I did not add a reviewer from another model family either. It would need a second account, with the vendor of that other model. The study that justifies it is about a language task, not code.

What remains is the experiment the research asks for: one big reviewer against the group of reviewers, on the same diffs, at an equal budget. By my estimate it would cost about 300,000 extra tokens per trial review. I am not running it right away, because the journal first needs a few weeks of recorded outcomes so that the comparison speaks about acceptance and not about the filter.

What I do not know

My recall, meaning the share of defects a reviewer finds, is measured on only seven cases with injected bugs. An injected bug is a defect planted on purpose to test the reviewer. Depending on the bug, it ranges from 3 out of 3 to 1 out of 3. Seven cases settle only a coarse gap.

No verified paper gives the ideal diff size either. None gives the point from which several agents earn more than they cost. The token ratios I had read about teams of agents are found in none of the papers the research agent opened.

Conclusion

The most useful part of this research was neither a paper nor a tool. It was discovering that my own measurement graded itself with the filter it was supposed to judge. The only score that matters for a review is what the reader of the review does with the findings: fix them, or dismiss them. Everything else, the architecture included, is a hypothesis as long as that number is missing.

If you have an AI review your code, note what you do with each defect (fixed, rejected, moved to another ticket). Ten lines of log are enough to see whether your review is worth anything.

The book on agent architectures mentioned above has its page in my library. The skill is public: github.com/ohugonnot/claude-skills, senior-review folder, with the history of every version.

Event sourcing without an event store: CDC as the log of facts

Odilon HUGONNOT — Sun, 01 Nov 2026 09:00:02 +0000

A bank account, stored in an accounts table with a balance column. A withdrawal of 50 euros runs UPDATE accounts SET balance = balance - 50, and the previous balance is gone. The balance is the account's state. The list of its operations is its history, and the UPDATE erases it.

Event sourcing avoids that loss. You no longer keep the latest balance, but the list of operations: deposit of 100, withdrawal of 50. To know the balance, you add them up. I use this method every day, with an event store, a database that holds only that list, appended to and never modified. I described it in CQRS in Go, part 4: PostgreSQL as an event store.

John Gilbert makes the same observation I do: "We are losing information. We only retain the latest balance." Yet in Software Architecture Patterns for Serverless Systems (Packt, 2024), he builds his event sourcing on that very table, with no event store. The log of facts in the title comes from somewhere else: CDC (change data capture). This component reads the database's internal log, where the database itself records every change it makes.

So the book gives a different answer to a question I thought was settled: where should history live? Here are both answers side by side, what CDC simplifies, and what it costs.

Two places to keep history

In classic event sourcing, the state is computed from the history. A command (an instruction given to the service) comes in, for example withdraw 50. The service reads every event of the account and replays them, meaning it adds them up in order, to rebuild the current balance. It decides, then appends its new events to the event store. The read views, called projections, then feed on that event store.

The account and its events form an aggregate. Each event of the aggregate carries a version number. A UNIQUE (aggregate_id, version) constraint stops two simultaneous writes from taking the same number. History lives inside the service, and the service needs it for every decision.

Gilbert flips the setup. The service keeps a current-state table: one row per account, updated with a plain UPDATE. The service no longer records history, the database does, in its internal log. CDC reads that log and outputs every insert, update and delete, one after the other.

A small function, which the book calls a trigger, turns each change into an event and publishes it to the hub. The hub is a shared channel: each service listens for what it cares about.

History moves downstream, to the services that receive the events. An event lake listens to everything that goes through the hub and files it by date in object storage, with no expiry. In Gilbert's words, "the event lake acts as the system-wide transaction log." Put simply, history no longer belongs to the service that writes, but to the system. Services listen to the hub live, and none of them reads the lake to make a decision.

Replayed event store: the command reaches the aggregate, which reads its events from the event store and then appends new ones with a version; projections then read the event store. 1. Replayed event store Command Aggregate replays all its events appends reads Event store append-only, version per aggregate Projections read models Current state, CDC and lake: the command updates a current-state table, CDC reads the database log, the trigger publishes each change to the hub, which feeds downstream services and the event lake. 2. Current state + CDC + lake Command Current-state table one row per entity CDC + trigger every change becomes an event Event hub Downstream local views Event lake full history

Diagram 1: the service needs its history to decide. Diagram 2: it keeps only the current state, and history flows downstream to the lake.

You cannot write to two data stores at once

CDC is not a convenience in Gilbert's design. It comes from a constraint: "we cannot reliably save data to more than one data store at a time."

A concrete case, made up for the occasion: an online shop service saves a purchase in its database. Then it publishes a purchase created event to tell the other services. That is two writes. A crash between them loses one.

I dealt with the same problem in part 2 of my idempotence series, with a different answer: the outbox. The event is written to an outbox table, in the same PostgreSQL transaction as the purchase. A transaction groups several writes: either all succeed, or none. A worker, a small program, then reads the table, publishes the events and marks each row as sent.

Gilbert offers two variants:

Stream-first (the stream goes first): the service publishes to the hub first, and a downstream service stores the data.
Database-first: the service only writes its table, and that table's CDC publishes the event.

Gilbert's rule for choosing is simple. If users need to see what they just created right away, write to the database first, which requires a database with CDC. Services with no user in front of them, such as gateways to external systems, publish first.

The book never uses the word outbox in the transactional sense. Yet as I read it, database-first is an outbox whose table is the database's own internal log. The gain is concrete: no outbox table, no published column to keep up to date, no worker for you to write. The service reads and writes a single model, its table.

The work does not vanish, though: someone has to run the CDC. With PostgreSQL, a tool like Debezium reads the internal log, and it is one more service to keep running. Gilbert instead relies on the CDC of a database managed by the cloud provider, such as DynamoDB Streams.

One detail from the book is worth borrowing. The trigger first publishes each change as it is, with the row before and the row after. Only then does a rule publish a clean domain event, for example thing-submitted when the status field becomes submitted. The raw changes serve as an audit trail and help a young subsystem discover its true domain events.

Idempotence and ordering without a version number

A concrete case, made up for the occasion: the message debit 50 euros arrives twice, because the sender resent it after an outage. If we debit twice, the customer loses 100 euros instead of 50. An operation is idempotent when playing it ten times gives the same result as playing it once. Making the debit idempotent means recognizing the second message and ignoring it.

Without an event store, consumers lose the safety net I know well: the per-aggregate version. A consumer is a service that reads events to update its own view. Yet it receives duplicates, and out-of-order events. Gilbert accepts this from the start: "exactly-once delivery of messages is unrealistic."

Duplicates have three sources:

a client request resent after a timeout;
a batch of events that fails halfway and restarts in full;
a resend from the lake.

Disorder comes from a faulty producer, a traffic spike, or an event set aside and resubmitted later.

First line of defense: deterministic identifiers, meaning identifiers recomputed identically on every retry. The simplest reuses the ID of the change provided by CDC, which stays stable across retries. If one change yields two domain events, add a fixed suffix per type.

Second line of defense: inverse optimistic locking. Classic optimistic locking rejects a write when the data changed since it was read, and the user has to reload. On the stream side, Gilbert does the opposite. The row keeps, in a field called the oplock, the timestamp of the last applied event, meaning its date and time. His rule: "we drop incoming events if the timestamp is older or equal to the current oplock value in the datastore."

To picture it: the row's oplock says 2:00 pm. An event stamped 1:59 pm arrives: older, so we ignore it. An event stamped 2:00 pm: already seen, so we ignore it too. An event stamped 2:01 pm: we apply it.

The book adds a refinement. A view that gathers the data of several event types into one row keeps one oplock per type. Otherwise a late order-created would be dropped because of a newer order-submitted, even though it carries fields the other one lacks. Here it is in Go and PostgreSQL:

// Example written for this article, not taken from the book (its code is JavaScript on DynamoDB).
// An "order" view joined from two event types, received in any order.
// One oplock per type: created_at_ms and submitted_at_ms (business timestamp in milliseconds).

const applyCreated = `
INSERT INTO order_view (id, customer, created_at_ms)
VALUES ($1, $2, $3)
ON CONFLICT (id) DO UPDATE
   SET customer = EXCLUDED.customer, created_at_ms = EXCLUDED.created_at_ms
 WHERE order_view.created_at_ms IS NULL
    OR order_view.created_at_ms < EXCLUDED.created_at_ms`

const applySubmitted = `
INSERT INTO order_view (id, status, submitted_at_ms)
VALUES ($1, 'submitted', $2)
ON CONFLICT (id) DO UPDATE
   SET status = 'submitted', submitted_at_ms = EXCLUDED.submitted_at_ms
 WHERE order_view.submitted_at_ms IS NULL
    OR order_view.submitted_at_ms < EXCLUDED.submitted_at_ms`

func (v *OrderView) Apply(ctx context.Context, e Event) error {
    var res sql.Result
    var err error
    switch e.Type {
    case "order-created":
        res, err = v.db.ExecContext(ctx, applyCreated, e.EntityID, e.Customer, e.Timestamp)
    case "order-submitted":
        res, err = v.db.ExecContext(ctx, applySubmitted, e.EntityID, e.Timestamp)
    default:
        return nil // type this view does not care about
    }
    if err != nil {
        return fmt.Errorf("apply %s: %w", e.ID, err)
    }
    if n, _ := res.RowsAffected(); n == 0 {
        v.dropped.Add(1) // duplicate or late event: dropped, but counted
    }
    return nil
}

Each query is an upsert: it creates the row if it does not exist, otherwise it updates it. In that case, it only does so if the received timestamp is newer. The first event received creates the row's skeleton, and the next ones fill it in, in any order. The dropped counter follows the book's advice: measure what you ignore, so you notice the day you ignore too much.

Rereading my own idempotence article, I found the same idea without the name. My balance projection did an upsert with WHERE account_balances.last_event_id < EXCLUDED.last_event_id, so an old event would not overwrite a newer state. That is an inverse oplock: the write is ignored if it is older than the row. Only the compared value differs. In my case, it is an event's position in an ordered event store. In Gilbert's, it is the time the user acted.

What CDC costs

Gilbert's model takes plumbing away from the writing service. The bill lands on the consumers, and it comes down to three cases, on top of running the CDC itself, as seen above.

Two events in the same millisecond

In the book's event format, the timestamp is a number of milliseconds, taken when the user acted. Picture two address changes on the same purchase, both stamped 1574100988321. The second one arrives with a timestamp equal to the row's oplock. It is dropped as a duplicate, and the view keeps the old address until the next change.

The book does not cover this case in its inverse oplock section. It comes close elsewhere, when choosing how to sort events. The timestamp is "frequently the best option, so long as it does not result in duplicates."

One way out comes from me, not from the book. The service that writes, upstream of the hub, already has a classic optimistic lock for its own writes: a field used to refuse a stale write. Gilbert says it can hold "a timestamp or counter." Pick the counter, publish it in the event, and compare on it downstream. You get a version per entity back, without an event store.

The late event after a snapshot

A snapshot is an already computed state that you reuse as a starting point instead of recomputing from the beginning. Gilbert does keep computations over history, but he moves them out of the writing service.

An intermediate service, which he calls a control service, collects deposits and withdrawals, computes the balance and publishes a balance event. That event acts as a snapshot: the next calculation starts from the latest published balance and adds the newer events. His example shows the same deposits and withdrawals received in two different orders, with the same final balance of 170.

The book admits the limit. Its example "ignores edge cases such as when an older detailed event arrives after a snapshot," and the author leaves them "as a thought exercise." By detailed event, he means a deposit or a withdrawal, as opposed to the balance. The code assumes older events are "already counted in the latest snapshot."

As I read it, here is what can happen:

the control service publishes a balance, so a snapshot;
a deposit older than that snapshot arrives after it;
the next calculation starts from the snapshot and never sees that deposit.

Gilbert offers a lead without coding it: each snapshot lists the IDs of the events that produced it. You can then check afterwards and emit an adjustment.

In an event store, this case does not exist for the aggregate. The version constraint forces writes to go one after the other. A snapshot taken at version 100 is completed with events from version 101 onward. None of them arrives late. Disorder only affects projections, where the conditional upsert is enough.

Soft deletes and row expiry

In a system where events arrive late, a DELETE breaks idempotence. If an updated event arrives after the deleted one, the row comes back to life. So Gilbert deletes logically, without erasing the row. The command sets deleted = true. CDC sees a plain update, and the trigger publishes thing-deleted. Queries filter on the deleted field.

Consumers stay lean thanks to a TTL: a time to live set on each row, computed from the event's timestamp. The book suggests 33 days by default. The TTL has a side effect worth knowing: it bounds how long a duplicate is recognized. An event resent after expiry is processed a second time. Hence the book's warning to "ensure that ttl is not too short."

Resending means sending old events again to a consumer that needs to catch up. You never republish them to the hub: Gilbert notes that "this would broadcast the events to all consumers." You send them from the lake to that single consumer. It must therefore absorb duplicates and disorder, which brings us back to the inverse oplock.

Event store or CDC: how to choose

The two approaches answer different needs. My criterion: what does the command need to decide, the history or the present state? Deciding means, for example, accepting or refusing a withdrawal.

The event store, when the decision depends on the aggregate's history. A balance that must never go below zero is one case. Another: a state machine, meaning steps to go through in an exact order. The per-aggregate version gives you that order for free, and the computation of the state stays inside the service.
Current state with CDC, when the decision only looks at the present state. History then mostly serves audit, resending and bootstrapping new services. That covers most data-entry screens.
Both in the same system, which is what Gilbert does himself. At the edges of the system, ordinary tables with CDC. In the middle, services that compute over history. His event sourcing does not disappear, it moves.

One condition comes before everything else: without a database that exposes its internal log, database-first does not exist. You are left with stream-first, or a hand-written outbox.

Conclusion

Gilbert tells the story of his first cloud e-commerce system, kept in sync by streams and functions. Here is his admission: "I didn't realize it at the time, but I was implementing the Event Sourcing and CQRS patterns." I took the opposite path: I was writing an inverse oplock in a WHERE clause without knowing the term existed.

So the mechanisms of the two schools look more alike than people say: outbox or CDC, conditional upsert, stable identifiers. The book taught me I was looking for the difference in the wrong place. It lies not in the tool but in the question the service asks before writing.

Going further. John Gilbert's book has its own book review of Software Architecture Patterns for Serverless Systems. For per-aggregate event sourcing, see my review of Understanding Eventsourcing and the order in which to read Event Sourcing and CQRS books. The basics of HTTP idempotence are in part 1 of the idempotence series.

Shipping a ticket with Claude Code: TDD, proof, review loop

Odilon HUGONNOT — Sat, 31 Oct 2026 09:00:02 +0000

From October 1 to 7, the same code change was reviewed fifteen times in a row by an AI. Each review is called a round.

In round 7, the AI found nothing to say: the change held only nine changed lines. In round 8, it held 400, and the AI started hunting for defects again. Nobody had set a maximum number of rounds.

That is the trap: "zero defects" does not mean "done", because the code keeps changing. If nobody writes down when to stop, the review goes on as long as the code changes.

This review AI is a skill of Claude Code. Claude is Anthropic's AI, and Claude Code is the programming assistant you drive from the keyboard. A skill is a folder of instructions that the assistant loads for a type of task. Mine is called senior-review.

Since then, every step of my work has a stop rule you can check. I tell it through a real case, shipped on October 27.

That real case is a ticket: a written request, such as a bug to fix, delivered as one reviewed change. A second skill, ticket, chains five steps for me: plan, code, proof, review, wrap-up.

Each step has an owner. It is often an agent, an instance of Claude launched for one precise task. I describe the method, not the skill file.

One model per step, because none of them reviews itself

Planning, coding and reviewing are three different jobs. I give them to three different agents. Each runs on a model of the Claude family: Fable, Sonnet or Opus.

A comparison: the architect, the mason and the building inspector. The mason does not sign off on his own wall. I explained in another article why the reviewer must never be the author.

The plan lists the steps, the files to touch and the tests to write first. I launch everything with one sentence: "plan fable execution sonnet /senior-review opus". The ticket skill reads it word by word.

I write

What the skill launches

"plan fable"

a Fable agent writes the plan

"execution sonnet"

an executing agent, on Sonnet, writes the code

"/senior-review opus in a loop"

Opus reviewers, round after round

If I say nothing more, the skill applies that same trio.

This choice of models is a setting, not a measurement. The diagram shows the five steps and their owner.

Chain of a ticket in five steps. 1, plan by Fable. 2, execution by Sonnet: the test fails, then passes. 3, proof by me: I undo the fix and look. 4, review by Opus, stop when a round changes nothing, five rounds at most. 5, wrap-up approved by me. An important defect sends the work back from the review to the execution. 1. Plan model: Fable ✓ steps, files, tests to write 2. Execution model: Sonnet ✓ the test fails, then passes 3. Proof me, live ✓ I undo the fix and look 4. Review model: Opus ✓ stop if nothing changes, 5 rounds max 5. Wrap-up approved by me ✓ 3 rules at most, to approve ✗ important defect: back to step 2

Each step has its owner and a result you can check. In red, the way back when the review finds an important defect.

The test turns red before the first line of code

My contrast checker tells whether a text stays readable on its background, for example grey on white. Contrast measures the gap in lightness between the two.

It had a defect. A link pasted into a mockup review, where people comment on the drawing of a screen, reopened the tool with its usual colors. The real ticket is called "Share a contrast check through a link".

The two colors are written at the end of the URL, the address of the page: ?fg=767676&bg=ffffff. For this grey on white, the tool must show the contrast ratio, the number it computes: 4.54.

The ticket adds two requirements. Changing a color updates the URL without reloading the page, with history.replaceState. An invalid value such as ?fg=zzz brings back the usual colors, with no error.

A test is a small program that checks that the code does what you expect. TDD means writing that test before the code. It fails first (red), then passes (green) once the code arrives.

My test is e2e, end to end: a real browser opens the real page, like a user. It uses Playwright, a tool that drives a browser from code.

Real result of the first run, before any code (excerpt; the test titles are in French):

✘  5 […] › 1. ?fg=767676&bg=ffffff : couleurs du lien appliquées, AA normal réussi (8.9s)
    Expected: "767676"
    Received: "1A1A2E"
  5 failed
  2 passed (10.9s)

The message names the defect: the field shows 1A1A2E, the usual color, instead of 767676. The test fails for the right reason. It computes the expected ratio itself with the official formula (WCAG).

Two tests already passed. One checks my own ratio calculation. The other is the ?fg=zzz case, which worked even without the feature. It guards against a future break, but proves nothing about reading the URL.

A Sonnet agent then writes the code. It starts from a brief, written instructions that inherit nothing from my session: the plan, the allowed files, the bans.

Undo the fix: if the test stays green, it proved nothing

A green test proves nothing until you have seen it fail without the fix. Reverse mutation means undoing the fix. The test must then go back to red.

If it stays green, it was not checking what you thought. Mutation testing automates this move across a whole project: see my article on mutation testing.

Proof by undoing, in three steps. 1, without the fix, the test is red. 2, with the fix, the test is green. 3, fix reverted, the test must turn red again. If red, the test proves something. If green, the test proved nothing. 1. Without the fix the test fails for the right reason ✗ red 2. With the fix the code is written the test passes ✓ green 3. Fix reverted we remove the fix it must turn red ✗ red ✓ Red: the test proves it the fix works ✗ Green: it proved nothing even without the fix

The third step is the one people skip: undo the fix and watch the test turn red.

Before the review, on the first draft of the code, I undid the fix in four ways. I removed the reading of the URL, then its writing. I replaced replaceState with pushState, its cousin that adds a page to the history. I removed the check on values.

Each attempt made at least one test fail. More followed during the review, and one made nothing turn red: that is the story of round 2. Real results of two attempts (kept verbatim, in French):

=== M1 : lecture de l'URL retirée
  4 failed
  3 passed (10.7s)
=== M2 : replaceState retiré
  1 failed
  6 passed (2.2s)

Then I look at the page like a user, with screenshots at three sizes: mobile (390×844), tablet (820×1180) and desktop (1280×900).

My proof report separates three things, one line each:

the code is right: the tests seen red then green, and the reverse mutation;
the effect is visible: a screenshot read the way the user would see it;
what stays unchecked: what the test really went through, and what it did not touch.

On this ticket, two areas have no test. No test clicks the "Apply" buttons. The language switch links lose the colors of the link.

Before trusting the agent, I rerun the project's automatic checks myself, then I read each screenshot. Below, the same link at the three sizes, taken on the live site.

The same link at the skill's three sizes. On mobile the page is small: you look at the layout, not the text.

All three screenshots show 4.54, with no horizontal scroll. The method for catching display bugs with Playwright is in my article on responsive testing.

The review runs until a round changes nothing

On this ticket, two reviewers worked in parallel in rounds 1 and 2, one on the code and one on the tests. Only one remained in round 3. All ran on Opus, five reviewers in total.

They are "blind": they do not know who wrote the code or what the author thinks of it.

A reviewer flags defects. Each flagged defect is called a finding, blocking if it prevents adding the code to the project, important if it must be fixed before moving on.

The rounds form a loop: review, fix, review again. It stops when a round meets three conditions:

no blocking finding;
no important finding still open;
no code change.

A change that only touches tests counts as a change: another round is due. At the fifth round without a conclusion, the loop stops and hands control back to me.

My reviewers were only meant to read, and I had written it in their instructions. In fact they had every right, Edit and Write included: they could modify files. An instruction is only a request. Only the agent's definition really forbids an action.

Round 1: a real bug, three test gaps

In the tool, you pick a color with a picker. When you move it, it signals an event at every small movement. My code rewrote the URL at every event.

Now WebKit, Safari's engine, accepts at most 100 rewrites in 10 seconds. Beyond that it returns an error, SecurityError. The URL then stays frozen on an old color, while the displayed ratio is correct.

Playwright can launch that engine, and a reviewer reproduced the bug with it. Chromium, Chrome's engine, ends on the right URL. The fix: write the URL 250 ms after the last event, not at each one.

The reviewer also found three gaps in my tests:

swapping the colors and typing the background were not checked in the URL;
the English version of the tool was not checked after the URL rewrite;
the reading of the background color had no explicit check.

Round 2: my test did not watch the delay

I replaced 250 ms with 0 in the code. All nine tests stayed green, so none really watched the delay. The reason: my test sent the 150 events at once, too fast for the delay to change anything.

The test now spaces its events by 16 ms, the pace of a drag. Real results, excerpt (kept verbatim, in French):

# M-a : debounce retiré (setTimeout 250 -> 0)
  9 passed (10.3s)
# Tour 2, M-c : setTimeout 0 avec rafale espacée de 16 ms
  1 failed   8 passed
# Tour 2, code restauré
  9 passed (5.2s)

Two more remarks. Leaving the page before 250 ms loses the last color of the URL: rejected, no human copies a URL in 250 ms. The ticket announced "4.54:1", the tool shows "4.54": a badly worded criterion, not a bug.

Round 3: nothing to report

A last reviewer raised nothing. The delay test passed 90 times out of 90, even under load, with 4 CPU loops at 100%. The code did not change: the loop stops.

Total: 9 findings. 5 fixed, 1 rejected, 3 deferred, meaning put off until later. Those 3 are only two topics. The first is the rewrite of the bare URL at load: seen by two rounds, so counted twice. The second is the wording gap in the ticket, which is not a code defect.

My journal said everything was fine

My review skill keeps a journal: one line per review, with the findings raised, confirmed and dropped. As of October 27, it showed 981 confirmed findings. But "confirmed" only meant "passed my filter". The journal did not say whether I then fixed the defect or refused to fix it. I tell that discovery, figures and cost per finding included, in my article on AI code review.

The journal now records the outcome of each confirmed finding: fixed, rejected or deferred. This ticket is the first review to carry it: 5 fixed, 1 rejected and 3 filed as "deferred". That is one real deferral seen twice, plus the wording gap in the ticket.

So I announce no acceptance rate: over a single review, it would mean nothing. Nor can I say that Fable plans better, or that Opus reviews better than Sonnet. It is my setting, not a result.

Last step, the wrap-up. I reread my corrections from the session ("did you test it?", "are you sure?"): each one reveals a missing rule. The skill proposes at most three to me, with their destination. Nothing is written without my approval. This ticket produced one: a test that proves a delay spaces its events.

Conclusion

Every rule in this chain was born from a failure observed from the outside. A loop that ran fifteen times. "Read-only" reviewers who could write. A journal that graded itself with its own filter.

I have no before and after measurement: I promise you neither time savings nor quality gains. I offer two moves for tomorrow. Write the stop condition of your loop before you launch it. Undo your fix before you trust your test.

Single Agent vs Multi-Agent: 11 AI Agents, $50 and Zero Lines of Code

Odilon HUGONNOT — Fri, 30 Oct 2026 09:00:02 +0000

An AI agent is a program that makes a language model act in a loop. It reads files, runs commands and writes edits, until a task is done. The language model is the program that reads and writes text behind an AI assistant. A team of agents brings several together, sharing the work.

A course reports an experiment: the same task, writing a program, given to four teams of agents organized differently. All had the same model. One of them, called a pipeline, lined up eleven agents in a chain: each took over the previous one's work. It had $50 to pay for the model's usage.

The pipeline didn't crash. It worked: plans, breakdowns, architectures. It went through five planning stages, meaning preparing the work without writing it yet. Then it ran out of money before writing its first function. Final output: zero lines of code.

A single agent, with the same model, got the top score: 28 out of 28.

The experiment comes from Beyond Code, a 2026 video course by Jeremy McEntire. I read it as a transcript for my book review, my critical summary of the course.

It bothered me, because I lived the opposite last spring. I had 26 AI agents work together on my own code. They found a real security vulnerability, a flaw an attacker can exploit.

So, single agent or multi-agent (a team)? Both results are true, because they don't measure the same kind of work.

Four agent setups, one task, 28 points to 0

The course's task: build the server side (the backend) of a vacation rental booking platform. It is split into seven microservices, seven small independent programs that talk to each other over the network. Every setup got the same language model and the same task. The pipeline had a $50 budget. The only thing that changed was how the work was organized:

Single agent: one agent does everything, start to finish. Score: 28 out of 28.
Hierarchy: a coordinator hands work to specialists. Score: 18. The coordinator ended up coding everything itself. Two limits held it back. First, a cap on steps per agent: past that number, it has to stop. Second, a smaller context: what the agent can keep in view at once. A comparison: a work table of limited size.
Swarm: eight agents work at the same time on shared files, with no leader. They share tasks through a board where each one writes them down. Score: 9. The swarm delivered five services out of seven, 152 files, and service boundaries that were all incompatible. A boundary is the way two services exchange data. In the run, half the agents named fields in snake_case, like customer_name. The other half wrote camelCase, like customerName. The services no longer understood each other.
Pipeline: eleven specialized agents in sequence, one per stage. The stages: diagnosis, decomposition, architecture, execution, testing, verification, review, and so on. A review is an agent that rereads another agent's work and can reject it. Score: 0.

The course puts it bluntly: "The pipeline spent its entire budget on five planning stages. It produced zero lines of implementation code." In other words, the $50 went to planning before a single line of code existed. The single agent cost roughly as much as the pipeline's planning alone, and it scored 28 out of 28.

Where do the 28 points come from? From a rubric of seven criteria, which checks whether the program follows a specification written in advance. The course names only four: endpoints, data models, API contract consistency and error handling. Endpoints are the addresses a service offers. Data models describe the shape of the information, such as a customer or a booking. An API contract is the written description of what a service promises the others: endpoints, formats, errors. The course doesn't detail how the 28 points are split across these criteria.

This rubric doesn't measure quality. The course admits it: it is a proxy. The protocol isn't published, the text doesn't name the model, and it doesn't say how many times each setup was run. Nobody is named behind "the researchers." The author also warns that these systems don't represent the state of the art. So 28 to 0 is an experiment story, not a measurement. I read it as an illustration of a mechanism, not as proof.

Part of the mechanism can be computed. When every agent has to agree with all the others, each pair opens a communication channel. With 5 agents, that makes 10 channels. With the swarm's 8 agents, 28. With 10 agents, 45. The formula is n(n-1)/2, for n agents. Coordination cost grows much faster than the agent count.

The pipeline, though, is a chain: each agent waits for the previous one's work. The course also explains its failure through another mechanism. At each handoff, an agent passes on only a summary of its work. What is lost never comes back. In another run, with a similar architecture on a more complex version of the task, the review stage rejected seven pieces of work out of eight. Four of those rejections cited no actual problem, only subjective objections, 15 to 23 each.

The course also reports a fifth setup, which wrote the contracts before the code. Its contracts were typed, meaning written in a strict format, and tests replaced reviews. It avoided the subjective rejections. In return, it burned its budget on ever more elaborate contracts: 607 lines of JSON for a four-function module. A contract, too, can become the goal instead of the means.

My 26 audit agents found a real vulnerability

Last spring, I had 26 agents audit ShareBox, one of my side projects: a PHP site that streams videos. An audit is a methodical read of the code to look for defects. The full story is in my 26-agent AI code audit.

Eleven reader agents each read a part of the code, in parallel. Two agents linked their reports. Twelve agents each scored a single axis: security, performance, architecture, tests. A last agent, the judge, was told to be strict. It reread all the scores to bring down the too-kind ones. That makes 11 + 2 + 12 + 1 = 26 agents.

The agents found a code injection: a value that ends up executed as program code. It hid in the application's startup script. That application runs in Docker, a tool that packs a program so it runs anywhere. The injection sat right under a comment, a note left in the code, claiming to prevent it. A real vulnerability, which I checked in the source before believing it.

My audit and the course's pipeline don't contradict each other. My audit also chained stages: read, link, score, judge. The difference lies in what each agent had to produce. My reader agents read. Each had its own slice, and none waited for another's work to start. Their output, a list of findings, didn't have to fit with the neighbor's. The final judge scored harshly, and I checked every flaw in the source.

The course's pipeline, on the other hand, was building. Each agent waited for the previous one's output, and everything had to come together into one working system.

Here's a rule I draw from it, and it's mine: the course doesn't state it this way. Parallel agents work on independent slices to read, when every finding can then be checked in the code. A chain of agents that must coordinate to build something burns itself out preparing.

On the left, three independent readers each read one slice of the code, then a strict judge scores their findings. Reading in parallel readers never talk to each other The code, cut into slices Reader 1 Reader 2 Reader 3 Strict judge Findings checked in the code

On the right, a chain of planning stages where each agent waits for the previous one, and the coding stage is never reached. Building in a chain 11 agents, each waits for the last Planning 1 Planning 2 Planning 3 Planning 4 and 5 Code: never reached $50 spent on planning

Reading: each reader works alone and every finding gets checked in the code (my audit had eleven readers). Building: each agent waits for the previous one, and the budget runs out before any code.

The course's swarm confirms the rule from the other side. Its eight agents did work in parallel, but they were building, with no contract between them. The course sums it up: "Parallel decomposition without contracts produced volume without coherence." Plainly put, splitting the work in parallel without contracts produced lots of files, not a system whose parts fit together.

Feeling faster with AI: the METR study and the perception gap

The course cites a study you may have come across. It serves my topic: with AI, you can feel faster without being faster. METR is a nonprofit research organization that evaluates AI models. From February to June 2025, it ran a randomized controlled trial: chance decides, task by task, whether AI is allowed.

Sixteen experienced developers of open source projects, whose code is public, worked through 246 real tasks. Each task was randomly assigned: AI allowed or not. The main tool was Cursor Pro (a code editor with built-in AI), with Claude 3.5 and 3.7 Sonnet.

Before the study, the developers expected AI to make them 24% faster. Afterwards, they believed it had made them 20% faster.

On the clock, they took 19% longer. So they were slower, while feeling faster. That's a 39-point gap between perception and measurement (20 + 19). Sources: METR's write-up and the paper by Becker, Rush, Barnes and Rein on arXiv.

The limits are real. Sixteen people is a small sample. The tasks were on mature projects these developers knew inside out, with five years of experience on average. METR itself framed the result as a snapshot of early-2025 AI.

In February 2026, METR published a follow-up and now flags the 2025 study as out of date. It covers 57 developers. The new margins of uncertainty (confidence intervals) include zero: they cover both a gain and a loss. So no effect is demonstrated, either way.

METR calls these numbers unreliable: some developers now refused to work without AI. Its unquantified view is that early-2026 tools help more than the 2025 ones did.

So the 2025 number no longer holds, and no new one has replaced it. Still, nothing suggests the gap between perception and measurement has gone away. The eleven-agent pipeline is its extreme form: plenty of visible activity, plans, documents, and nothing that runs.

Mechanical verification over an opinion

The course's practical lesson fits in two sentences: "Advisory review asks whether someone did the work. Mechanical verification asks whether the work succeeded." An opinion asks whether someone did the job. A mechanical check asks whether the job works.

An opinion, even an agent's, says the work looks done. A test, a compiler or a script answers yes or no, and gives the same answer every time. The course calls it a mechanical gate: an automated check that blocks the next step until it passes.

In my setup, with Claude Code (Anthropic's coding agent, which you drive from a terminal), the idea took four forms. Each one answers a real problem.

A hook that proofreads prose. A hook is a script Claude Code runs on its own before or after an action. After every edit to a book review, mine runs a prose check. It counts sentences over 25 words and spots semicolons and a list of banned words. It gives no opinion: it answers yes or no. The agent gets the feedback and fixes it before moving on, without me repeating the rule. More examples in my article on Claude Code hooks.
A written instruction that loses to a default setting. My instruction file said: never put an AI signature in a commit (a saved set of changes in Git, the tool that keeps a project's history). By default, the tool adds a Co-Authored-By line naming Claude. The default won. The line landed in the shared history, the list of changes saved by everyone, and I had to rewrite that history. Since then, a hook rejects any commit or pull request command whose message carries that signature. A pull request is a request to merge your work into the main project. An instruction hopes to be read. A hook runs every time.
A colon that switches off a skill. A skill is an instruction file Claude Code loads when its description matches the request. That description is written at the top of the file (the header), in YAML: a configuration format where the colon separates a name from its value. A colon followed by a space, in unquoted text, makes the header invalid. Claude Code then loads an empty header without any message, and the skill stops triggering. I saw it on three skills in a single day. Only a script that reads every header one by one catches it.
An AI judge that copies the score it's shown. Each of my book reviews carries a score out of 10. I had agent judges re-score four of them, with the published score as reference. First test: the judge first saw a proposed score, deliberately off by 1.5 points. Its score landed 0.95 points from the published one on average. Second test: the judge saw no score, but got a rubric and book-to-book comparisons. It landed 0.4 points away. Four books is a small sample. The published scores used as reference were themselves set by an AI, and I only adjusted a few of them. The test mostly measures resistance to anchoring: the first number you see pulls the next ones toward it. When you still need an opinion, remove whatever biases it (more in the article on why the reviewer is never the author).

Single agent or agent team: the rule I follow

The course offers a split I find sound:

A single agent as long as the task fits in its context (what it can keep in view at once).
Coordination through shared traces for loosely coupled work, meaning work whose parts depend little on each other. This requires mechanical checks that reject what doesn't fit. The agents read these traces and extend them. The traces are the files, the tests and continuous integration, a tool that reruns the tests on every push of code.
A hierarchy only for ambiguous trade-offs, when several choices are defensible.

My current settings point the same way. These days, never more than two or three agents in parallel. One agent for a batch of small, similar tasks, rather than one agent per task. These rules came from a subscription that caps usage (a quota) and that I didn't want to burn through. They help quality too: each agent gets its slice, and nothing else.

That last point ties into another idea from the course: "The well-written tutorial is a trap." Well-written but off-topic text, placed in the agent's context, hurts the model more than random noise. Noise is easy to spot and ignore. A clean tutorial on the wrong subject, the model follows.

The course's example comes from AWS, Amazon's cloud host. A tutorial on the pricing of one of its services (EC2) is pasted in for a question about the pricing of another (Lambda). The two topics look alike, so the model believes it has the right page. Handing every agent the whole project code "just in case" sets exactly that trap.

To go further, two other articles on the blog summarize books on the subject. The first: what two books say about parallel agents. The second: four books on Claude Code.

Conclusion

The eleven-agent pipeline wasn't short on intelligence. Each agent produced a document for the next one, and none could fail a test, because there was nothing to run. My audit worked for the opposite reason: every finding could be checked in the code, and I checked it before believing it.

Adding an agent mostly adds coordination. When an agent team stalls, it rarely needs one more agent. It almost always needs something that can say no without arguing: a test, a hook, a script.

BOLA, the #1 API Vulnerability: a One-Line Fix and an A-B Test in Go and PHP

Odilon HUGONNOT — Thu, 29 Oct 2026 09:00:02 +0000

An invoicing application exposes an API: web addresses, or URLs, that other programs call to read or change invoices.

Bob is logged in with his account. He asks for his invoice at the URL GET /invoices/7f3b, where GET means "read". The API answers with his invoice and the response code 200, which means "OK".

Bob changes one character in the URL and asks for GET /invoices/7f3a. The API answers 200 again. This time, the invoice is Alice's.

This vulnerability is called BOLA, for Broken Object Level Authorization: authorization is broken at the level of each object, here each invoice. OWASP, a foundation that ranks web security flaws, puts it at the top of its 2023 ranking of API vulnerabilities.

The fix is one line: the condition that checks the invoice belongs to the logged-in user. A single line is also very easy to forget.

To check that it is there, you write a test: a small program that replays a scene on its own. It shows green if all is well, red otherwise. A course on API security, Defending APIs, describes such a test without ever writing it. Here are the fix and the test, in Go and then in Symfony.

BOLA (or IDOR): a vulnerability that fits in a URL

The mechanism never changes. The URL carries the invoice's identifier, here 7f3a. The server reads that identifier and fetches the invoice from the database, where the application stores its invoices. It never asks who the invoice belongs to.

The confusion comes from two similar words:

authentication says who you are. With every request, Bob sends a token, a kind of badge received at login. So the API knows it is Bob;
authorization says what you are allowed to touch. That is what is missing here.

An API that checks the first and forgets the second lets Bob read any invoice whose identifier he knows. You will also see this same flaw called IDOR.

The usual reflex is to make identifiers impossible to guess, with long identifiers drawn at random. OWASP recommends it, on top of a real check. Colin Domoney's course calls this kind of camouflage insufficient:

"None of these are sufficient to prevent BOLA and will merely serve to delay an attacker."

None of these measures is enough; they only slow the attacker down. An identifier can end up elsewhere: in a shared link, in a screenshot. The day Bob knows it, only the ownership check stops him from reading the invoice.

Why a single-account scanner misses it: the A-B test

A vulnerability scanner is a tool that floods an API with requests and flags suspicious answers. A scanner that uses a single account sends GET /invoices/7f3a and gets 200. The scanner is delighted: it has no idea the invoice is Alice's. To see the flaw, you need two accounts and a memory of who created what.

The course says so plainly:

"Unfortunately, most current test frameworks are not designed to perform the multistage A-B testing required to identify a broken object-level vulnerability."

Most testing tools are not designed to chain the requests of two different users. Yet that chain is what exposes the flaw. The course calls it the A-B test:

"First, create a resource as user A. Then, attempt to access the resource as user B. If you can, then you have an authorization issue."

User A, here Alice, creates an invoice. User B, here Bob, tries to read it. If he succeeds, authorization is broken. The A-B-A variant goes further: B tries to delete the invoice, then A asks for it again to check it is intact.

In this test, Bob must get 404, the response code for "not found". Bob must not even know the invoice exists. A "forbidden" would tell him he hit a real one.

Between the creation and Bob's request, the course has you check that A can reach their invoice. That check looks useless; it is not. The test only looks at the answer Bob gets. A broken API that answers 404 to everyone would give Bob a 404 too. The test would be green for the wrong reason. So Alice asks for her invoice again and must get 200.

The diagram below walks through these steps. It uses two more words: POST means "create", DELETE means "delete".

A-B then A-B-A test: Alice creates an invoice and asks for it again, Bob asks for it and must get 404, Bob tries to delete it, Alice asks again and finds it intact. A 1. Alice creates an invoice POST /invoices → 201, id 7f3a A 2. Alice asks for it again GET /invoices/7f3a → 200 B 3. Bob asks for the same invoice GET /invoices/7f3a 404: test green 200: BOLA, red A-B-A variant B 4. Bob tries to delete it DELETE /invoices/7f3a → 404 A 5. Alice asks for it again GET /invoices/7f3a → 200, intact

Diagram made for this article: the A-B test (steps 1 to 3), then the A-B-A variant (steps 4 and 5).

The 201 in step 1 is the response code for "created". The course advises writing these tests yourself. It gives no code example: the Go and Symfony sections below fill that gap.

The course's example guards deletion, not the invoice

The course's own authorization example shows the most common mistake. The author hands decisions to a rules engine, Casbin: a tool that reads rules written in a file and answers "yes" or "no".

The course's example handles "items", which play the part of our invoices here. Its rules say alice may delete items and bob may only read them.

The rule covers /items/*: it answers for all items at once. So the engine answers the question "may this user delete invoices?". It never asks the real BOLA question: "may this user delete this particular invoice?".

The line that deletes shows it:

items_dao.delete_item(item_id)

This line only receives the item's identifier, never its owner. So nothing checks who it belongs to.

As I read it, as soon as two users both have the right to delete, each can erase the other's invoices. The A-B-A test would show it on the first try.

When the engine refuses, the example answers with the response code 401. It means "I don't know who you are, log in". But the server knows exactly who bob is. The course's 401 is simply wrong: it sends you looking for a login problem, when what is missing is the right.

404 rather than 403

The response code 403 means "I know who you are, and the answer is no". For someone else's invoice, 403 and 404 each have an argument:

404 hides the invoice's existence. Bob cannot tell "it does not exist" from "it belongs to someone else". So he cannot try identifiers one by one to list the ones that exist;
403 helps a developer hunting for the cause of a bug: they see right away that authorization said no.

My recommendation: 404 for any data that belongs to a user, such as an invoice or an account. The developer can find the cause of the refusal in the server's logs, the files where it records every request. A list of valid identifiers, on the other hand, cannot be taken back once Bob has built it. So both tests, Go and Symfony, require 404.

RFC 9110, the document that defines HTTP, allows for this: a server that wants to hide the existence of a forbidden resource may answer 404. One limit: if Bob may read the invoice but not delete it, he already knows it exists. Refusing the delete is then a 403.

In Go: filter by owner, then prove it

The fix: the owner goes in the lookup

The principle: never fetch an invoice by its identifier alone. The lookup also carries the owner.

Databases speak SQL. There the condition reads WHERE id = $1 AND owner_id = $2. It means "the invoice with this identifier AND this owner".

The example keeps invoices in memory, without a real database. The condition is the same:

// FindForOwner is equivalent to:
//   SELECT id, owner_id, amount FROM invoices WHERE id = $1 AND owner_id = $2
func (s *Store) FindForOwner(id, ownerID string) (Invoice, error) {
    s.mu.RLock()
    defer s.mu.RUnlock()
    inv, ok := s.byID[id]
    if !ok || inv.OwnerID != ownerID { // the one-line fix
        return Invoice{}, ErrNotFound
    }
    return inv, nil
}

The fix reads like this. If the invoice does not exist, OR if it belongs to someone else, answer "not found". Both cases get exactly the same answer. A twin function, DeleteForOwner, carries the same condition for deletion.

The code that answers a URL is called a route.

Before each route runs a middleware: a filter that reads the token and finds the logged-in user. The route gets that user from the middleware, never from a value Bob typed in the URL. Otherwise Bob would only have to write "I am Alice".

mux.HandleFunc("GET /invoices/{id}", func(w http.ResponseWriter, r *http.Request) {
    inv, err := store.FindForOwner(r.PathValue("id"), currentUser(r))
    if errors.Is(err, ErrNotFound) {
        http.NotFound(w, r) // 404: does not reveal whether the invoice exists
        return
    }
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(inv)
})

Putting the owner in the lookup beats checking afterwards. Checking afterwards means loading the invoice, then comparing its owner with the logged-in user. If someone forgets the comparison in a new route, the invoice is sent to Bob. With FindForOwner, the function requires an owner: a call without one does not even compile; Go refuses to build the program. The code offers no other way to look up an invoice anyway. The order of middlewares, when there are several, is covered in my article on Go middleware in production.

The A-B test with httptest

The test starts the whole API, middleware included, with Go's httptest package. It sends three requests, the ones in the diagram:

Alice creates an invoice and gets its identifier;
Alice asks for her invoice again and must get 200;
Bob asks for the same invoice and must get 404. If he gets 200, the test fails and names the flaw.

Two small helper functions, not shown, keep the code short. createAsAlice does step 1. call sends a request with Alice's token (token-alice) or Bob's (token-bob):

func TestBOLA_AB_Read(t *testing.T) {
    srv := newAPI()
    defer srv.Close()
    id := createAsAlice(t, srv) // A: POST /invoices, expects 201

    // Alice reads her invoice back. Without this step, an API that
    // answers 404 to everyone would pass the test for the wrong reason.
    if got := call(t, srv, "GET", "/invoices/"+id, "token-alice", "").StatusCode; got != http.StatusOK {
        t.Fatalf("A reads: status %d, want 200", got)
    }

    // B: Bob asks for Alice's invoice.
    got := call(t, srv, "GET", "/invoices/"+id, "token-bob", "").StatusCode
    if got == http.StatusOK {
        t.Fatalf("BOLA: Bob reads Alice's invoice (200)")
    }
    if got != http.StatusNotFound {
        t.Fatalf("B reads: status %d, want 404", got)
    }
}

A second test plays the A-B-A variant: Bob attempts a delete, Alice asks for her invoice again and must still get 200. With the fix in place, both tests are green.

A security test that stays green when you break the code proves nothing. To check it, break the code on purpose: disable the owner condition in FindForOwner and in its twin. Then run the tests again. The following output is therefore intended:

--- FAIL: TestBOLA_AB_Read (0.00s)
    main_test.go:60: BOLA: Bob reads Alice's invoice (200)
--- FAIL: TestBOLA_ABA_Delete (0.00s)
    main_test.go:74: B deletes: status 204, want 404
FAIL

Both tests turn red. The first one names the flaw. In the second, the 204 is the response code for "deleted": Bob really did erase Alice's invoice. Put the condition back, and the tests turn green again. This round trip was run on the article's code.

In Symfony: a Voter and an A-B test

Symfony is a PHP framework, a toolbox for building web applications. It puts the ownership check in a Voter.

A Voter answers yes or no to one question: "may this user read this invoice?". In the code below, everything happens on the final return line:

final class InvoiceVoter extends Voter
{
    public const VIEW = 'INVOICE_VIEW';

    protected function supports(string $attribute, mixed $subject): bool
    {
        return $attribute === self::VIEW && $subject instanceof Invoice;
    }

    protected function voteOnAttribute(string $attribute, mixed $subject, TokenInterface $token, ?Vote $vote = null): bool
    {
        $user = $token->getUser();
        if (!$user instanceof User) {
            return false; // not logged in: deny by default
        }

        /** @var Invoice $subject */
        return $subject->getOwner()?->getId() === $user->getId();
    }
}

That line compares the invoice's owner with the logged-in user. The rest is the wrapping Symfony requires from every Voter, here for Symfony 7.3 or later.

The route calls the Voter through the #[IsGranted] line placed above it. On refusal, Symfony answers 403 by default. The statusCode: 404 option asks for "not found", as in Go:

#[Route('/invoices/{id}', methods: ['GET'])]
#[IsGranted(InvoiceVoter::VIEW, subject: 'invoice', statusCode: 404)]
public function show(Invoice $invoice): JsonResponse
{
    return $this->json(['id' => $invoice->getId(), 'amount_cents' => $invoice->getAmountCents()]);
}

One difference from Go, stated plainly. The Invoice $invoice parameter tells Symfony to fetch the invoice by its identifier first. The Voter rules afterwards.

So this is the check-afterwards approach described above. Its strength: the check lives in a single class, which every route calls by name. To filter at lookup time, you would write your own lookup by identifier and owner.

The test replays the scene of the Go test. WebTestCase boots the application. loginUser logs Alice or Bob in without going through the login form:

public function testBobCannotReadAliceInvoice(): void
{
    $client = static::createClient();
    $users = static::getContainer()->get(UserRepository::class);
    $alice = $users->findOneBy(['email' => 'alice@example.test']);
    $bob = $users->findOneBy(['email' => 'bob@example.test']);

    // A: Alice creates an invoice.
    $client->loginUser($alice);
    $client->jsonRequest('POST', '/invoices', ['amount_cents' => 12000]);
    self::assertResponseStatusCodeSame(201);
    $id = json_decode($client->getResponse()->getContent(), true)['id'];

    // Alice asks for her invoice again: without this step, a 404 everywhere would pass.
    $client->request('GET', '/invoices/'.$id);
    self::assertResponseIsSuccessful();

    // B: Bob asks for the same invoice.
    $client->loginUser($bob);
    $client->request('GET', '/invoices/'.$id);
    self::assertResponseStatusCodeSame(404); // not found: neither 200 nor 403
}

The test assumes Alice and Bob already exist in the test database. It was not run against a real application: only its syntax was checked.

So the proof by red exists only for Go. The creation route is not shown. The A-B-A variant would need a delete route, not written here.

One test per route beats a code review

The temptation: rely on code review, when a colleague reads the changes before they go live. The fix is one line, so the omission is one line too. The colleague reads what is written. A missing line, they do not see.

My article on iterative security audits asked the same question. During an audit, each flaw spotted is called a "finding". The article only kept as a permanent test what answered yes to this question:

Could a future code change, made in good faith by a developer who doesn't know about this finding, reintroduce the vulnerability?

For BOLA, it is yes for every new route that takes an identifier. A developer adds GET /invoices/{id}/pdf, fetches the invoice by its identifier and forgets the owner.

A commit is a saved set of changes to the code. The same article sums it up in two sentences: "The report is read once. The tests run on every commit." OWASP asks for the same thing: tests on authorization, and no change shipped that makes them fail.

In practice:

every route that takes an identifier gets its A-B test;
if that route can modify or delete the invoice, it also gets the A-B-A test.

The test copies from one route to the next; only the URL and the data sent change. The reading notes on Defending APIs cover the rest of the course, and the ones on Web Application Security cover the attacker's side.

Conclusion

As I read it, BOLA stays at the top of the ranking because the flaw leaves no trace in the response. A 200 on Alice's invoice has exactly the same shape as a 200 on Bob's. The difference is not in the response: it is in who asked.

A tool that looks at one response at a time therefore cannot see the flaw. The A-B test asks the question on every commit: who created this invoice, and who is asking for it? Two accounts are enough to ask it.

Testing your code — free interactive course

Odilon HUGONNOT — Wed, 28 Oct 2026 11:00:04 +0000

Testing your code

The angle: tests built to catch the mistakes an AI slips into your code, not just theoretical TDD.

Free interactive course on testing code and TDD in 9 lessons. Write tests that catch AI mistakes: assertions, Arrange-Act-Assert, red-green-refactor, mocks, Vitest, the test pyramid.

Free, interactive, no signup: https://www.web-developpeur.com/en/learning/tests/

Best Event Sourcing and CQRS Books, and the Order to Read Them

Odilon HUGONNOT — Thu, 15 Oct 2026 09:00:02 +0000

Five years ago, CQRS kept coming up in meetings and I had no idea what anyone was talking about. There are plenty of books on the subject. A 2012 Microsoft guide built on Windows Azure, a 650-page brick with Kotlin in it, a 72-page booklet, two DDD books twenty years apart. None of them tells you which one to open first, and the order changes everything.

I read them, I do Event Sourcing every day, and I wrote a reading note for each. Here is the order I would give you, and what you can skip.

Start with DDD, but not with Evans

Event Sourcing and CQRS answer a modeling problem first and a technical one second. Learning Domain-Driven Design by Vlad Khononov (2021, 8.6/10) is the book that put them in their real context for me, instead of presenting them as fashionable gadgets. The chapter worth the price is the appendix, where Khononov tells the story of his own startup, "where we made every DDD mistake possible". You start at "aggregates everywhere" and end at "shared language everywhere".

Eric Evans's blue book (2003, 7/10) is still the foundation. The vocabulary (Bounded Context, Aggregate, Ubiquitous Language) is worth the trip on its own. But every example is 2003 Java, and nobody reads it cover to cover: you open the chapter that matches today's problem. To look up a definition, the DDD Reference (6.4/10) is sixty free pages. It is a glossary, not a course, and only useful if you already know the terrain.

The only one that shows you the bill

Exploring CQRS and Event Sourcing (Microsoft patterns & practices, 2012, 200 pages, 8.2/10) is the oldest book on this list and the most honest. Most introductions sell the benefits and skip the plumbing. This one makes you look at the cost before you commit: twice the time you planned, eventual consistency in the UI, atomic publishing between the store and the bus.

Its chapter on Sagas is worth the read by itself. I have watched the Process Manager versus Saga confusion cause real architecture mistakes: teams building compensating-transaction machinery for a flow that lives entirely inside one bounded context. The guide clears that up in ten pages. The limit you have to accept: the sample code targets Windows Azure in 2012. Read it for the ideas, don't copy it.

The big one, in two halves

Martin Dilger's Understanding Eventsourcing (2024, 650 pages, 7.5/10) has two faces. The first half, Foundations and Modeling, is the clearest thing I have read on Event Sourcing and Event Modeling: the toy-box metaphor, the completeness check, the difference between internal events and Integration Events. I wish I had it five years ago.

The second half is another planet. The Axon implementation, in Kotlin, assumes a Spring/Java ecosystem most web developers don't live in. I'm a PHP/JS dev, and it shows. What still travels: the eight read patterns, the Reservation Pattern, the internal-versus-Integration-event boundary, the Given-When-Then test structure. Read the first half, mine the patterns, and don't feel guilty about the rest.

The small one, for when you join a team

The Little Eventmodeling Book (Dilger, 2025, 72 pages, 6/10) answers seventeen questions to decode Event Modeling before you walk into a workshop. The Information Completeness Check justifies the read on its own: you draw a read model, someone asks "where does this field come from?", and nobody knows. Making that a hard constraint that blocks implementation is a simple, powerful idea.

Two caveats. About 15% of the content is commercial. And Event Modeling is a visual method: in a PDF you lose half the substance. It is enough to follow a team conversation, not to practice alone.

Kleppmann, underneath it all

It is not an Event Sourcing book, but Designing Data-Intensive Applications (2017, 8.4/10) explains why these patterns exist: replication, consistency, failures, the log. It is the best backend engineering book I have read, and the opinion is boring because it is right. A web dev will have a great time in chapters 1 to 8. The Hadoop-era batch processing now reads like history, so skim it.

The order I would follow

Book

Pages

Rating

What it is for

Learning DDD

340

8.6

The frame, before the patterns

Exploring CQRS and ES

200

8.2

Seeing the real cost, understanding Sagas

Understanding Eventsourcing

650

7.5

The first half, then the patterns

Little Eventmodeling Book

Following a team workshop

as needed

Evans + Reference

560 + 59

7 / 6.4

The vocabulary, chapter by chapter

background

Kleppmann

~600

8.4

The ground under the patterns

To go from reading to practice in Go, my articles follow the same path: the practical guide to Event Sourcing and CQRS, then PostgreSQL as an event store and idempotency.

Conclusion

Almost every Event Sourcing book sells the benefits. Only one of these six shows you the bill, and it is the oldest, the least glamorous, the one that talks about Azure in 2012. The rating doesn't follow the date: the best starting point isn't the newest, and the newest isn't even the most useful.

If reading six books in a precise order sounds heavy, it is because each one speaks to a different level of the craft. I stitched the marrow of dozens of tech books into a single thread, from bits and Big O up to the AI era: that is The Developer's Great Book. DDD and architecture have their place in it, at the level where they become useful.

📚 Other comparisons. Best Clean Code and Refactoring Books, and the Order to Read Them and Best DevOps and Engineering Team Books, and the Order to Read Them.

Web accessibility — free interactive course

Odilon HUGONNOT — Wed, 14 Oct 2026 11:00:03 +0000

Web accessibility

Covers the 2026 legal obligations (European Accessibility Act, RGAA) that most teams only discover once an audit lands on their desk.

Free interactive web accessibility course in 7 lessons: semantic HTML, keyboard navigation and visible focus, alt text, accessible forms, WCAG contrast ratios, ARIA as a last resort, free testing tools.

Free, interactive, no signup: https://www.web-developpeur.com/en/learning/accessibilite/

Best Clean Code and Refactoring Books, and the Order to Read Them

Odilon HUGONNOT — Wed, 14 Oct 2026 09:00:02 +0000

Clean Code was the first text I handed to my students. Since then I have read the six other books fighting for the "clean code" shelf, and they do not all say the same thing. Sometimes they say the opposite.

I wrote a reading note for each one. Here is the order I would follow if I were starting over, and what you can leave on the shelf.

The craft before the rules

The Pragmatic Programmer by Thomas and Hunt (2019 edition, 8/10) is my way in. What strikes you first is the tone: few commandments, many stories, never a condescending look. And this edition publicly corrects the previous one. The Law of Demeter is demoted, DRY (do not duplicate a piece of knowledge) is clarified because people misread it, and on security the authors write "we were wrong". A book that owns its mistakes twenty years later earns trust for everything else.

The limit is mechanical: 53 topics in 352 pages means skimming. The tooling chapter is thin, and the code samples jump from Ruby to Elixir to Clojure without digging into any of them. You read it for the stories and mental models, not the code. Skip the samples, keep the rest.

The dogma and its counterweight

Clean Code by Robert C. Martin (2008, 6.8/10) taught a generation that code is read ten times more than it is written. The chapters on naming, functions and comments have not aged a day. The rest is verbose Java, and the last three case-study chapters are 90 pages of listings nobody reads anymore, me included. One thing the fans forget: Martin himself admits that no research backs the short-functions rule, and that his own functions start out long and messy before being refined under tests. The "three lines maximum" dogma comes more from his readers than from the book.

To hear the other bell, read A Philosophy of Software Design by John Ousterhout (2021, 8/10). A hundred and ninety-six pages, and in my eyes the best software design book of the last twenty years. His deep module idea changed how I judge a class: I stopped counting lines and started comparing what the interface costs to learn with what it gives back. The examples circle around two or three cases (a text editor, RAMCloud) and smell like a lecture hall. And "make your modules deep" is easier to understand than to apply. Where the two books contradict each other, on function size and on comments, Ousterhout is almost always right.

The first hundred pages of construction

Code Complete by Steve McConnell (2004, 960 pages, 7.4/10) is the book I wish I had read at the start of my career, but for its first hundred pages, not its 960. The thesis on complexity, abstract data types, cohesive routines, defensive barricades: you usually learn all of that by trial and error, after humbling code reviews. The book shortcuts that painful education.

What has aged is visible: samples in C++, Visual Basic 6 and Pascal, "code tuning" chapters about processor pipelines, a tools catalogue straight from a museum. What holds up is everything about humans: the numbers on code inspections, the psychology of debugging, complexity as the universal enemy. You do not need all of it. Parts I, II and VII plus chapters 20 and 21 make roughly 250 pages, and the rest is reference material you open now and then.

Refactoring and legacy, the two moves

Refactoring by Martin Fowler (2018, 8.2/10) holds the best code lesson I have ever read. Chapter 1 is seventy pages of watching someone work, every move explained, the code never broken. Fowler has a rare honesty: his central argument is labeled "unproven hypothesis". The limit is that half the book is a dictionary of transformations nobody reads in one sitting. Read chapters 1 to 4 like a novel and keep the rest within reach during code review. The Refactor menu in your editor exists because of this book.

Then comes Working Effectively with Legacy Code by Michael Feathers (2004, 7/10), which redefines the word: legacy is not old code, it is code without tests. His characterization tests (which snapshot current behavior) answer a question nobody phrases well: how do you start testing when you do not know what "correct" means for this code? The seam model (places where you can change behavior without editing the code) teaches you to read code by looking for joints. Feathers also admits that his techniques often make code uglier in the short term, and that the scar is temporary. That frankness makes the book reliable.

It is dated, and every example is Java, C++ or C. Part II, twenty chapters in FAQ format, often repeats the same four moves under a different class name. Open it at the chapter matching your problem of the day.

Test first, to lose the fear

Test-Driven Development: By Example by Kent Beck (2002, 240 pages, 7.6/10) does not sell a recipe, it shows a state of mind: the calm of knowing, every second, that the code still works. The fear of breaking things turned into quiet boredom. Since then I no longer write tests to "have coverage", I write them to move forward without dread.

Part I, the money example, drags: you repeat the same micro-cycle dozens of times and you got it long before. The code is 2002 Java and Python 2. TDD is still debated, and Beck is the first to say no study settles it. Read it for the mindset, keep the cycle, the TODO list and "fake it" when you are stuck, and adapt the micro-steps to your own pace.

The order I would follow

Book

Pages

Rating

What it is for

The Pragmatic Programmer

352

The craft's mental models

Clean Code

464

6.8

Naming, functions, comments

A Philosophy of Software Design

196

The counterweight: deep modules

Refactoring

448

8.2

Chapters 1 to 4, then the catalog as needed

TDD by Example

240

7.6

Moving forward without fear of breaking

Working Effectively with Legacy Code

440

Testing code that never had tests

as needed

Code Complete

960

7.4

The useful 250 pages, then reference

Conclusion

The book I used to hand my students has the lowest rating on the list, 6.8/10. And it is Fowler and Feathers who hold the moves you actually use on a Monday morning, in code you did not write.

If you want to see these moves applied to a real project, I wrote about the order of a legacy Symfony audit.

I stitched the marrow of dozens of tech books into a single thread, from bits and Big O up to the AI era: that is The Developer's Great Book. Four of these books (Clean Code, A Philosophy of Software Design, The Pragmatic Programmer, Refactoring) are melted into it at the "Writing for humans" level, and three others (TDD by Example, Working Effectively with Legacy Code, Code Complete) at the "Prove it, ship it, hold the load" level.

📚 Other comparisons. Best DevOps and Engineering Team Books, and the Order to Read Them and Best Event Sourcing and CQRS Books, and the Order to Read Them.