DEV Community

Devanshu Biswas
Devanshu Biswas

Posted on

A Payload Hash Plus a Per-Attempt Request ID Is Byte-For-Byte Identical to Having No Idempotency At All

New series. Arc Rector answers what do I build on. Arc Ops answers the other half: how do I run an agent that touches real systems without getting paged? Nine levels - L0 Idempotency, L1 Retry budgets, L2 Cost caps, L3 Approval gates, L4 Determinism and replay, L5 Sandboxing, L6 Audit trails, L7 Queues and concurrency, L8 Incident response. Every level ships a measured demonstration and a control: the same code with the guard removed, run over the same input, so the guard is proved load-bearing rather than assumed.

Repo: https://github.com/dev48v/arc-ops - PUBLIC, MIT, dependencies = [], 113 tests, no network and no model. Level 0 computes in your browser on a page that is genuinely self-contained - no CDN, no font host, not one asset fetched off the page: https://dev48.infy.uk/arcops/level0-idempotency.html

Exactly-once delivery does not exist. What exists is at-least-once delivery plus an effect that only lands once, and that second half is the whole level. One fixed loss plan turns 8 invoices into 14 attempts, and five handlers see identical traffic.

handler attempts effects duplicates missing charged
no guard (control) 14 14 6 0 94,000c
intent key 14 8 0 0 48,500c
payload hash, stable bodies 14 6 0 2 40,100c
payload hash + stamped client_request_id 14 14 6 0 94,000c

The guard is worth exactly 45,500c on this workload - the difference between two ledgers, not a claim about one.

The headline is the fourth row

A payload hash is what people reach for, and a client that stamps a per-attempt request id is ordinary hygiene. Together they reproduce the control's ledger byte for byte: 14 effects, 94,000c, every retry charged again. And with stable bodies the same strategy silently loses two charges while reporting zero duplicates - so a dashboard that counts duplicates and nothing else scores it as clean. Two error columns, always. Duplicates is the loud failure; missing is the quiet one, an intent that produced no effect at all because a different intent hashed to its key and answered on its behalf.

Fourteen hand-labelled pairs of wire requests, scored in both directions:

strategy correct missed dedup wrong dedup
hash the body 4/14 7 3
hash the sorted body 6/14 5 3
intent-shaped, untyped encoding 12/14 0 2
intent key 14/14 0 0

The three wrong-dedup pairs have byte-identical bodies and different meanings, because the invoice, the account and the actor are not in the body. A body hash merges them, and the second call gets a 200 carrying somebody else's receipt.

Count is not order, and a key does not fix order

set_status is naturally idempotent: replay it three times and you get three writes and one state. The trap is reading that as replay-safe. It bounds the count, not the order. Land a stale v2 paid after v3 shipped and last-write-wins reverts the record to paid with no duplicate anywhere to show for it; compare-and-set on a version rejects the stale write and holds at shipped. A key stops a duplicate append and says nothing about ordering. A version stops a stale assignment and does nothing at all to an append. Neither substitutes for the other.

Two workers, one key, deterministically

A lease expiring is indistinguishable from a worker that died, so a broker will hand the same message to two consumers. A threading.Barrier forces every worker to read before any worker writes, which makes the failure assertable rather than flaky. Check-then-act admits N of N at 2, 3, 4, 8 and 16 workers. An atomic claim and a sqlite INSERT OR IGNORE admit exactly 1 at every count. Both stores are interface-identical and behave identically on one thread, which is precisely why the broken one ships: nothing in the calling code changes and the tests pass.

Nine levels, one at a time: https://dev48.infy.uk/arcops.php

Top comments (0)