DEV Community: polycratia

Sign the quote behind every 402, or the retry pays a different price

polycratia — Mon, 28 Sep 2026 09:34:12 +0000

The x402 exchange comes in two halves with an unbounded gap between them. A server answers an unpaid request with 402 and the terms it accepts; some time later the client repeats the request with an X-PAYMENT header built against those terms. If the gate only knows its current price, it checks the second half against state that may have moved since the first half went out. The client can pay a price it was never shown, or get refused for paying exactly the price it was.

I have spent most of the last eight years on payment systems where the interesting failures sit in the bookkeeping rather than the transfer, and this is a bookkeeping failure in a pricing costume. It is an evidence problem: at retry time the server holds no artifact of what it offered, so it substitutes what it offers now and hopes the two are the same thing.

Where the gap actually is

Look at what a gate has in hand when the paid retry arrives. In paygate402 the accepted terms come from either a fixed Accepts list or a Prices table keyed by route. Both are server configuration, read at request time. The payment payload stays opaque on purpose: the package matches only the scheme and the network, because those are the only fields a web layer can honestly compare, and the amount, the asset and the recipient are checked by the facilitator that can read the payload.

That division of labour is right, and it is exactly what makes the stale-price case invisible. The facilitator verifies the payment against the terms the gate hands it, and the gate hands it today's terms. Nobody in the chain holds the terms that were actually printed in the 402. If the price moved between the challenge and the retry (a deploy, a config reload, a route that started resolving to a different row of the pricing table), the client's signed authorization gets judged against an offer it never saw. When the new price is higher the payment is declined, and the client is refused for doing precisely what the server told it to do. When the new price is lower, the authorization still covers it, and the client pays above the number now on the board.

The operational version of this is worse than the technical one. A client writes in saying it was quoted one amount and charged against another, and the server cannot contradict it, because the server kept no copy of the quote. Reconciliation work teaches you fast that a disagreement between two parties resolves only when both hold a record of the same row. A 402 that carries a bare number gives the client a record and keeps none.

The offer is a bearer artifact, not server state

The fix is to stop treating the price as something the server remembers and start treating it as something the client carries. paygate402 does this with a signer: when Quotes is set, the gate signs the priced offer behind every 402 with its own key (amount, asset, expiry and a nonce) and sends one X-PAYMENT-QUOTE header per accepted term. The client echoes the quote back with its payment.

gate := &paygate402.Gate{
    Accepts: []paygate402.Requirements{{
        Scheme:            "exact",
        Network:           "base",
        MaxAmountRequired: "10000",
        Resource:          "https://api.example.com/report",
        PayTo:             merchantAddress,
        Asset:             usdcAddress,
        MaxTimeoutSeconds: 60,
    }},
    Facilitator: &paygate402.HTTPFacilitator{BaseURL: "https://facilitator.example.com"},
    Quotes:      &paygate402.QuoteSigner{Key: quoteKey},
}

Nothing is stored for this. There is no quote table and no expiry sweeper, nothing to replicate between instances, because the offer proves its own provenance. The practical consequence: a price change stops being a race. Outstanding quotes stay honourable until they run out, new requests get the new number, and the two coexist for exactly the lifetime you chose when you signed. That is the property I actually wanted out of this, and it is not the one I expected to want.

Three checks, and only one of them is about forgery

With a quote in hand, the gate checks three things before a facilitator is asked anything: that this server signed the offer, that the offer has not run out, and that it is the offer for the terms being paid for now.

They fail differently, so they are worth separating.

The first is forgery. Without a signature the price is a number the client hands back, which means the client sets it. This one is obvious, and it is the one everybody implements.

The second is staleness. A quote without an expiry is a perpetual option on your price, written by you and held by whoever asked once. Anyone who has priced anything against a moving asset knows what a free option is worth to the holder and what it costs the writer. Call the expiry the term of the offer, because that is what it does, and hygiene has nothing to do with it.

The third gets forgotten: substitution. A signature says the artifact is genuine. It does not say the artifact is genuine about this. A valid, unexpired, correctly signed quote for a cheap route, presented on an expensive one, passes both of the first two checks. Binding the offer to the terms being bought is what closes that, and the binding has to live inside the signed bytes rather than get checked alongside them.

The ordering matters too. The quote is checked before the facilitator call, not after:

window := g.replayWindow()
if g.Quotes != nil {
    offer, err := g.presented(r, terms)
    if err != nil {
        g.challenge(w, accepts, reason(err))
        return
    }
    window = ReplayWindowFor(offer, g.Quotes.now(), 0)
}

verification, err := g.Facilitator.Verify(r.Context(), payment, terms)

A payment answering no offer this server made, or one that has run out, is not a question worth putting to a facilitator. The facilitator call is the expensive, rate-limited, occasionally unreachable part of the request. Anything you can refuse from local evidence, refuse from local evidence.

Canonical bytes, not the JSON

The offer is signed over a canonical length-prefixed form rather than over its JSON representation. Two separate reasons, and both bite in production.

JSON is not a stable byte string. Key order, whitespace, number formatting and escaping all vary by encoder and by version of encoder. A verifier that reconstructs the JSON to check the signature is betting that its serializer agrees byte for byte with the one that signed. That bet holds until the day it does not, and the symptom is a fleet where some instances reject quotes issued by others.

The length prefixes are the second reason. Concatenating fields to sign them lets the boundaries move: an amount of 10 followed by a field starting 000 signs the same bytes as an amount of 100 followed by 00. Length prefixes make the parse of the signed bytes unique, which is the whole point: you are not signing some fields, you are signing one unambiguous reading of them.

The expiry you already wrote is the retention you needed

There is a quiet payoff here. A spent payment has to be remembered until the offer behind it stops standing, plus a margin for the difference between the clock that stamped the quote and the clock that reads it. Without a signer, paygate402 remembers a used payment for a default hour, a number chosen because some number was required. With a signer, the window comes from the quote's own lifetime: ReplayWindowFor, above. Shorter leaves a window open, longer only makes the store grow.

This does not make the replay ledger optional. A settled X-PAYMENT header is still a bearer artifact and still has to be recorded as spent. But the retention parameter stops being a guess, because the offer already states when it stops being presentable.

What I would do differently

I built the quote signer as a way to bound replay retention and only afterwards understood it as a correctness property of pricing. That ordering cost me time. The argument that should have come first is the plain one: if you cannot show what you offered, you cannot defend what you charged.

I would also treat the skew margin as a first-class decision rather than a 0 passed at the call site. Two machines that disagree by a few seconds will produce a narrow band of quotes that one instance considers live and another considers dead, and in the logs that band looks exactly like a client bug...

The key deserves the same treatment. A signer with a single key works until the first rotation, and rotation on a bearer artifact with a lifetime means accepting the previous key for at least as long as the longest outstanding quote. Small amount of design done early, or an outage done late.

The gate is at https://github.com/polycratia/paygate402. The signer is optional there, and the gate without one behaves exactly as it did before. But where the price can move between the challenge and the retry, which is probably most things worth charging for, the optional part is what makes the 402 mean something.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

A cryptography inventory is a reconciliation problem

polycratia — Tue, 22 Sep 2026 13:27:19 +0000

Most post-quantum migration plans stall on the inventory, before the migration starts. Your source code says one thing about the cryptography you run, the TLS handshake says another, and until both sides are written down in the same format you are guessing about which one holds in production. You cannot migrate what you cannot see, and the algorithms that actually protect your traffic were picked by a handshake nobody in the building has read.

I have spent most of the last eight years on systems where money moves, so the shape of this problem is familiar. An internal ledger and a bank statement do not agree on the first pass. The rows that matched were the boring part: the work was in deciding what an unmatched row meant, and in refusing to auto-match anything just to make the report look clean. A cryptography bill of materials is the same exercise with different inputs: two sources, a disagreement, and a discipline about what you are allowed to infer.

That is the premise behind a small Go tool I maintain, cbomscope. It reads both sides and writes the result as a CycloneDX 1.6 cryptography bill of materials.

Two ledgers, and they disagree on purpose

The first ledger is the code. Someone wrote a key size into a call years ago, and that line is a fact about intent at the moment it was written:

rsa.GenerateKey(rand.Reader, 2048)

Scanning for that is easy, and badly incomplete: almost nothing about the cryptography that protects a live connection gets decided in your repository. The second ledger is the endpoint itself:

$ cbomscope probe cloudflare.com:443
POSTURE             ASSET                   WHERE               WHY
quantum_vulnerable  ECDSA-P-256             cloudflare.com:443  Shor's algorithm solves the underlying elliptic-curve discrete logarithm
quantum_vulnerable  ECDSA-SHA256            cloudflare.com:443  Shor's algorithm solves the underlying elliptic-curve discrete logarithm
quantum_reduced     TLS_AES_128_GCM_SHA256  cloudflare.com:443  Grover halves this to about 64 bits of quantum work; 256-bit keys are the usual answer
hybrid              X25519MLKEM768          cloudflare.com:443  classical and post-quantum key exchange combined: secure if either half holds
not_applicable      TLS 1.3                 cloudflare.com:443  a protocol version has no quantum posture of its own: see the key exchange and cipher it negotiated

5 asset(s): quantum_vulnerable=2 quantum_reduced=1 hybrid=1 not_applicable=1

That hybrid key exchange group is the whole argument for the probe. No source file chose X25519MLKEM768. A runtime default, a terminating load balancer, a proxy in front of the service, or an operating system package upgrade chose it, and the repository is silent on all four. A scan of the code would have reported this deployment as less prepared than it is.

The reverse case is the one that costs you. An asset that shows up on the wire and nowhere in your code is a dependency on a decision made outside your change control. The day it moves (a base image bumps, a proxy config changes, a cipher is retired upstream) no diff in your repository will show it. In reconciliation terms that is an unmatched statement line: money arrived that your ledger does not explain, and the explanation lives in a system you do not own. You do not get to ignore it because your side balances.

What the scanner refuses to guess

The temptation with a static scan is to grep. I did not, because grep produces the exact failure mode that makes reconciliation reports useless: confident rows that are wrong.

The scanner reads Go syntax trees. Import aliases are followed, so renaming a package at the import does not hide a call. A comment or a string literal that mentions rsa.GenerateKey is not a finding, because it is not a call. Parameters come from the call site where they are literal (the line above becomes RSA-2048), and where the key size arrives in a variable, the asset is reported with no size at all rather than a plausible default.

That last rule is the one I would defend hardest. AES-128 and AES-256 do not share a verdict: one is weakened by a quadratic speedup on unstructured search, the other is not meaningfully touched. Filling in a default so the row looks complete is the same move as auto-matching a payment to the nearest invoice because the amounts are close. It closes the ticket and leaves the exposure. An honest gap in an inventory can be assigned to a person. A wrong entry cannot, because nobody will look at it again.

The probe has a matching rule pointing the other way: it skips certificate verification on purpose. The job is to record what an endpoint presents, and an expired or self-signed certificate is exactly the inventory that needs attention, not an error that should abort the reading. The connection carries no application data.

Every verdict is a row, and every row cites a document

What makes this usable by someone other than me is that the classification is not code. It is a table in internal/classify/table.go, and every row names the document it came from. RSA, DSA, DH, and everything elliptic-curve are quantum-vulnerable on Shor; AES and SHA-2 are judged by key and digest size; ML-KEM, ML-DSA and SLH-DSA are quantum-safe on FIPS 203, 204 and 205; MD5, SHA-1, RC4, 3DES and TLS 1.0/1.1 are broken already, on RFC 6151, RFC 7465, SP 800-131A and RFC 8996.

The citation travels with the finding, into the JSON output and into the CBOM as a cbomscope:citation property, so a reviewer can check a posture against the standard instead of trusting the tool. And because the verdicts are data rather than a chain of conditions, the tool can print what it believes without anyone reading its source:

$ cbomscope table
FAMILY          VERDICT             SOURCE
X25519MLKEM768  hybrid              draft-ietf-tls-hybrid-design: hybrid key exchange in TLS 1.3
ML-KEM          quantum_safe        NIST FIPS 203: ML-KEM, module-lattice key encapsulation
RSA             quantum_vulnerable  Shor 1997, SIAM J. Comput. 26(5): polynomial-time factoring and discrete logarithms; NIST IR 8547
AES             by size             Grover 1996, STOC: a quadratic speedup for unstructured search; NIST IR 8547
...

A family with no row comes out as unknown, with a rationale saying a person has to judge it and with no citation attached. That omission is deliberate. A source printed next to an answer nobody actually has would make the gap look reviewed, and that is worse than leaving it visibly open.

The gate has to be one you will still respect next quarter

Only broken fails the command by default. quantum_vulnerable describes almost every deployment on earth right now, mine included: making it an error would turn the exit code into noise on the first run and probably get the tool pulled from CI inside a week. -fail-on is there for teams that have already done the work and want a tighter gate.

This is the same judgement as alerting on reconciliation breaks. If the alert fires on every unmatched line, it fires on all of them, and nobody reads any of them. You alert on the class you intend to act on this week, and you report the rest.

The artifact is what you are after:

cbomscope cbom . -probe api.example.com:443 -o cbom.json

What I would do differently

I first treated this as an audit: run it, read the output, file the work. That was wrong, for the same reason a once-a-year reconciliation is wrong. The source-code side of the inventory only changes when someone commits, so a diff already covers it. The wire side changes without anyone committing anything, which means a CBOM generated once is a statement about a day that has already gone by. Generate it on a schedule against the real endpoints, commit the artifact, and diff it. The signal you want is the change in the file that no pull request explains.

The tool is narrow. Source scanning is Go only, covering the standard library's crypto packages, x/crypto/chacha20poly1305 and crypto/mlkem; the probe covers TLS version, cipher suite, key exchange group, and the certificate's key and signature. It needs Go 1.25 or newer, because reporting the negotiated key exchange group requires it, and it has no dependencies outside the standard library.

None of this predicts when a cryptographically relevant quantum computer arrives. It says what would fall if one did, from a table you can check, against two sources that are allowed to disagree.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

Why leniency in a DER parser becomes a signature bypass

polycratia — Mon, 21 Sep 2026 13:22:21 +0000

Two implementations can read the same X.509 certificate and disagree about what it says. The cause is almost never the cryptography. BER permits several encodings of one value, DER permits exactly one, and a parser that quietly accepts the alternates has handed the document a second meaning, one the signature covers just as well as the first.

Most of my production work since 2018 has been payments, crypto rails and compliance tooling, and a good part of it involved documents that have to hold up as evidence after the fact: legally binding e-signature services and the document workflow around them, multi-party deal signing, on-chain tokens that accumulate signed state from listing to closing. The lesson that kept repeating there has nothing to do with algorithms. A signature attests to bytes. Between bytes and meaning sits a parser, and if that parser admits more than one reading, the signature attests to less than you think it does.

I wrote derstrict (https://github.com/polycratia/derstrict) as a header-only C++17 reader that refuses everything DER already forbids. What follows is the reasoning behind that, not an API tour.

One value, four encodings

Here is the integer 5 inside a SEQUENCE, written four ways. Only the first is DER.

30 03 02 01 05           SEQUENCE { INTEGER 5 }        valid DER
30 80 02 01 05 00 00     indefinite length             BER only
30 81 03 02 01 05        length written long-form      not minimal
30 04 02 02 00 05        INTEGER 00 05                 padded

The same holds one level down. An OBJECT IDENTIFIER arc is a base-128 number, and base 128 has no leading zero digit any more than base ten does, so 80 01 and 01 are one arc written two ways.

None of these are corrupt. Each one decodes, under BER, to exactly the value the first one holds, and that is the whole problem. A parser that accepts all four has not become more robust than a parser that accepts one: it has agreed to four documents where the standard defines one.

Leniency is a claim about every other parser in the system

"Be liberal in what you accept" is advice for a transport. It inverts for a signed document, because there the parse result is the thing being attested, and the attestation is shared with implementations you did not write and cannot inspect.

A signature bypass of this family does not require anyone to break a hash. It requires only that two components in one pipeline read the same bytes differently: the component that checks a name, an extension or a constraint reads encoding A, the component that acts on the value reads encoding B, and both verify the signature successfully, because the signature is over the bytes and the bytes never changed. The attacker does not need to control what the document means. They need the two readings to differ, and every alternate encoding you tolerate is one more place where that difference can be manufactured.

So the question a DER reader has to answer is not "can I make sense of this?" It is "is there exactly one sense to make?" Refusal is the mechanism that forces two independent implementations to agree, and I think that is worth more than any accommodation I could add on top.

The length rules are decided before any content is read

What matters structurally is ordering. Every length rule in derstrict is settled against the enclosing element before a single content byte is touched, so an inner element cannot reach into bytes its parent does not cover. Containment becomes arithmetic done up front, instead of a check somebody remembers to perform after the fact.

Four things are refused at that point. Indefinite length (0x80), because it is BER and because guessing where an element ends is precisely how parsers diverge. A long-form length where the short form fits, because 81 05 and 05 mean the same thing and only one encoding may exist. The reserved length byte 0xFF, because X.690 reserves it and it therefore announces no byte count at all: reading it as "127 length bytes follow" would be a parser inventing a meaning. And a length that runs past the end of the enclosing element, measured before descent.

The content rules apply the same idea one layer in: a leading 0x00 in an INTEGER is a sign byte only in front of a set top bit and a leading 0xFF is sign extension only in front of a clear one, an OID arc may not end with the continuation bit still set, a BIT STRING's unused-bits count is at most seven, is zero when there is no last byte, and counts bits that are themselves zero.

$ make demo
well formed                accepted: 1.2.840.113549.1.1.11
indefinite length          refused: indefinite length is BER, not DER
length written long-form   refused: the length is encoded the long way
oid ending mid-arc         refused: the object identifier ends mid-arc
bytes after the element    refused: bytes remain after the element

A cursor that cannot lie about how much it read

The last refusal in that list is the one people skip, and it carries the thesis most directly. Bytes after the outermost element mean somebody else read this document differently from you. There is no benign version of that.

Enforcing it means knowing exactly how many bytes were consumed, which is why the cursor carries a sticky failure and an exact remaining(). remaining() never underflows, and it is zero once a read has failed, because a failed parser reads nothing further. That is what makes a whole run of reads checkable at the end instead of at every step:

#include "derstrict/derstrict.hpp"

derstrict::parser outer{data, size};
const auto seq = outer.expect(derstrict::tag::sequence);
if (!seq) return derstrict::describe(outer.failure());

auto inner = outer.into(*seq);
const auto algorithm = inner.oid();
if (!inner.at_end() || !outer.at_end()) return "trailing data";

The two at_end() calls are not defensive noise. The inner one says the constructed element contained what it claimed and nothing more, the outer one says the document did. Without an exact byte count you have neither statement available, and "I parsed it and it looked fine" is not the same claim at all.

Nothing is copied and nothing is owned: content points into the caller's buffer. For a library whose entire job is to decide what a specific byte range means, that is a correctness property as much as a performance one, because there is no second copy that could drift from the bytes the signature covers.

Integers, where exactness usually dies

unsigned_integer() handles the small fields: versions, counts, the things that fit in 64 bits. Serial numbers and key material do not fit, so integer() reads one of any width or sign and hands back a view of the encoding: negative(), and magnitude() for a non-negative value's bytes where they already lie.

A negative value is the interesting case, and it is where I made the deliberate ergonomic sacrifice. Its magnitude is not in the document (the document holds the two's complement), so producing the magnitude means writing bytes somewhere, and this library owns no memory to write them into. magnitude_into() therefore writes into a buffer the caller owns. That is less convenient than returning a big integer, and it is the right trade: the alternative is allocation inside a parser that is supposed to be a pure decision about bytes that already exist.

Refusing something that is legal

High-tag-number form is valid DER. derstrict refuses it anyway, because no field it currently reaches uses it, and a parser that half-supports a construct is worse than one that declines it: the half-support is exactly the region where two implementations diverge. Refusing something legal is a smaller error than guessing at it, and it is a loud error rather than a quiet one.

The same logic governs what is not there yet: UTCTime and GeneralizedTime, string types with their character-set rules, context-specific tags. Those have real encoding rules, and shipping a lenient version of them to look complete would undo the point of the library. Every refusal listed above has a test built from hand-written bytes, 181 checks under AddressSanitizer and UndefinedBehaviorSanitizer, because in a library like this the tests for what it declines are the tests that carry the value.

What I would do differently

The pressure I would resist harder, and earlier, is convenience. Every request a strict reader receives is a request to accept one more thing: a certificate from a device fleet that emits long-form lengths, an old signer that pads its integers. Each one arrives as a small compatibility fix, and each one is a permanent statement that this document has two readings.

The honest way to carry that pressure is scope. derstrict reads the encoding and not the semantics: there is no certificate structure in it, it builds no chains, it verifies no signatures, and pretending otherwise would be the dangerous kind of convenience. It is not a replacement for a reviewed library in a codebase that already has one. It is for the case where the alternative is a hand-rolled loop over a buffer, and that is a case I have walked into more than once...

The framing I would keep is the one I started with. A parser's permissiveness is not a local property. It is a claim about every other implementation that will read the same bytes, and in a signed document that claim is the security boundary.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

Constant-time comparison and erasure the compiler cannot delete

polycratia — Sun, 20 Sep 2026 13:27:16 +0000

Two defects outlive most of the code that still contains them. A MAC compared with memcmp returns at the first differing byte, so anyone who can time the call learns how many leading bytes they guessed right, and recovers the tag one byte at a time (CWE-208). A key cleared with memset at the end of a function is a store into a buffer that is dead afterwards, and the optimizer is permitted to delete it, which leaves the key sitting in memory for a core dump or the next allocation to find (CWE-14). Neither needs a clever fix. Both need a fix the compiler is not permitted to undo.

I have been shipping payment and crypto systems since 2018, and in that time these same two lines have turned up in webhook signature verification, in session token checks, and in custodial wallet code handling key material. They are not exotic. They are what you write when you are thinking about the protocol and not about the code generator. I eventually pulled the fix into a small header so I would stop re-deriving it: ctsafe, header-only, C++17, no allocation, no exceptions. The API is trivial. The threat model is not, and that is what the rest of this post is about.

The shape they ship in

if (memcmp(mac, expected, 16) == 0) { /* accept */ }

// ... and later, at the end of the function
memset(key, 0, sizeof key);

Both lines are correct in the sense that a careful reader will nod at them. memcmp compares the bytes, memset writes zeroes. What neither line is is a statement about the machine. memcmp is specified to return a sign, not to take a fixed time, and every real implementation exits early because early exit is what makes it fast. memset is a store, and in the abstract machine a store nobody can read back is unobservable, so deleting it changes nothing the standard cares about. At -O2 it commonly vanishes.

The uncomfortable part is that the compiler and the attacker want the same thing here. The compiler wants to skip work no one can observe. The attacker's whole premise is that they can observe it.

Reading every byte, and saying so

The comparison has to read the full length, keep every branch independent of the contents, and push the accumulated difference through something the optimizer is not allowed to reason across.

#include "ctsafe/ctsafe.hpp"

// Spans: both lengths travel with the data, so the size is never retyped.
if (ctsafe::equals(expected_tag, presented_tag)) { /* accept */ }

// Or a pointer and a length, when that is what the caller has.
if (ctsafe::equals(mac, expected, 16)) { /* accept */ }

The span overload accepts anything with data() and size(), so std::array, std::vector, std::string, a C array, std::span on C++20, and that matters more than it looks. Most instances of this bug I have found were not a missing constant-time routine. They were a correct routine called with a length retyped at the call site, where it can drift away from the buffer it describes. When the length travels with the data, that drift is not expressible.

A size mismatch answers false before a byte is read. That is deliberate, and worth saying plainly: the length is not secret. Two buffers of different sizes are a structural error rather than a guess to protect, and reading the shorter one past its end to preserve symmetry would be a far worse bug than the leak you avoided.

The demo in the repository prints the property directly:

correct tag            accepted=1
wrong in the last byte accepted=0
wrong in the first     accepted=0
  (all three read all 16 bytes; memcmp would not)

Wrong in the first byte costs what wrong in the last byte costs. That is the whole claim.

The leak moves up one line

Nobody warns you about this part. You replace memcmp, you feel finished, and then you write the code that uses the answer:

if (ctsafe::equals(expected_tag, presented_tag)) {
    memcpy(session_key, derived_key, 32);
}

The comparison no longer leaks. The branch around the memcpy does, the same signal one line further out. Accepted and rejected requests now take visibly different paths, with different stores and a different cache footprint. The fix is to stop treating the answer as a branch condition and start treating it as arithmetic.

ctsafe::mask accepted = ctsafe::mask_from_bool(ctsafe::equals(expected_tag, presented_tag));
ctsafe::copy_if(accepted, session_key, derived_key, 32);
ctsafe::select(accepted, out, derived_key, fallback_key, 32);

select and copy_if read both sides and write every byte of the destination whichever way the mask goes, so a rejected copy costs exactly the stores an accepted one costs. The subtle part is the mask itself, which passes through a value barrier before the loop. Without it the compiler can discover what the mask holds and helpfully rewrite the masked loop as a branch around a memcpy, putting back the leak the mask was there to remove. That recurs across this whole area: each of these routines is one inference away from being optimized back into the bug it fixes.

And a mask is 0xFF or 0x00, nothing else. These functions are arithmetic, not a test. Handed 0x01, copy_if mixes the two sides bit by bit instead of rejecting it. Masks come from eq or mask_from_bool, and a value invented elsewhere is a bug that no validation inside the loop could catch without (of course) branching on it.

Erasure is a platform routine, not a clever loop

The right move for zeroing is not to argue with the optimizer. Every platform I ship on already provides a routine whose entire purpose is to not be elided:

Platform	`erase` calls
Windows	`SecureZeroMemory`
glibc >= 2.25, OpenBSD, FreeBSD	`explicit_bzero`
Annex K available (`__STDC_LIB_EXT1__`)	`memset_s`
anything else	stores through a `volatile` pointer

The volatile row is the last resort, not the design. Every path ends in a compiler barrier, an empty asm block with a memory clobber on GCC and Clang, _ReadWriteBarrier() on MSVC, so the buffer cannot be treated as dead across the call. On Windows the header reaches SecureZeroMemory through <windows.h>, and defining CTSAFE_NO_WINDOWS_H keeps that out of the translation unit and takes the fallback instead.

ctsafe::erase_object(session);     // size taken from the type, not retyped
ctsafe::erase_backend_name();      // which routine that erase actually called

Two things there I would defend in review. erase_object takes the size from the type, which removes the retyped-length failure that the span overload removes from the comparison. And erase_backend_name exists because a guarantee you have to guess at is not one: preprocessor-selected behaviour you cannot interrogate at runtime is how a build quietly lands on the fallback path and nobody notices for two years. The test suite runs at -O2 as well as under the sanitizers, because an erasure that only survives at -O0 is the bug being tested for.

What none of this buys

A unit test cannot prove constant time. Only reading the generated instructions can, and even then the CPU has the last word: caches, branch prediction and speculative execution sit outside the reach of a portable header. What you get is source that does not ask the compiler to leak, plus barriers against the two specific optimizations that break these routines. That is a smaller claim than "constant time", and I think it is the honest one.

Erasing a buffer also does not erase its copies. Nothing in that table reaches a register spill, a block that realloc moved, or a page the kernel already wrote to swap. Erase clears the bytes you name, at the moment you name them. Everywhere else the value travelled is a different problem, and mostly an architectural one.

What I would do differently

Earlier I treated the comparison as the fix, and I was wrong in two directions. The branch at the call site kept the leak alive while I congratulated myself on the loop, and I verified the erasure in a debug build, where it is present anyway and therefore proves nothing. The checks I care about now: the erase backend is asserted in the test run rather than assumed, secret material lives in one type so its size comes from the type at every erase site, and the real work is reducing the number of places a key exists at all rather than trusting erase to clean up after a value that was copied four times on the way in.

Both defects survive review because the code says what the author meant. What the machine does with it is a separate document, and the only durable repair is to state the property in a form the compiler is not permitted to discard, then make the build tell you which form it chose.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

An address validator should return a reason, not a boolean

polycratia — Sat, 19 Sep 2026 13:27:17 +0000

A user pastes an address into a withdrawal form and gets back "invalid address". Sometimes the string was truncated by a copy that clipped at a line break. Sometimes it is a perfectly good address for a network you are not sending on. Sometimes it is an encoding your build does not know yet. Three different situations, three different fixes, and a validator that returns a boolean has thrown away the distinction before any caller could act on it.

I have built custodial wallets for BTC and ETH, an on-chain payment system, and a fiat-to-crypto onramp where addresses arrive from forms, from support tickets, and from other systems' APIs. The shape that survived all of it is small: validation returns a result carrying a typed reason, the UI and the support tooling branch on that reason, and the list of supported chains grows underneath without any of those branches changing. chain-addresses (https://github.com/polycratia/chain-addresses) is where I keep the current version of that shape.

The rejections are not interchangeable

Start from what the caller has to do next, not from what the validator knows.

A checksum that does not match means the bytes on screen are not the bytes anyone intended. The fix is mechanical: ask the user to copy it again, and there is a good chance the second attempt works. This is the one failure where "try again" is honest advice.

An address that decodes cleanly but belongs to another network is the opposite. Nothing is mistyped. Asking the user to re-copy sends them back to the same clipboard for the same string, and they will paste it again, more annoyed. The useful message names the network the address is actually for and stops there, because the real fix lives somewhere else entirely: a different withdrawal flow, a different asset, or a support conversation.

An unexpected version byte inside a well-formed encoding is a third thing, and often not the user's fault at all. Someone is handing you an address type you do not serve yet. That is a roadmap signal rather than a validation error, and it deserves to be counted separately from typos.

One boolean cannot carry any of that, so every call site invents its own message out of the absence of information. That is how you end up with three screens telling the same user three different half-truths.

from chain_addresses import Reason, validate_address

result = validate_address(destination, network="testnet")
if not result:
    if result.reason is Reason.BAD_CHECKSUM:
        return ask_again("that address did not check out - copy it again")
    if result.reason is Reason.WRONG_NETWORK:
        return stop(f"that address belongs to {result.network}")
    return stop("that address format is not supported here")

The result is falsy when the address is not acceptable, so the guard reads like a boolean check and the detail is sitting there when you need it. A valid result carries what it recognised: result.format is one of the format names the package exposes, and result.network is the network the address belongs to.

Precedence is the design, not a detail

This part only shows up once you stop returning booleans. A single bad string is usually wrong in more than one way at once, and you have to decide which truth to report.

Take a testnet address with one character mangled. It fails its checksum, and its version byte says testnet. Both statements are true. If you check the network first, you tell a mainnet user "this is a testnet address", advice derived from bytes you have no reason to trust, since a failed checksum means you may be reading a corruption rather than a version.

So encoding is checked first: a corrupted testnet address is a bad checksum, not a wrong network. You do not name a network from bytes that did not survive their own integrity check. A validator returning a boolean never has to make this call, which is exactly why the question goes unasked until support is reading a ticket that says "it told me this was a Bitcoin address, it is not." Confidently wrong output is worse than a vague rejection, and precedence is the only place you get to prevent it.

The set that grows must not be the set you branch on

This is the structural argument, and it is why adding a chain does not ripple into callers.

There are two sets in play. Address formats are an open set: bitcoin-p2pkh, bitcoin-p2sh, bitcoin-p2wpkh, bitcoin-p2tr, their testnet variants, EIP-55 hex for EVM chains, and whatever comes next. Failure reasons are a closed set: the encoding did not verify, the address is for another network, the version is not one you serve. New chains land in the first set continuously. The second set has barely moved for me in years.

Callers must branch on the closed set. The moment a call site matches on format names or parses an error string, every new chain becomes an edit in the UI, in the support panel, in the API serializer, and in whatever internal script someone wrote last quarter. That is the real cost a boolean hides: it does not just lose information, it pushes the open set into the branch structure of code that has no business knowing about it.

The same separation holds on the generation side. Every encoder takes a 33-byte compressed public key and returns a string, behind one interface:

@runtime_checkable
class AddressEncoder(Protocol):
    """Turns a compressed public key into an address for one chain and format."""

    name: str

    def encode(self, public_key: bytes) -> str:
        ...

Because the interface is that narrow, code holding an encoder never learns which chain it is serving, and a new chain is a new entry in the ENCODERS registry instead of a change in every caller. get_encoder raises AddressError on a name it does not know, so a typo in a configuration value fails at lookup instead of producing something plausible further down.

When a flow only serves one chain, you narrow at the call rather than branching afterwards:

result = validate_address(destination, network="mainnet", formats=["evm"])

EVM addresses are worth a note here, because they show the limit of what a validator can honestly say. EIP-55 hex carries no network marker at all: the checksum is over the hex digits, not over a chain identifier. In this package such an address belongs to Network.ANY and passes any network check, which is not a shortcut but the truth, since the same address is valid on every EVM chain. "Wrong network" is not expressible for it, and a validator that pretended otherwise would be guessing. The chain selector owns that decision; validation does not.

The addresses you can pay to outnumber the ones you can generate

The tempting implementation of validation is a round trip: decode the string, re-encode it with each encoder, accept it if something matches. It is compact, it reuses code you already trust, and it is wrong.

Those two sets are not the same size. A version 0, 32-byte witness program (P2WSH) is a perfectly reasonable withdrawal destination, and it validates as bitcoin-p2wsh even though no encoder in the package produces one. That is deliberate: deriving deposit addresses is something you do for yourself and can keep narrow, while accepting a destination is something you do for the rest of the world, which is wider than your own wallet. A round-trip validator quietly makes the two identical and rejects a customer's perfectly good address because your key derivation happens not to produce that shape.

A boolean cannot express that asymmetry either. A result that names a format can: the format it recognised may be one you never generate, and that is fine.

What I would do differently

I would introduce the reason type before the second chain, not after the third. Retrofitting reasons into a boolean API is easy enough. Replacing the messages that every call site has already invented is the slow part, because by then support has memorised them and someone has written a runbook against the wording.

I would also log the reason rather than the address from the start. Reasons aggregate; addresses do not, and storing them is a liability you do not need. A count per reason is a genuinely useful operational signal: a rise in bad checksums probably points at something clipping strings in a UI, and a rise in wrong-network rejections after a release usually means a field got relabelled, not that users suddenly got careless. Neither is visible when the only thing you recorded was that something was invalid.

The general form of this is not about addresses at all. Any validator at a boundary where a human is still available should return what it learned, not whether it approved. The boolean is a summary you can always compute later; the reason is the thing you can never get back once you have thrown it away.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

The only stateful part of an x402 payment gate is the replay ledger

polycratia — Fri, 18 Sep 2026 13:27:17 +0000

An x402 paywall is three steps: the server answers an unpaid request with 402 and the terms it accepts, the client repeats the same request carrying an X-PAYMENT header, and the server verifies, settles, and only then serves. Two of those steps can be stateless. The third cannot. Unless the server remembers which payments it has already settled, a captured X-PAYMENT header buys the resource again on every replay, and the gate is decoration with a status code.

I wrote paygate402 to sit in front of a Go HTTP handler, and the part of it I keep coming back to is not the protocol plumbing. It is the ledger of spent payments and the exact moment a key gets written to it.

Three steps, one of which repeats

The wiring is small. The gate holds the terms it will accept and a facilitator to ask about payments:

gate := &paygate402.Gate{
    Accepts: []paygate402.Requirements{{
        Scheme:            "exact",
        Network:           "base",
        MaxAmountRequired: "10000",
        Resource:          "https://api.example.com/report",
        PayTo:             merchantAddress,
        Asset:             usdcAddress,
        MaxTimeoutSeconds: 60,
    }},
    Facilitator: &paygate402.HTTPFacilitator{BaseURL: "https://facilitator.example.com"},
}

http.Handle("/report", gate.Handler(reportHandler))

Note where the boundary is. This package does not check signatures and does not move money. In x402 that is the facilitator's job: it reads the scheme-specific payload, recovers the signature, checks the allowance, submits the transfer. So the payload stays opaque here, and matching compares only what a web layer can honestly compare — the scheme and the network. The amount, the asset and the recipient are checked by the component that can read the payload. A middleware that decoded some base64 and called it verification would be worse than no middleware, because it would look like a paywall while charging nobody.

That boundary has a consequence people miss. Everything the gate holds in its hands is a bearer artifact. The X-PAYMENT header is a self-contained, signed instruction, sent over a stateless protocol, with nothing binding it to a session, a connection, or a single attempt. Anything that sees the header holds a complete payment: a proxy log, a retry loop, a shared trace, a browser extension, a support ticket with a captured request. The signature inside it stays valid on the second presentation — that is what a signature is for. The only thing that can make the second presentation different from the first is server memory.

The offer is fine stateless

Before the spend there is the offer, and the offer is the half of this that genuinely does not need a database.

A quote is the priced offer behind a 402: an amount, an asset, the moment the offer stops standing, and a nonce that makes it one of a kind. It is signed with the server's own key, so an offer that comes back can be checked against what was actually offered rather than against what a client says was offered.

signer := &paygate402.QuoteSigner{Key: secret, TTL: time.Minute}

quote, err := signer.Issue(terms)
// …later, with the quote a client returned:
err = signer.Verify(quote) // the signature first, then the expiry

The signature covers a canonical form rather than the JSON: a domain tag, then every set field in a fixed order, each part written as its length followed by its bytes. Lengths rather than separators mean no value can be read as two fields — the classic way a signature over concatenated strings gets forged is by moving the boundary between two of them. A field left empty is not written at all, which buys forward compatibility for free: a field added in a later version leaves the signed bytes of an older quote exactly as they were, and signatures issued before it keep verifying.

The expiry is signed as whole Unix seconds, and that is not fussiness. A timestamp is only safe inside a signature if both sides can reconstruct the identical bytes; carry sub-second precision through a JSON round trip and the two sides eventually disagree in the last digit, which reads as a forged quote rather than as a serialization bug. Truncating to whole seconds removes the disagreement.

All of that makes the offer re-checkable without the server storing anything. But a quote signature is this server's, not a chain's. It says "these were my terms", and nothing whatsoever about whether anyone paid. Collapsing those two classes of fact — treating a valid signed artifact as evidence of settlement — is the single most expensive mistake available in this design.

The spend cannot be stateless

So the ledger. A captured X-PAYMENT header is refused the second time, and the interface for remembering is deliberately one method:

// SeenStore remembers payments that have already been used, so that a captured
// X-PAYMENT header cannot be replayed against the same server.
type SeenStore interface {
    // SeenBefore records a key and reports whether it was already there.
    SeenBefore(key string, ttl time.Duration) bool
}

// PaymentKey is the replay key for a payment.
func PaymentKey(payment Payment) string {
    sum := sha256.New()
    sum.Write([]byte(payment.Scheme))
    sum.Write([]byte{0})
    sum.Write([]byte(payment.Network))
    sum.Write([]byte{0})
    sum.Write(payment.Payload)
    return hex.EncodeToString(sum.Sum(nil))
}

Two decisions are load-bearing here.

The key is a digest of the whole payload rather than a nonce field. Every x402 scheme carries its own payload shape, and reaching into one to find "the nonce" breaks the moment a new scheme appears — the web layer would have to grow a parser for a format it explicitly refuses to interpret everywhere else. Digesting the bytes keeps the gate scheme-agnostic. The trade-off is real and worth stating plainly: two distinct encodings of the same underlying authorization would hash to two keys. That is a property of the scheme, and it is catchable in the component that actually reads payloads. What the digest does refuse, completely, is the exact captured header replayed verbatim — which is the threat that actually exists in a bearer-token protocol.

The second decision is SeenBefore recording and reporting in one call. Check-then-set as two calls is a race, and it is the specific race that costs money: two concurrent requests carrying the same header both read "not seen", both settle. One call that atomically records and tells you whether it was already there is the only shape that survives concurrency, and it happens to be the shape a shared store implements naturally.

When the key gets written

This is the part I would argue about with someone, so here is the reasoning.

The key is recorded only after verification passes. A payment the facilitator rejected never touched the chain, and a client who fixes their allowance may legitimately resend the same signed payload. Burning the key on a rejection turns a recoverable client error into a permanently unusable payment.

Once settlement has been attempted, the payment stays spent — even when settlement failed. A rejected verification is a known non-event. A failed settlement is an unknown: the transfer may have landed and the response may have been lost. The ambiguous case must never risk a double charge, so it refuses in the direction of refusal.

Two orderings around it matter just as much. Settle after the handler and serve after settlement: the handler runs into a buffer, so if the work fails nothing is charged, and if settlement fails nothing is served. And an unreachable facilitator is a 502, not a 402. "We could not ask" and "the answer is no" are different outcomes; answering 402 when the facilitator is down tells a client to pay a second time for something they may already have paid for. That is the same instinct as the ledger, expressed in a status code.

The default store is wrong for the deployment you will actually have

The bundled store keeps seen payments in the process:

func (m *MemoryStore) SeenBefore(key string, ttl time.Duration) bool {
    m.mu.Lock()
    defer m.mu.Unlock()

    now := m.now()
    // Expiry is swept on write. A payment gate sees writes on exactly the
    // requests that matter, so no background goroutine has to exist.
    for existing, expires := range m.seen {
        if now.After(expires) {
            delete(m.seen, existing)
        }
    }
    if expires, ok := m.seen[key]; ok && now.Before(expires) {
        return true
    }
    if ttl <= 0 {
        ttl = time.Hour
    }
    m.seen[key] = now.Add(ttl)
    return false
}

It is right for one instance and wrong for several: a second replica does not share the map, so a payment could be replayed once per replica. That is why SeenStore is an interface rather than a struct field — a shared implementation drops straight in. Sweeping expiry on write is a small pleasure: a payment gate sees writes on exactly the requests that matter, so retention costs nothing and no background goroutine has to exist.

The ttl argument is a retention question with a security floor. The key has to outlive every window in which the same payload could still be accepted by anything downstream — the quote's standing time, the facilitator's own tolerance, the settlement's finality. Short of that, the entry expires and the replay works.

What I would do differently

If the service will ever run more than one replica, I would not ship the in-process store at all, not even briefly. Replay windows do not announce themselves; they show up as a reconciliation discrepancy weeks later, and by then the header that caused it is long gone from the logs. Eight years of payment work has taught me exactly one durable thing about this class of bug: a refusal is cheap and a double charge is not, so every ambiguous branch should point at refusal.

The other thing I would change is smaller. SeenBefore returns a bool, which is enough to gate a request and not enough to explain one. If I were operating this at any scale I would want the ledger to also be able to say when the key was first seen, because "this payment has already been used" is a support ticket, and the answer to that ticket lives in the store.

The gate is the part that looks like the product. The ledger is the part that makes it true.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

Threshold approvals belong on the request, not on the button

polycratia — Thu, 17 Sep 2026 13:27:16 +0000

Above a configured amount, a withdrawal should not leave the system because somebody clicked approve. It leaves because enough distinct people have signed that specific request, and because those signatures still satisfy the policy at the moment a rail is actually touched. Most of the approval bugs I have found in payment and custody systems come from the opposite arrangement: the click is treated as the event that moves money, and the request record is a log line written afterwards.

That inversion is what makes both classes of failure possible. If the click sends, then a second click sends again, and a policy change tomorrow cannot invalidate a signature collected yesterday, because there is nothing left to re-measure. I have built moderation and issuance tiers for a lending marketplace, and real-time compliance queues for an onramp, and the same lesson showed up in both: the reviewer's action has to be recorded as a property of the thing under review, not as a trigger wired to a side effect.

I keep this pattern in withdrawals. It is deliberately small: a request, a state machine, an approval gate, a routing decision, and idempotent submission. The senders are not implemented, and that is on purpose: everything interesting happens before the rail.

Waiting is a property of the state machine, not of the handler

If the gate is a check at the top of the send handler, it is one refactor away from being skipped, and every new call path has to remember it. It holds up better when approved is simply unreachable until the policy is satisfied. A request travels requested -> approved -> sending -> sent -> confirmed, and the sender only ever receives requests that are already on the far side of the gate.

from decimal import Decimal

from withdrawals import (
    ApprovalPolicy,
    WithdrawalRequest,
    WithdrawalState,
    approve,
)

policy = ApprovalPolicy.four_eyes(above=Decimal("1000"))

large = WithdrawalRequest(
    id="w-1042",
    amount=Decimal("5000.00"),
    currency="USDC",
    destination="0xab5801a7d398351b8be11c439e05c5b3259aec9b",
    requested_by="alice",
)

waiting = approve(large, "bob", policy)
assert waiting.state is WithdrawalState.REQUESTED

approved = approve(waiting, "carol", policy)
assert approved.state is WithdrawalState.APPROVED
assert approved.approvers == ("bob", "carol")

Bob's approval is not rejected and it is not parked somewhere else. It is recorded on the request, and the request stays in requested, because the amount cleared a rule that asks for two distinct approvers. The policy is a set of rules, each with the amount it starts at and the number of approvers it asks for: the rule with the highest threshold the amount clears is the one that applies, and one rule starts at zero so that every amount matches exactly one. No amount falls through the policy unmatched.

Note that request.approve() and approve(request, who, policy) are different calls. The first is the bare state move, useful in tests and for amounts below any threshold. The second is the one that asks the policy first. Keeping them apart means the policy-aware path is a real function with a name, not an implicit behaviour of a method that also does something else.

An approval is evidence, and evidence gets re-checked

Every approval names who gave it, when they gave it with a timezone-aware timestamp, and the rule that was in force at the time. All of it travels with the request all the way to confirmed.

assert approved.approvals[0].policy == "four-eyes"

That last field is the one people leave out, and it is the one that matters most a year in. Thresholds move. A limit that asked for two approvers above one amount gets tightened after an incident or a compliance review. If the only thing you stored was a boolean, every request approved under the old rule is now indistinguishable from one approved under the new one, and you find the difference in an audit rather than in your code.

So the check happens twice: once when the approval is given, and once at the boundary, immediately before the request is handed to a rail.

from withdrawals import ensure_approved

ensure_approved(approved, policy)  # raises NotApproved if the signatures no longer suffice

This works because the request is an immutable value and the approvals are part of it. Transitions return a new request and you decide where to store it, so nothing can quietly mutate the evidence between the gate and the send. ensure_approved measures the request against the policy as it stands right now. A request approved yesterday under a looser rule does not slip out today.

The same click twice is not a second pair of eyes

Two more things the approval layer refuses. The person who asked for the money cannot be one of the approvers, which raises SelfApproval. And the same approver recorded twice counts once, so a double-clicked button, a retried request, or a duplicated webhook from an approval UI does not manufacture the second signature.

That is idempotency at the human layer, and it is worth naming as such, because it has the same shape as the machine-layer problem further down the pipeline. The identity that matters is the approver, not the click. Once you see it that way, the rest of the pipeline is a question of asking, at each step, which identity is this step keyed by?

After the gate, two more identities

They are not the same identity, and collapsing them is where money gets sent twice.

Routing is keyed by the destination. A withdrawal to an address you custody does not reach a chain: crediting it is a ledger move with no fee, no confirmations, and nothing to wait for. The decision comes back as a value with the reason behind it, and it is logged, so the two paths stay distinguishable afterwards:

from withdrawals import InMemoryCustody, Rail, route

custody = InMemoryCustody({"0x5c69bee701ef814a2b6a3edd4b1652cb9cc5aa6f": "acct-77"})

decision = route(request, custody)

assert decision.rail is Rail.INTERNAL
assert decision.account == "acct-77"
assert not decision.needs_confirmations

An internal route without an account and an external route with one are both refused at construction. The decision cannot be half-formed.

Submission is keyed by neither the destination nor the withdrawal id, but by a request key the client chooses. A client that does not hear back retries, and it retries with the same key. The first call under a key runs the send, every later call returns what the first one produced, and the rail is touched once:

from withdrawals import Submissions

submissions = Submissions()

def send(pending):
    return pending.approve().start_sending().mark_sent("0xdeadbeef")

first = submissions.submit("client-req-9f21", request, send)
again = submissions.submit("client-req-9f21", request, send)

assert again is first

The same key carrying a different withdrawal raises IdempotencyConflict rather than paying twice. And the case I care about most: when the send itself raises, the key stays claimed and a retry raises SubmissionInFlight, because nobody yet knows whether the rail saw it. An RPC timeout is not a failure and it is not a success. It is a third outcome, and I think the only honest response is to refuse to guess. You close it deliberately once it has been reconciled, with resolve(key, outcome) if the send did land, or release(key) if it did not.

The state machine holds the same line on the callback side. Replaying a transition that already happened returns the same value, so a confirmation delivered twice costs nothing, while a replay carrying different data (the same withdrawal marked sent under a different reference) raises InvalidTransition instead of silently overwriting what you already told your accounting system.

What I would do differently

In earlier systems I stored approvals in a generic audit table and a boolean on the withdrawal row. It reads as separation of concerns and it is not: the audit table is written for humans, so nothing in the code path ever reads it back, and the boolean is the only thing the sender consults. Once the threshold changes, the boolean is a claim with no supporting evidence attached, and reconstructing which rule applied means joining against a table nobody maintained as a source of truth.

I would also not key retries on the withdrawal id again. It feels natural (one withdrawal, one send) but it quietly assumes the client cannot generate two withdrawals for the same intent. Clients under a timeout do exactly that. The key has to come from the caller, because the caller is the only party that knows whether this is a new intent or the same one again.

Write the three identities down explicitly when you design this: the approver decides whether the request may leave requested, the request decides which transitions are legal, and the client's request key decides how many times the rail is touched. None of them is the click.

What is left is a pipeline in which the interesting decisions are values you can log, replay and test without a chain, a bank, or a person in the room. The senders are the easy part.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

Parse crypto amounts like hostile input, declare rounding first

polycratia — Sat, 12 Sep 2026 13:21:43 +0000

Every crypto amount enters a system as a string: a form field, a JSON body from an exchange API, a row in a payout file, a webhook from a provider. Between that string and the ledger two decisions get made: whether the amount is representable in the asset at all, and what happens when it is not. In most codebases the first decision is skipped and the second one is implicit, so the amount becomes whatever the last arithmetic operation, or the database column, happened to leave behind. That is most of the bug class, I think. Precision belongs to the asset, and rounding is a policy you declare before you compute, not a residue you discover afterwards.

I have been building payment and crypto systems in production since 2018: custodial wallets for BTC and ETH, a fiat-to-crypto onramp, exchange order books, stablecoin rails in daily use. The amounts that gave me the most trouble were not the large ones. They were the ones that arrived with one digit too many and were quietly made to fit.

The parse is the last place there is still someone to ask

When a user types 0.000000001 into a BTC withdrawal field, that string carries information: they asked for something the asset cannot express. There are two honest responses, and both of them require knowing it happened. You can refuse and show them the problem, or you can round by a rule you decided on in advance and tell them what you did.

The response that is not available, though it is the one that happens most in practice, is to truncate it on the way into a NUMERIC(18, 8) column and move on. By the time the value reaches the ledger the user is gone, the request is over, and the difference between what was asked for and what was recorded has turned into dust that will show up much later as reconciliation drift nobody can attribute.

So in cryptomoney the precision check lives in the parser, and its default is refusal:

from cryptomoney import BTC, USDT, parse_amount, parse_money

parse_amount("0.5", BTC)          # 0.5 BTC
parse_amount(" 1 234.50 ", USDT)  # 1234.50 USDT
parse_amount("1.25e3", USDT)      # 1250 USDT
parse_money("0.5 BTC")            # 0.5 BTC
parse_money("12.5btc")            # 12.5 BTC
parse_money("BTC 0.5")            # 0.5 BTC

The input is treated as text written by someone who may not be careful and may not be friendly. Whitespace, a leading sign, thousands separators, exponent notation, a symbol before or after or glued to the number: all of it is accepted, because all of it occurs in real payloads. What is not accepted is text that does not describe exactly one amount of exactly one asset, or an amount finer than the asset:

from decimal import ROUND_DOWN

parse_amount("1e-9", BTC)                       # ParseError: needs 9 decimal places
parse_amount("1e-9", BTC, rounding=ROUND_DOWN)  # 0.00000000 BTC
parse_amount("NaN", BTC)                        # ParseError
parse_amount(0.5, BTC)                          # TypeError: float is refused
parse_money("0.5 XMR")                          # UnknownAsset

The second line is where the whole design sits. Rounding is possible, it is just not free: you name the mode, at the call site, in the code path where you know what the amount means. A quote engine that floors a displayed rate and a withdrawal endpoint that must not create value out of nothing are different call sites with different answers, and neither of them should inherit a default written by a library author who has never seen either.

Floats are refused because the float already lost the evidence

parse_amount(0.5, BTC) raising TypeError looks pedantic until you ask where the float came from. It came from a JSON decoder that turned the sender's text into a binary double before your code ever saw it. The original digits are gone at that point, and with them your ability to say whether the sender wrote a representable amount or not. A parser that accepts floats is not parsing, it is laundering a decision that was already made badly upstream.

The same rule holds at construction, so a rounding error cannot enter a balance through a different door:

from decimal import Decimal

from cryptomoney import BTC, ETH, Money

Money(Decimal("0.5"), BTC)         # 0.5 BTC
Money("0.000125", BTC)             # 0.000125 BTC
Money(0.1, BTC)                    # TypeError: float is refused
Money("0.000000001", BTC)          # ValueError: BTC is divisible into 8 decimal places
Money("1", BTC) + Money("1", ETH)  # CurrencyMismatch

In practice this means the boundary code has to hand over the raw text. For HTTP that is a decoder configured to keep numbers as strings, or a schema that types the field as a string. A small amount of friction in exactly one place, and what you get for it is the guarantee that every Money in the system was checked against its asset when it was born.

Still on the subject of hostile input: the parser also caps input length and bounds the exponent range, because an unbounded exponent in decimal arithmetic is a way to make a single field allocate an enormous number. Any parser that will run against public input needs the equivalent.

Precision is a property of the asset, so the asset has to be a value

The check has to compare against something. In this library that something is not a constant in the codebase but a small frozen value object:

@dataclass(frozen=True, slots=True)
class Asset:
    """A crypto asset and the number of decimal places it is divisible into."""

    symbol: str
    decimals: int

Its constructor validates what it is given: the symbol must be a non-empty, unpadded string, and decimals must be a real int (a bool is explicitly rejected) within a fixed upper bound. An Asset that exists is an Asset that makes sense, which means the precision rule cannot be None at the moment a parse needs it.

The package ships a registry of common assets, and that registry is a convenience rather than an authority. The same ticker has different precision on different chains, and a system that hardcodes six decimals for USDT will eventually meet an eighteen-decimal deployment of it:

from cryptomoney import ASSETS, Asset, AssetRegistry

assets = ASSETS.copy()
assets.register(Asset("USDT", 18), replace=True)  # USDT on BNB Smart Chain
assets.register(Asset("XMR", 12))

own = AssetRegistry([Asset("POINTS", 0)])         # or start from nothing

parse_money looks symbols up in the default registry unless you pass assets=, and a symbol that is not registered raises UnknownAsset instead of guessing. An unknown ticker is a configuration gap, and the worst thing a parser can do with a configuration gap is invent a precision for it.

Rounding has no default anywhere, not just in the parser

If rounding is a declared policy at the edge and an accident inside, you have moved the bug rather than fixed it. So the same rule runs through the arithmetic. Operations that are exact are plain operators. Operations that may not fit the asset's precision are methods that require a mode:

from decimal import ROUND_DOWN, ROUND_HALF_UP

from cryptomoney import BTC, Money

Money("0.5", BTC) / 3                                      # TypeError: / is refused
Money("0.5", BTC).divide(3, rounding=ROUND_DOWN)           # 0.16666666 BTC
Money("1", BTC).multiply("0.015", rounding=ROUND_HALF_UP)  # 0.01500000 BTC

Refusing / is the part people argue about, and it is the part I would keep. Division is where the money disappears, and an operator is a syntax that invites you not to think. A method with a mandatory keyword argument puts the policy where the reviewer sees it, in the diff.

Quantized division still loses the remainder, which is correct for a fee and wrong for a distribution. When the total has to survive (splitting a settlement across investors, dividing a batch payout) quantization is the wrong tool entirely, and the operation works in base units instead:

shares = Money("0.00000010", BTC).split(3)
[str(share) for share in shares]   # ['0.00000004 BTC', '0.00000003 BTC', '0.00000003 BTC']
sum(shares[1:], shares[0])         # 0.00000010 BTC

The remainder goes to the first shares and the parts add back up to the original. That property is worth more than any fairness heuristic, because it is the one an auditor checks.

The payoff for refusing unrepresentable amounts at the parse arrives at the chain boundary. Chain APIs speak integers (satoshi, wei), and conversion in both directions is exact precisely because an amount finer than its asset never existed in the first place:

from cryptomoney import BTC, USDT, Money, from_wei, to_satoshi

to_satoshi(Money("0.5", BTC))            # 50000000
from_wei(1)                              # 0.000000000000000001 ETH

Money("12.5", USDT).to_base_units()      # 12500000
Money.from_base_units(12500000, USDT)    # 12.500000 USDT

There is no rounding mode on those calls because there is nothing to round. The refusal at the edge is what makes the conversion at the other edge total.

What I would do differently

Earlier systems I built put the leniency at the front and the truncation at the back: accept whatever the client sends, coerce it into a decimal, let the column width decide the precision. It reads as robustness. It is really a silent policy decision made by a schema migration, applied uniformly to fees, quotes, payouts and refunds, none of which want the same rule. The failure does not announce itself as a parse error. It announces itself as amounts that do not reconcile, in the smallest units, long after the request that created them.

The inversion I would now apply from the first commit is small: the only way to construct an amount is from text plus an asset, precision violations raise by default, and every operation that cannot be exact takes a rounding mode with no fallback. It costs a few explicit arguments at call sites. In exchange every rounding decision in the system is greppable, and the amounts that could not be represented were rejected while there was still someone on the other end of the request to tell...

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

Issuing deposit addresses without holding a private key

polycratia — Fri, 11 Sep 2026 13:21:43 +0000

A custodial service that shows a user a deposit address has to answer a question the user does not ask: which process produced that address, and what else can that process do? If the answer is that the node generated it, then the component reachable from an HTTP request is also the component that can spend. Deriving deposit addresses from an account-level extended public key splits those two jobs apart. The issuer walks derivation paths and encodes public keys, and it holds nothing that could move a coin.

What asking the node for an address couples together

Asking a node for a fresh address is the shortest path to a working deposit flow, and it hands the node three jobs at once. It owns the keys. It owns the address gap, meaning how many addresses have been issued and how far ahead the wallet is willing to look for funds. It also owns the recovery story, which becomes a wallet file you restore and cannot verify by reading.

None of that has to be a node problem. Which index was handed to which user is application state, and it belongs in the same database as the user. How far ahead the issuer may go before it refuses is an issuance policy, and the gap limit is where that policy gets written down. Recovery, done properly, is re-derivation from a seed that never touched the server.

The first job is the one that shapes the architecture. If issuing an address is a call into something that can sign, then every route that reaches the issuer is a route that reaches spending authority, and the only thing standing in between is configuration: an unlocked wallet, an RPC allowlist, a method filter. Configuration can be wrong for months without anything failing visibly... That is the property I want to remove, not tighten.

The boundary is arithmetic, not configuration

An extended public key can derive its non-hardened children and nothing else. Hardened derivation mixes the private key into the child, so a process holding only public material cannot perform it. Not because it is forbidden, but because the input does not exist. That turns a policy into a fact about the code path, and a fact is reviewable.

This is the whole interface of chain-addresses (https://github.com/polycratia/chain-addresses), a small package I maintain for exactly this job:

from chain_addresses import ExtendedPublicKey, get_encoder

account = ExtendedPublicKey.parse(account_xpub)
deposit = account.derive("0/17")

encoder = get_encoder("bitcoin-p2wpkh")
encoder.encode(deposit.public_key)   # bc1q...

Nothing in the package accepts or stores a private key. Passing "0'/17" or an xprv raises instead of silently doing something else: a hardened path and a private key are both refused at the edge rather than half-handled. The signer stays offline and exports one account-level extended public key, and the service that answers "give me a deposit address" imports this and nothing else.

The reason to care about the refusal rather than about a convention is code review. "The issuer cannot sign" becomes something you confirm by reading imports, instead of a claim about how a node was configured on a host you are not looking at.

One key, many address formats

Every encoder in the package takes a 33-byte compressed public key and returns a string. Which encoding that is (Base58Check, bech32, bech32m, or EIP-55 hex) stays inside the encoder:

from chain_addresses import ENCODERS

child = account.derive("0/17")
addresses = {name: e.encode(child.public_key) for name, e in ENCODERS.items()}

That maps one derived key to bitcoin-p2pkh, bitcoin-p2sh, bitcoin-p2wpkh, bitcoin-p2tr, their testnet variants, and evm. The package's own test asserts that all of those addresses come out distinct, and that is the part worth internalising: the address format is a presentation decision downstream of derivation, not a key decision. Moving deposits from P2SH-wrapped segwit to native segwit, or adding taproot, does not touch the account key, the derivation policy, or the offline signer's export. It changes which script the signer will later have to satisfy, and nothing before that.

The evm encoder is the sharper case. There is exactly one of it because an EVM address carries no chain identity at all, so the same key produces the same twenty bytes on every EVM chain. Your database owns the chain and the address does not. A deposit record without a chain field is unfinished, and a user who sends a stablecoin on a network you are not watching has sent it to an address that is genuinely theirs, on a chain nobody is polling. On the stablecoin rails I have run in production this is a support queue, not a thought experiment. The issuer cannot prevent it, but it can refuse to render an address unless the deposit intent named the chain it is for.

Re-derivation is a free integrity check

Because the issuer holds no secret, everything it does can be repeated anywhere: in a second process, in another language, in a test. That changes what a stored address is. It is a cache of a pure function of account key, path and format, rather than a fact you have to preserve.

Application-side, issuance is two lines and a database write:

def derive_address(account: ExtendedPublicKey, path: str, address_format: str) -> str:
    child = account.derive(path)
    return get_encoder(address_format).encode(child.public_key)

Store the path and the format next to the address. A periodic job can then walk the table, re-derive, and compare. A mismatch means the configured account key is not the one that row was issued under, and that is the failure I actually worry about.

It has no natural signal. A wrong extended public key in configuration produces perfectly valid, perfectly checksummed addresses for a wallet you cannot spend from, and there is nothing in the string to inspect. Two cheap defences, both built from what the extended key already carries: check at boot that the parsed key's depth and parent_fingerprint match what the offline signer exported, and keep one known index (an address the signer itself produced during setup) as a fixture the issuer must reproduce before it serves traffic.

The package takes the same approach internally. Its test suite carries its own private-key derivation and checks that public derivation reaches the same children, so the public-only path is verified against the private one it is meant to replace, and the encoders are checked against published vectors rather than against themselves. It is pre-alpha, and honest about the edges: BIP32 public derivation and the address formats above are implemented, while chain metadata and gap-limit scanning are not written yet.

What I would do differently

On the first custodial wallet work I did, the node handed out addresses because that was the fast path and deriving them myself was an afternoon of work. The cost did not arrive as an incident. It arrived as a permanent constraint: the deposit service could not be deployed anywhere the keys could not go, so its blast radius quietly set the security posture of everything sitting next to it.

Two things I would set up from the start now. First, treat the account extended public key as a pinned configuration object verified at boot, using depth, parent fingerprint, and one reproduced fixture address, rather than a string that gets trusted the first time somebody requests a deposit. Second, keep the issuer format-agnostic from day one. Per-chain issuer services look reasonable until the third chain, and at that point you probably have three copies of the index-allocation logic and three separate ways to be wrong about which address was handed to whom.

Close

HD derivation being elegant is beside the point for me. The property worth having is that a service can hand out an unbounded number of deposit addresses while being unable to spend from any of them, and that this reduces to two things a reviewer can check in an afternoon: the issuer only ever sees an extended public key, and non-hardened derivation is the only derivation it is capable of performing.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

ERC-20 amounts are meaningless without the token's decimals

polycratia — Thu, 10 Sep 2026 13:25:30 +0000

An ERC-20 balance is a uint256 counting the token's smallest unit, and nothing in that integer tells you how many units make one token. You have to ask the contract. Assume eighteen decimals against a six-decimal token and every amount you compute is off by a factor of a million, with no revert, no exception, and a perfectly valid integer on both sides of the mistake.

I have run stablecoin payment rails in production long enough to stop treating this as a display concern. It is not formatting. It is the gap between a payout of one and a half dollars and a payout of one and a half million, written in code that type-checks either way.

The chain stores units, your product stores amounts

The whole problem fits in one table. These are the tokens most payment systems actually touch:

Token	Decimals	1.5 tokens, in units
USDT	6	1500000
USDC	6	1500000
WBTC	8	150000000
DAI	18	1500000000000000000

Every one of those integers is a legal uint256. If your encoder scales 1.5 by 10**18 and hands the result to USDT, the call data is well-formed, the ABI encoding is correct, the transaction is valid, and the transfer either moves a million times too much or reverts on an insufficient balance. Which of the two you get depends on how well funded the sender happens to be. That is the genuinely dangerous part: the failure mode is not deterministic. On a thin hot wallet it looks like a funding bug. On a well-funded one it looks like nothing at all until reconciliation runs.

No invariant in your system catches it either. There is no checksum on scale. A number does not describe itself: 1500000000000000000 is a valid amount of some token, just not of this one.

Decimals are a property of the contract, and optional at that

In the ERC-20 standard, decimals is an optional method. Most tokens implement it, tokens are free not to, and the value that comes back is the token's business rather than a convention you can assume. Six for the dollar stablecoins, eighteen for DAI and most of the long tail, eight for WBTC because it inherited Bitcoin's granularity. There is no default.

So it gets read. In erc20-transfers (https://github.com/polycratia/erc20-transfers) that read is one call, encoded and decoded like every other:

from erc20_transfers import decode_decimals, encode_decimals

decimals = decode_decimals(eth_call(token, encode_decimals()))

What the library deliberately does not do is remember the answer for you. decimals is a required argument on every conversion. That looks like friction until you price the alternative: a library that caches on your behalf owns a cache whose key it cannot see. It does not know which chain you are on, whether the address you passed is the token you think it is, or whether your process lives long enough for the cache to matter. Making the parameter explicit pushes that decision down to the only layer with enough context to make it.

In my own services that layer is the token registry, and the cache key is the pair that actually identifies a contract:

_decimals: dict[tuple[int, str], int] = {}


def decimals_for(chain_id: int, token: str) -> int:
    key = (chain_id, token.lower())
    if key not in _decimals:
        _decimals[key] = decode_decimals(eth_call(token, encode_decimals()))
    return _decimals[key]

Not the symbol. Symbols are not unique, are not stable, and are the first thing a counterfeit token copies. The same three letters on two chains are two contracts with independent decimals, and the cheap way to learn this is to key your registry on the address before you find out the expensive way.

Convert at the edges, and only at the edges

Once decimals stop being a constant, the interesting question is where in the system they get to appear. My answer, after enough rewrites of the same code: in exactly two places, both of them boundaries with a human on the other side.

Input is the first one. A person types 1.5, and that string means nothing until it is bound to a token:

from decimal import Decimal

from erc20_transfers import encode_transfer, to_units

amount = to_units(Decimal("1.5"), decimals=decimals)  # 1500000 on USDT
data = encode_transfer(to=bob, amount=amount)

Display is the second one, and it needs the same pairing to read a balance back off the chain:

from erc20_transfers import TokenAmount, encode_balance_of

balance = TokenAmount.from_return_data(
    eth_call(token, encode_balance_of(account=alice)), decimals=decimals
)
print(balance.amount, balance.units)  # 1.500000 1500000

Between those two edges, nothing needs decimals, and that is the actual payoff of the discipline. Comparing units to units is scale-free. check_allowance takes a required and a current and never asks what a token is; measure_received compares a balance before against a balance after; the amount inside encode_checked_transfer_from is an integer of the smallest unit. An internal function that handles money in units cannot be given the wrong scale, because it does no scaling at all.

The corollary is a lint rule you can apply by reading: any internal function whose signature takes an amount as a Decimal or a float, with no token beside it, is a scale bug waiting for the wrong token to arrive. The amount and the token identity travel together, or the amount is not yet meaningful.

Refusing is better than rounding

The boundary has a second job, and it is the one most homegrown helpers get wrong. Not every decimal number is representable in a given token. Six decimals cannot hold a seventh digit. The convenient behaviour is to round it away. The correct behaviour is to refuse:

to_units(Decimal("1.0000005"), decimals=6)  # raises

A silently dropped digit is a loss that reconciles to nothing. It is tiny, half a millionth of a token, and it happens on the leg where you convert a price, an exchange rate or a fee percentage into a transfer. Do it on every payout and you get a slow drift between what your ledger recorded and what the chain moved, with no single transaction anyone can point at. Raising turns that into an input validation problem, visible at the top of the call stack, where a human decides whether to round up, round down, or reject the request. That is a product decision and it does not belong inside a units helper.

The same reasoning is why floats are refused outright. 0.1 is not 0.1 in binary, and a value that is already approximate does not become exact by being multiplied by a power of ten. Decimal in, integer out, or an exception.

What I would do differently

Three things, all learned by not doing them first.

Read decimals when a token is admitted to the system, not when a transfer is about to go out. Onboarding is a place where a failed contract call is an operator's problem and the token simply does not become available. The payout path is not: there, an RPC hiccup on a metadata read has no good outcome, and whatever fallback you write under deadline pressure will probably turn into the constant you were trying to avoid.

Treat a token that does not answer decimals as unsupported rather than as eighteen. The method is optional in the standard, so a missing answer is legitimate, but defaulting is guessing, and guessing about scale is the one guess with a million-fold error bar.

Store the decimals you used alongside every recorded amount. Contracts are immutable and in practice decimals do not change, but your registry row can be corrected, re-imported, or written by an earlier version of your code. If a historical payout carries the scale it was computed with, a bad registry row is a display artefact you can fix. If it does not, every amount you ever recorded hangs off a mutable row, and you have to trust that row to read your own books.

None of this is difficult. It is the difference between an amount and a number, held consistently, at two boundaries instead of everywhere...

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

Idempotency belongs in the ledger, not in every caller

polycratia — Wed, 09 Sep 2026 13:25:30 +0000

A payment callback that arrives twice must not move money twice. The usual answer is a unique constraint on the entry id, which turns the second attempt into an error, and an error is not the same thing as knowing the first attempt landed. That gap is where duplicate postings come from.

I have been building payment and custodial systems since 2018, and the shape of this bug has not changed once in that time. A provider redelivers a deposit callback. A queue redelivers what it already delivered. An HTTP client gives up at thirty seconds on a write that committed at thirty-one. In every one of those cases the caller ends up asking the ledger for the same movement a second time, and the ledger has to answer without moving anything again.

A refusal is not an answer

The first defence you reach for is identity. Give the entry a deterministic id, put a unique index behind it, let the second write fail. ledger-core does exactly this at the journal level: a duplicate id is rejected, and Journal.extend applies the same rule to a batch, so if any entry in it is refused, none of the batch is kept.

from datetime import datetime, timezone
from decimal import Decimal

from ledger_core import Account, AccountType, Entry, Journal, Money

cash = Account("cash", AccountType.ASSET, "EUR")
customer = Account("customer:42", AccountType.LIABILITY, "EUR")

deposit = Entry.transfer(
    entry_id="e-1",
    occurred_at=datetime.now(timezone.utc),
    debit=cash,
    credit=customer,
    amount=Money(Decimal("25.00"), "EUR"),
    memo="card deposit",
)

journal = Journal()
journal.append(deposit)
journal.append(deposit)  # refused: the id is taken

Necessary rule, insufficient one. Identity protects the journal and does nothing for the caller. When the second append fails, the caller is holding an exception that fits at least three different worlds: my earlier attempt landed and this is my own retry, my earlier attempt never ran and something else is sitting on this id, my earlier attempt landed but wrote something other than what I am holding now. The exception does not separate them, and the caller needs that separation before it can answer the provider with a success.

So the caller writes recovery code. Catch the error, read the entry back, compare it field by field against what it meant to write, decide whether the write counts as done. That is idempotency logic, and once it exists it exists in the deposit handler, and then again in the reconciliation job that reposts stuck movements, and again in the admin tool an operator uses at two in the morning. Three copies, each subtly different, each written under different pressure. The one in the admin tool is the one that will be wrong.

The key is the question the caller asked

Stop treating the retry as a collision and treat it as a repeated question. The entry id says which posting this is. The operation key says which write request this is, and the same request asked twice is one write.

from ledger_core import Operations

operations = Operations(journal)

def record_deposit(event_id: str, entry: Entry) -> Entry:
    return operations.post(f"psp-deposit:{event_id}", entry)

post writes the entry the first time and, every time after that, returns the entry the first call wrote, read back out of the journal rather than out of a cache of whatever the caller happened to pass in. post_many does the same for a batch under one key, under the journal's all-or-none rule.

The return type carries the design. post hands you back an Entry, not a boolean and not None. A caller written against it cannot branch on whether the write was new, because that fact is never offered. It gets the postings that are in the journal under its key, which is the only thing it ever needed in order to answer the provider. Retry-awareness stops being a code path and becomes the ordinary path.

Same key, different work

A key that only stored "this key was used" would be a trap. Keys get derived badly: someone keys a payout by user id and calendar day, someone reuses a request id across two movements, and a naive dedup table would answer the second, genuinely different write with the first write's entries and no complaint. Money would go missing quietly.

Operations stores a fingerprint alongside the key: a stable digest over each entry's id, timestamp, memo, correction target, every posting's account, amount, currency and side, plus the entry's metadata in sorted order. The amount is normalized before it is hashed, so a retry that rebuilt 25.0 where the first attempt built 25.00 is recognised as the same work instead of a conflict. Applying a settled key to different work is refused:

from ledger_core import OperationMismatch

try:
    operations.post("psp-deposit:evt_88", other_entry)
except OperationMismatch:
    # this key already carries different postings; the derivation is wrong,
    # not the retry
    raise

The fingerprint turns a class of silent corruption into a loud failure at the boundary. It also has a consequence worth knowing before you deploy it, and I will come back to that...

The record has to outlive the process

Deduplication that lives only in memory is deduplication with a hole in it the width of every restart. A deploy, a crash, an autoscaler killing a pod, and the key that was settled two seconds ago is unknown again, right when the provider is still retrying.

Operations keeps its records as plain frozen values, so you can store them beside the journal and hand them back:

records = operations.operations()   # every key settled so far
# ... persist records, restart, reload the journal ...
operations = Operations(journal, records)

restore refuses a record naming entries the journal does not hold, because such a record describes some other journal, and settling a key against a write nobody made is worse than having no record at all. So the record and the entries have to be persisted together, atomically. If they can diverge, you have two truths about the same write, and the whole point of the key was to have one.

The library itself keeps the journal in memory, and persistence stays the host application's job. What the library fixes is the shape: because an Operation is a value with a key, its entry ids, its fingerprint and a timestamp, storing it is a row rather than a serialization problem, and the same key settled in one process stays settled in the next.

What I would do differently

Two things, both learned the annoying way.

Derive the key at the edge, from something the upstream already owns. The provider's event id, the payout request id, the settlement batch id. Do not generate it inside the retry loop: a key built from uuid4() or from datetime.now() at the top of the handler is a new key on every attempt, which is a dedup mechanism that deduplicates nothing while looking like it works.

Capture the entry the same way. The fingerprint covers entry_id and occurred_at, which means the retry has to rebuild the same entry, not an equivalent one. A handler that stamps datetime.now(timezone.utc) fresh on each attempt produces a different fingerprint and gets an OperationMismatch where it expected its original postings back, technically the system protecting itself, practically a page at three in the morning. Decide the timestamp and the entry id once, when the request first arrives, and carry them through every retry alongside the key. Deterministic ids and a captured timestamp cost nothing at write time and remove a whole category of incident.

The underlying move is small and it probably generalises past ledgers: when a write can be asked for twice, make the second answer be the first result. Refusing the duplicate protects your data and leaves the caller guessing. Returning the original protects your data and ends the conversation. Only one of those keeps the guessing out of six different call sites.

The code is at https://github.com/polycratia/ledger-core.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

under_investigation is a status, not an excuse

polycratia — Tue, 08 Sep 2026 13:21:26 +0000

A scanner tells you a component in your build carries a known advisory. It cannot tell you whether the vulnerable code is reachable in your product, and that judgement is the entire content of a VEX document. So the interesting design question is not how to express affected or not_affected. It is what your tooling does with the findings nobody has judged yet. The common answer, defaulting them to not_affected because the release is on Friday, turns a triage backlog into a signed assurance.

I build payments and crypto backends, so I ship services whose dependency lists are long, boring, and audited by people who did not write them. The Cyber Resilience Act expects manufacturers to answer the reachability question quickly and in writing for every product they ship, which is a schedule problem before it is a security problem. Schedule problems are where defaults do their damage: nobody decides to publish a false claim, they decide not to block the release, and the default publishes the claim for them.

What the default actually asserts

not_affected is a claim about your code. It says: I looked, and the vulnerable path is not in the execute path, or the component is not present, or the adversary cannot control the input. under_investigation is a claim about your process. It says: this is in the queue and it has not been answered.

Both are publishable. Only one of them can be falsified by an attacker with a weekend and a debugger, and it is not the honest one.

The consequence shows up in the shape of the document rather than in any single statement. If unreviewed findings default to not_affected, the number of not_affected statements stops tracking how much review happened and starts tracking how big your SBOM is. A downstream reader (a customer's security team, an auditor, the person who inherits the service) cannot tell eighty components somebody worked through from eighty components nobody opened. That distinction is the only reason the document exists.

The other failure mode is quieter: dropping undecided findings entirely, so they simply do not appear. Absence is worse than an honest status, because absence is ambiguous. Did the matcher not find this advisory, or did it find it and get ignored?

Three kinds of unknown, each with a reason attached

When I wrote vexdesk, the rule I started from was that nothing which cannot be determined gets quietly cleared. In practice "cannot be determined" is not one thing, and the reasons are not interchangeable:

$ vexdesk match -sbom sbom.cyclonedx.json -advisories ./advisories
4 component(s) compared against the advisory set

STATUS    ADVISORY      COMPONENT        VERSION  REASON
affected  FIXTURE-0001  widget           1.2.3    version falls inside the advisory's affected range
affected  FIXTURE-0004  cogwheel         4.0.0    version is in the advisory's affected version list
unknown   FIXTURE-0003  Example_Fixture  3.1.2    ECOSYSTEM range for PyPI needs that ecosystem's own version ordering, which is not implemented

Not checked (1):
  vendored-blob  no package URL: nothing to look up

Three different human actions hide behind those lines. A component with no package URL cannot be looked up in any advisory database, so somebody has to identify that vendored blob by hand, and no amount of tooling will do it for them. A PyPI ECOSYSTEM range needs PEP 440 ordering, which the tool does not implement; that is a gap in my matcher, not a property of your product, and the right response is to compare the version manually or fix the matcher. An affected line is a reachability question for whoever owns that code path.

The reason string is the payload here, not the status. An unknown with no reason is indistinguishable from a bug in the tool, and a status a reviewer cannot interrogate is a status they will either rubber-stamp or ignore. The same rule shapes version parsing: 2023-08-01 is not read as major version 2023, it is refused. A date that silently outranks every real version produces a confident wrong answer, which is the most expensive output a matcher can produce, worse than no answer, because it terminates the conversation.

The absence of a decision is itself a statement

Decisions live in a file, separate from the matcher, because they are human output and the match is machine output:

{
  "decisions": [
    {
      "vulnerability": "FIXTURE-0001",
      "product": "pkg:golang/github.com/example/widget@v1.2.3",
      "status": "not_affected",
      "justification": "vulnerable_code_not_in_execute_path",
      "impact_statement": "the affected parser is only reached from the admin importer, which this build does not include"
    }
  ]
}

The part that matters is what happens to the findings this file does not mention. They are not dropped and they are not cleared: they are emitted as under_investigation. The undecided finding survives into the published document as an undecided finding, and the document stays a rendering of the triage queue rather than a summary of the parts of it somebody got around to.

This also removes the incentive that produces bad not_affected statements. not_affected requires one of the five OpenVEX justification codes (component_not_present, vulnerable_code_not_present, vulnerable_code_not_in_execute_path, vulnerable_code_cannot_be_controlled_by_adversary, inline_mitigations_already_exist) and the document refuses to build without one. affected requires an action statement: what should the user do. under_investigation requires nothing, because there is nothing to say yet.

That asymmetry is deliberate. If the only way to make the build pass were to write a justification, engineers under deadline would pick the code that sounds closest to true, and a reviewer six months later would have no way to tell a considered vulnerable_code_not_in_execute_path from a guessed one. Leaving one status available at zero prose cost means the pressure valve is honesty rather than a plausible code.

A status that survives review carries its why

A VEX statement is read by someone who was not in the room. That reader has exactly two questions: what did you conclude, and on what basis. The justification code answers the second in a form they can compare against every other statement you have made, which is the whole reason to require a code alongside the prose rather than prose alone. Free text is unanswerable at scale. A code is sortable.

The same logic applies to the document as an artefact. In vexdesk the document id is derived from the statements, so an unchanged set of decisions rebuilds to the same id instead of looking newly issued on every CI run. That sounds cosmetic until you try to review a burn-down. If the id churns on every build, no consumer can tell a re-publish from a re-decision, and a finding flipping from under_investigation to not_affected, the single most important event in this whole workflow, disappears into noise. Document comparison lives in its own package in that repository for the same reason: the interesting object is the delta between two documents, not either document alone.

And because match exits 1 when anything needs attention, the backlog can gate a pipeline rather than sit in a wiki:

#!/usr/bin/env bash
set -euo pipefail

vexdesk match -sbom sbom.cyclonedx.json -advisories ./advisories || echo "findings need review"

vexdesk vex -sbom sbom.cyclonedx.json -advisories ./advisories \
            -decisions decisions.json -author "Example Ltd" -o vex.json

Note the shape: the match result informs, the document still builds. A gate that refuses to produce a VEX document until every finding is resolved gets disabled in a week. A gate that publishes the truth, including the unresolved parts, survives contact with a release schedule, which is probably the only property that matters, because a control nobody can ship past is a control nobody keeps.

What I would do differently

under_investigation has a half-life. It is honest on the day you publish it and it is an indictment three months later, and nothing in the document as it stands expresses that difference. A statement carries what you concluded, not when you first saw the finding, so a stale queue and a fresh one look identical to a reader.

If I were extending this, that is where I would go next: record when a finding first appeared undecided, and let the age of an under_investigation statement be visible to the person reading it. Not as a gate, since an aging finding is not automatically a problem and plenty of them are waiting on an upstream fix, but as the thing a reviewer should look at first. That is not built, and I would rather say so than describe it as though it were.

The rest of the design I would keep unchanged. A tool that refuses to answer a question it cannot answer is more useful than one that answers everything, because the second kind trains you to stop reading its output. under_investigation is not the tool admitting defeat. It is the one status in the vocabulary that is always available and never a lie, and a document that uses it freely is a document a reviewer can actually work with.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.