DEV Community: polycratia

A testnet xpub under a mainnet encoder: addresses nobody scans

polycratia — Sat, 24 Oct 2026 13:27:50 +0000

A deposit service pairs an extended public key with an address encoder, and there is exactly one way to get that pair wrong that produces no error at any layer. A testnet key under a mainnet encoder parses cleanly, derives cleanly, and hands back a well-formed mainnet address with a valid checksum, while the signer holding the seed watches a different chain and never sees the deposit. The defect is not in the key. It is not in the address either. It sits in the pair, which is why the pair is the only place a check can live.

The failure that reports success

Three steps stand between an account-level key and an address a customer pastes into a wallet, and each one has a narrow, honest contract.

Parsing validates structure. In bip32.py, ExtendedPublicKey.parse rejects a payload that is not 78 bytes, rejects the six private versions by name, and rejects any version not in PUBLIC_VERSIONS; __post_init__ then checks depth, fingerprint length, chain code length and the 33-byte compressed point. A tpub passes all of it, because a tpub is a perfectly valid extended public key.

Derivation is arithmetic. A chain code and a curve point go in, a child point comes out. There is no network anywhere in that computation: the same scalar addition serves every chain that runs on secp256k1.

Encoding is a lookup and a string. The AddressEncoder protocol in addresses.py has two members, a name and encode(public_key) -> str. P2WPKH takes a 33-byte compressed key, hashes it, and emits bech32 under its own hrp. It does not ask where the key came from, and that is what the protocol buys you: code holding an encoder never has to know which chain it is serving.

So the mismatch is invisible to every step on its own. It is also invisible to address validation after the fact, and that part is worth sitting with. Hand the resulting string to validate_address with network="mainnet" and it comes back valid, because it genuinely is a well-formed mainnet address. Nothing is wrong with the address. The wrong part is that no signer on mainnet derived it, so no scanner on mainnet is looking for it. Validating the output cannot catch a defect that exists only in the relationship between two inputs.

Where each network is written down

Both facts are present, just sitting on different objects.

On the key side, the fact is the version prefix. PUBLIC_VERSIONS maps the six serialisation versions to xpub, ypub, zpub, tpub, upub and vpub, and gap.py maps those names to networks:

_KEY_NETWORKS = {
    "xpub": Network.MAINNET,
    "ypub": Network.MAINNET,
    "zpub": Network.MAINNET,
    "tpub": Network.TESTNET,
    "upub": Network.TESTNET,
    "vpub": Network.TESTNET,
}

On the encoder side, the fact is the name. The registry is flat and the names carry their own scope (bitcoin-p2wpkh, bitcoin-testnet-p2wpkh, evm), so you can read the network straight off the string:

def _encoder_network(encoder: AddressEncoder) -> Network:
    if not encoder.name.startswith(_BITCOIN_PREFIX):
        return Network.ANY
    if encoder.name.startswith(_TESTNET_PREFIX):
        return Network.TESTNET
    return Network.MAINNET


def _check_networks_agree(
    account: ExtendedPublicKey, encoder: AddressEncoder
) -> None:
    prefix = PUBLIC_VERSIONS[account.version]
    key_network = _KEY_NETWORKS[prefix]
    encoder_network = _encoder_network(encoder)
    if encoder_network is Network.ANY or encoder_network is key_network:
        return
    raise LedgerError(...)

The PUBLIC_VERSIONS lookup is unguarded on purpose: __post_init__ has already refused any version that is not in that table, so by the time a key reaches the ledger its prefix is known to exist. An invariant established at construction is worth spending, not re-checking.

The message on that LedgerError names the prefix, both networks, and the encoder. That is not padding. A network mismatch is a configuration mistake, and whoever fixes it is reading a log line while deciding whether to roll back. tpub alone does not say what it was paired with, and network mismatch says nothing at all.

Networks, not script types

The comparison stops at networks, deliberately. A zpub is allowed to drive bitcoin-p2pkh, and that is not an oversight.

Version bytes describe the address format the exporting signer expects, not the derivation. The chain code and the point under a zpub are the same chain code and point you would get under an xpub at the same path; the prefix is a hint about the signer's preference, carried along in the serialisation. A key can produce every format in the table, and sometimes it has to. A signer that only exports one prefix, or a service still issuing legacy addresses for an old integration, is a legitimate setup rather than a mistake. Refusing it would turn the check into an obstacle, and an obstacle that refuses correct configurations is the kind of check people learn to route around.

Network is a different sort of claim. It is not a preference about formatting, it is a statement about which chain the watch-only wallet on the other side is scanning. Get it wrong and you do not get an ugly address, you get an unscanned one.

Network.ANY is the same reasoning applied to EVM chains. The evm encoder takes the key's x and y coordinates and produces EIP-55 hex; nothing in that path distinguishes one EVM network from another, so there is no claim to contradict and either key is accepted. validate.py treats the same format the same way: EIP-55 hex belongs to Network.ANY and passes any network check.

The check runs before the first derivation

In AddressLedger.__init__ the refusals are ordered, and the order is the design:

if gap_limit < 1:
    raise LedgerError(f"gap limit must be at least 1, got {gap_limit}")
_check_index(next_index, "next index")
if last_used is not None:
    _check_index(last_used, "last used index")
    if last_used >= next_index:
        raise LedgerError(...)
resolved = get_encoder(encoder) if isinstance(encoder, str) else encoder
_check_networks_agree(account, resolved)
_check_hardened_above(account, branch)
self._branch_key = account if branch is None else account.child(branch)

Policy arithmetic comes first, because it is free. Then get_encoder resolves a string name, raising AddressError for one it does not know: the network check needs a real encoder with a real name, so resolution has to come before agreement. Then the two refusals that are about safety rather than bookkeeping, networks agree and the key sits below a hardened step. Only after all of that does account.child(branch) run, which is the first moment any key material is touched.

The effect is that an AddressLedger cannot exist in a state where it would mint addresses nobody scans. No caller has to remember a check, no code path can skip it, and the failure arrives at the line that built the object, in configuration-loading code, with a stack trace pointing at the configuration.

The snapshot is a second door into the same room

A ledger that hands out addresses in order has state worth persisting: next_index, last_used, the gap limit, the branch, the account key, the format. So there is a restore path, and restore paths are where constructor invariants usually go to die. The tempting shape is to allocate the object and assign the fields back one at a time, which is fast, total, and bypasses every check above.

restore() instead requires account and format in the snapshot, raising LedgerError that lists whatever is missing, and then calls the constructor with the remaining values defaulted. Every refusal runs again on the way back in.

snapshot = {
    "account": account_xpub,
    "format": "bitcoin-p2wpkh",
    "next_index": 41,
    "last_used": 37,
}
book = AddressLedger.restore(snapshot)

Snapshots get edited apart from the code that wrote them. Someone copies a staging row into production, someone fixes a format by hand during an incident, someone restores a backup taken before a key rotation. Each of those can produce a stored pair that was never possible through the constructor, and a restore that trusts its input accepts all of them. With the check on the load path, a snapshot that disagrees with itself fails at startup. Without it, the first symptom is a customer deposit at an address no scanner reaches, found days later with nothing left to do about it.

All of this lives in chain-addresses, in gap.py alongside the gap-limit bookkeeping.

What I would do differently

In earlier custodial work the network was a deployment variable. One environment variable selected the key, another selected the address format, and nothing compared them, because on any single environment they were correct anyway: staging configured for testnet, production for mainnet, and the mismatch was a state no running process ever occupied. That is the shape of bug no test on one environment finds. Both halves are right, and only the combination is wrong.

What I would do earlier is make the network a property derived from the artifacts rather than declared next to them. The tpub already knows what it is. The encoder name already knows what it is. Once those two get read off the objects and compared, the mismatched pair stops being a configuration that happens to be correct today and becomes a configuration that cannot load.

The narrowing probably matters as much as the check. Comparing networks and not script types keeps the refusal aimed at the one disagreement that silently loses money, and leaves the formatting choices with the caller who knows which addresses the integration on the other end can read.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

No Patch Is Coming: PHP 7.4, 8.0 and 8.1 After PHP2Shell

polycratia — Sat, 24 Oct 2026 10:55:20 +0000

On 21 October 2026 CISA issued an alert and CERT-EU published advisory 2026-018, covering two remote code execution flaws in PHP core: CVE-2026-109618 and CVE-2026-109619, both rated CVSS 9.2. An unauthenticated attacker can run code on a web-facing PHP runtime. Emergency releases exist for PHP 8.5.12, 8.4.27, 8.3.36 and 8.2.35. For PHP 7.4, 8.0 and 8.1 there is no fix, and there will not be one. That second sentence is the whole story, and it is not a security story.

What happened

Both flaws sit in core, not in an extension and not in a framework. Per the php.net release notes and the two advisories, they reach every mainstream way of serving PHP over HTTP: mod_php, PHP-FPM, php-cgi, FrankenPHP, LiteSpeed. No configuration turns a web-facing runtime into a non-web-facing one, which is why CISA and CERT-EU wrote the same sentence: patch immediately. Exploit attempts have been reported.

Two days later, on 23 October, a third flaw was disclosed as CVE-2026-115560. It has no patch yet. The only published mitigation is one line of configuration:

; php.ini
session.upload_progress.enabled = 0

One piece of noise worth naming, because it wasted people's morning: details leaked on Hacker News before the official release, carrying an inflated CVSS 10.0. The official rating is 9.2. Operationally that difference is nothing. Unauthenticated remote code execution at 9.2 and unauthenticated remote code execution at 10.0 produce the same day of work. Argue about the decimal later, since the decimal has not once changed what you have to do.

What to do today if you stay on PHP

Four things, in this order.

Move to a patched build. The only versions carrying the fix are 8.5.12, 8.4.27, 8.3.36 and 8.2.35. If you are on 8.2 or newer, this is a package upgrade and a restart, and you should be done before lunch.

Set session.upload_progress.enabled = 0 for the third CVE, then verify the setting actually landed in the SAPI that serves traffic. The CLI ini file is frequently not the FPM ini file, and the mitigation in the wrong file is worth exactly zero.

Verify what your hosting vendor patched. Not the status page, the running binary. A status page saying 'PHP updated' is a marketing artifact: the version of the process answering port 443 is evidence. Ask for it explicitly, and ask which SAPI.

Then inventory every box, including the ones nobody calls a web server:

# each SAPI reports its own build; the web one is the one that gets exploited
php -v
php-fpm -v
php-cgi -v 2>/dev/null

# the running process, not the package manager's opinion
pid=$(systemctl show -p MainPID --value php8.2-fpm)
readlink -f /proc/$pid/exe

# confirm the mitigation in the ini set that FPM actually loads
php-fpm -i 2>/dev/null | grep -i upload_progress
grep -RIn 'upload_progress' /etc/php/

# the hosts people forget: cron, queue workers, build images
crontab -l | grep -i php
grep -RIn 'php' /etc/cron.d/
docker ps --format '{{.Image}}' | sort -u

A cron box running an unpatched build is still an unpatched build. It is less exposed, not safe, and it is usually the one holding credentials for everything else.

The part that matters more than the score

If you are on 7.4, 8.0 or 8.1, none of the above finishes the job. You can apply the one-line mitigation. You cannot apply the fix, because the fix was never built for your branch.

Here is what I want to be precise about: an unpatchable branch is not a security incident. It is a stack decision, made years ago, by not making it. Nobody approved staying on 7.4. The branch went out of support while the service kept working, and working software generates no tickets. PHP2Shell did not create that position. It published it.

And it will happen again. The structural fact about an end-of-life runtime is not that it has one unfixed hole today; it is that every future CVE lands on it with no upstream answer. You are no longer in a patch cycle. You are in a sequence of one-off mitigations, each one narrower than the last, each one depending on whether a workaround happens to exist for that particular bug. This one has a config flag. The next one may not.

So the honest question today is not 'how do I patch this'. It is: version jump, or stack jump?

Version jump or stack jump

Four signals, and you can read all four in an afternoon.

Size. Not line count, surface. How many HTTP entry points, scheduled jobs and external integrations does the service own? A service with twelve endpoints and one payment provider is a different animal from one with two hundred endpoints accumulated over eight years.

Glue versus rules. This is the decisive one. How much of the code is framework scaffolding, and how much is business logic that only exists here? Glue is cheap to replace: it is a solved shape, and you are rewriting something whose behaviour is documented. Business rules get expensive, because the rules are not written down anywhere except in the code, and half of them are bug-compatible with a client's expectations.

# rough shape: where do the lines actually live
find . -name '*.php' -not -path './vendor/*' -print0 | xargs -0 wc -l | tail -1
find . -path './vendor/*' -name '*.php' -print0 | xargs -0 wc -l | tail -1

# controllers and views are glue; count them separately from the rest
find . -path ./vendor -prune -o -name '*Controller*.php' -print | wc -l
find . -path ./vendor -prune -o -name '*.tpl' -o -name '*.phtml' -print | wc -l

# every place money, auth or an external party is touched
grep -RIl --include='*.php' -e 'curl_init' -e 'password_' -e 'session_start' . | grep -v vendor | wc -l

Test coverage. Not the percentage. Whether any executable description of correct behaviour exists at all. With a test suite, a migration is mechanical and verifiable. Without one, the only available oracle is the old service itself, which changes the plan (see below).

Who still knows it. If the people who wrote the business rules are gone and the rules are undocumented, a version jump on an unsupported branch buys you very little: you are paying migration cost to land on another runtime that will also age out, with nobody able to confirm you preserved behaviour.

Two strong branches and a staffed team: jump the version, stay on PHP, get back into a patch cycle. Mostly glue, thin rules, no owner left: moving off the branch entirely is probably cheaper than nursing it, and it ends the recurring exposure instead of deferring it.

What an honest rewrite looks like

Scope first. A day of reading a service tells you what it costs and how many weeks it takes. That is a real deliverable, and it is the only part that can be done quickly. Anyone quoting a rewrite without reading the code is quoting a different service than the one you have.

Strangler, not big bang. The old service keeps serving traffic. New endpoints come up behind the same entry point and take routes one group at a time, and each cutover is individually reversible. A big-bang rewrite has exactly one integration test, on the worst possible day.

Data model, then payment and auth paths. The schema is the hard constraint everything else gets shaped around, and the money and identity paths are where a silent behaviour difference costs real money rather than a bug report. Do them while attention is highest, not last.

Keep the old service running as the reference until parity. With no test suite, the legacy code is your specification: you run both, compare outputs on real traffic shapes, and the differences you find are the undocumented rules. That is how you discover the rounding rule nobody remembers writing.

Timelines, honestly: a small service with clear boundaries moves in days. A large one moves in planned weeks, endpoint group by endpoint group, with traffic shifted gradually. Nothing meaningful gets rewritten in a day. What takes a day is knowing the number.

What I would do differently

I would make the support window a build-time fact rather than a thing someone remembers. A pipeline check that fails when the runtime branch has fewer than six months of upstream support left turns this from an emergency into a scheduled quarter of work. The cost of moving a service does not grow because the code got worse. It grows because the distance to a supported branch gets longer every year you stand still, and because the people who understood the rules keep leaving.

The branches that got patched this week will be unsupported too, on a published schedule. The question that matters is whether you meet that date deliberately.

Getting a number

If you are sitting on 7.4, 8.0 or 8.1 right now, the useful next step is not a debate about risk appetite. It is a cost and a timeline. Send me the service and within a day you will have both: what moving it onto a supported, modern stack costs, and how many weeks it takes. I have done inherited-system rewrites and major runtime migrations before, including monolith-to-services splits where the old system stayed in production as the reference the whole way.

Apply the mitigation today either way.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

Cite the row, or say unknown: a checkable crypto classifier

polycratia — Fri, 23 Oct 2026 13:22:54 +0000

A cryptography inventory that prints RSA quantum_vulnerable is asking to be believed. Whether you can check it instead comes down to two design decisions that have nothing to do with scanning: where the published source of a verdict lives, and what the tool does with an algorithm name it has never seen.

I ran into both while building cbomscope (https://github.com/polycratia/cbomscope), which reads the cryptography out of a Go repository and off a live TLS handshake and writes the result as a CycloneDX 1.6 cryptography bill of materials. Scanning is the easy half. The hard half is that the document outlives the run.

A posture is a claim about cryptanalysis, not about the code

What an inventory asserts about an algorithm is not a property of the codebase. Shor's algorithm solves factoring and discrete logarithms in polynomial time, so RSA and ECDSA fall; Grover's quadratic speedup halves the effective strength of a symmetric key, so AES-128 becomes about 64 bits of quantum work. The RSA key in a test helper and the one signing production tokens carry the same posture. I separated those on purpose: which of the two you should fix first is a different question, and in cbomscope it belongs to plan rather than to the classifier.

It also means every posture in the output restates somebody's published result. Six months after the scan, the person reading the file is usually not the person who ran it. If the line says quantum_vulnerable and nothing else, they can trust the tool or redo the literature search. Both are bad outcomes for a document that is supposed to open a migration discussion.

So the verdict travels with its source.

The citation belongs in the row

Classification in cbomscope is a table. Each row is a family, a posture, a rationale you can read, and the citation the verdict was read from.

// Rule is one row of the classification table.
type Rule struct {
    // Family is matched as a prefix against the uppercased algorithm name. The
    // longest matching family wins, so SHA-1 is not read as SHA.
    Family string
    // Kind narrows the row: a protocol row is never applied to an algorithm,
    // nor the other way round.
    Kind asset.Kind
    // Posture is empty on rows whose verdict depends on a size.
    Posture   asset.Posture
    Rationale string
    Citation  string

    match func(name string) bool
    size  sizeRule
}

// PostureLabel is the verdict for display. A row settled by a size says so
// rather than showing an empty column.
func (r Rule) PostureLabel() string {
    if r.Posture == "" {
        return "by size"
    }
    return string(r.Posture)
}

The citations themselves are constants at the top of the file, which matters more than it sounds like it should. When they are string literals scattered through branches, nobody reads them as a set, and the same source gets paraphrased three different ways. As a block, they are an audit surface:

const (
    shor      = "Shor 1997, SIAM J. Comput. 26(5): polynomial-time factoring and discrete logarithms"
    grover    = "Grover 1996, STOC: a quadratic speedup for unstructured search"
    bht       = "Brassard, Hoyer and Tapp 1998, LATIN: quantum collision search in about 2^(n/3)"
    ir8547    = "NIST IR 8547: transition to post-quantum cryptography standards"
    fips203   = "NIST FIPS 203: ML-KEM, module-lattice key encapsulation"
    sp800131a = "NIST SP 800-131A Rev. 2: transitioning cryptographic algorithms and key lengths"
    rfc6151   = "RFC 6151: MD5 and HMAC-MD5 security considerations"
    shattered = "Stevens et al., CRYPTO 2017: the first SHA-1 collision"
    sweet32   = "Bhargavan and Leurent, CCS 2016: birthday attacks on 64-bit block ciphers"

    // What the attack costs is one half of the answer; what a migration is
    // expected to do about it is the other.
    shorMigration   = shor + "; " + ir8547
    groverMigration = grover + "; " + ir8547
)

That last pair is the part I would defend hardest. A citation that only names the attack explains why an algorithm is doomed and says nothing about what anyone expects you to do instead. Pair the result with the transition guidance and the row answers both halves of the question you actually have.

The table is data so the gaps stay visible

The other reason this is a table rather than a chain of conditions: a chain hides its own holes. A switch with no case for a family looks identical to a switch that was never asked about it. A list of rows can be counted and printed, and a missing row is a visible shape rather than an absent branch.

It also makes room for the families prefix matching cannot reach. A hybrid TLS group is spelled as a concatenation, and both of its halves would otherwise match rows of their own, so rows carrying a match function are tried before any prefix:

var (
    postQuantumHalves = []string{"MLKEM", "ML-KEM", "KYBER"}
    classicalHalves   = []string{"X25519", "X448", "SECP", "P256", "P-256", "P384", "P-384", "P521", "P-521", "ECDH"}
)

var Table = []Rule{
    {
        Family:    "X25519MLKEM768",
        Posture:   asset.Hybrid,
        Rationale: "classical and post-quantum key exchange combined: secure if either half holds",
        Citation:  hybridTLS,
        match: func(name string) bool {
            return strings.Contains(name, "HYBRID") ||
                (containsAny(name, postQuantumHalves) && containsAny(name, classicalHalves))
        },
    },
    // ...
}

After the match rows, the longest matching family prefix wins, and that is the rule keeping SHA-1 from being read as SHA.

Print the table without running a scan

If the citations sit in the rows, the cheapest useful command in the whole tool is the one that prints the rows:

tw := tabwriter.NewWriter(out, 0, 0, 2, ' ', 0)
fmt.Fprintln(tw, "FAMILY\tVERDICT\tSOURCE")
for _, rule := range classify.Table {
    fmt.Fprintf(tw, "%s\t%s\t%s\n", rule.Family, rule.PostureLabel(), rule.Citation)
}

cbomscope table takes no arguments, reads nothing, and ends with the line n row(s). A family with no row here is reported as unknown. If you disagree with a verdict in a report you can go read where it came from without pointing the tool at anything, and -json emits the same rows through Rule.MarshalJSON for whoever wants to diff two versions of the table instead of two scans.

Unknown is an answer, and it has to cost something

Now the second decision. A classifier handed a name no row knows has three options: guess, pass it, or admit it. Guessing is wrong. Passing is the one that looks reasonable and isn't, because a clean report and an incomplete report become the same document.

In cbomscope, no row matching returns PostureUnknown with an empty Asset.Citation. The empty citation is load-bearing: a source printed beside an answer nobody has would make the gap look checked.

An unknown that merely sits in the output still loses, though, because it sorts somewhere near the bottom and nobody reads it. So the ranking treats it as work:

func postureFactor(p asset.Posture) (Factor, bool) {
    switch p {
    case asset.QuantumSafe, asset.Hybrid, asset.NotApplicable:
        return Factor{}, false
    case asset.Broken:
        return brokenNow, true
    case asset.QuantumVulnerable:
        return shorFalls, true
    case asset.QuantumReduced:
        return resizable, true
    }
    return unjudged, true
}

The false return is reserved for postures that genuinely imply no migration, and those assets are listed as settled rather than dropped, since a report that omits what it checked reads exactly like one that never looked. Everything else, including an asset that never reached the classifier at all, falls through to unjudged: ranked above the posture that is merely resizable, because an algorithm nobody has judged is a worse position to be in than a 256-bit hash with a known answer. Silence is not a clean bill of health.

The citation has to leave the tool

The last step makes the terminal output irrelevant. CycloneDX 1.6 has no field for a quantum posture, so writing one into a standard field would make the document lie to anything that reads it by the spec. Instead the posture, the rationale and the citation go out as namespaced properties beside the component:

type Property struct {
    Name  string `json:"name"`
    Value string `json:"value"`
}

cbom.Build writes cbomscope:citation next to the posture property. What you get is a file that can be attached to a release and read a year later by someone who has never run the tool, where every claim about an algorithm carries the paper or the standard it came from, and the entries nobody could judge say so in the same vocabulary.

What I would do differently

A citation is a string. That is enough for you to go find the document and not enough for anything automated to notice when the document has moved. NIST SP 800-131A Rev. 2 already encodes a revision in prose, which is a hint that the field wants structure: a document identifier, a revision, and optionally a URL, so a row whose source has been superseded can be found by a tool rather than by someone rereading the constants block.

The second thing I would look at again is PostureLabel returning by size. Honest in a terminal column, but MarshalJSON emits the same label, so a program consuming the JSON table sees by size in a field that otherwise holds postures. Display logic leaked into the data representation, and the fix is for the JSON to expose the size-dependence as its own field instead of a magic string.

Neither of those changes the shape of the design. The shape is that a verdict without a printable source is an opinion, and an inventory of opinions is what this kind of tool exists to replace.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

Validate VEX on the way out, not on the way in

polycratia — Thu, 22 Oct 2026 13:22:54 +0000

A tool that both reads and writes VEX documents ends up with one struct carrying two opposite obligations. A document I issue is a claim somebody downstream will act on, so anything ambiguous has to be refused before it reaches disk. A document a supplier issued last quarter is a fact, and refusing to parse it hides exactly the statements someone needs in order to fix them. The mistake is serving both duties with one validator called from both sides.

I ran into this building vexdesk, a small Go tool that sits between a bill of materials and a VEX document: it reads what syft or trivy already wrote, works out which OSV advisories could touch the components, and turns my decisions into OpenVEX. From 11 September 2026 the EU Cyber Resilience Act expects manufacturers to answer the "is my product affected" question in writing, quickly, for every product they ship. That changes what these files are. They stop being an artefact of an internal process and become something a customer receives, stores, and compares against the one before it.

That last word carries the whole argument. Comparison is the only reason to be lenient on read, and it is a sufficient one.

Reading: refuse only what is not a document

The read path has exactly one job: decode. Whether the document obeys the rules is information, not a gate.

// Parse reads a document that has already been issued. It does not validate: a
// document published last quarter is a fact, and refusing to read it would hide
// exactly the statements someone needs to see in order to fix them. Writing
// stays strict; reading does not.
func Parse(data []byte) (*Document, error) {
    var doc Document
    if err := json.Unmarshal(data, &doc); err != nil {
        return nil, fmt.Errorf("openvex: %w", err)
    }
    if doc.Context == "" && len(doc.Statements) == 0 {
        return nil, errors.New("openvex: not a VEX document: it has neither @context nor statements")
    }
    return &doc, nil
}

The single check left in there is not a conformance check. A payload with neither @context nor statements is not a malformed VEX document, it is some other JSON file, almost always because the wrong path was typed on the command line. Telling you that is helpful. Telling you that a vendor's not_affected is missing a justification code, and then refusing to open the file, is not: the finding you wanted to look at is inside the file you just declined to read.

The payoff shows up in the diff command, which is what a recipient of a new document actually wants:

$ vexdesk diff released/vex-2.0.0.json vex.json
3 change(s), 4 claim(s) unchanged

CHANGE       ADVISORY      PRODUCT                                      DETAIL
new          FIXTURE-0005  pkg:npm/cogwheel@4.1.0                       under_investigation
restated     FIXTURE-0004  pkg:npm/cogwheel@4.1.0                       affected → fixed
rejustified  FIXTURE-0001  pkg:golang/github.com/example/widget@v1.2.3  justification vulnerable_code_not_in_execute_path → component_not_present

The comparison runs claim by claim, one vulnerability against one product. For that to work, both sides have to be loadable, including the old one I issued before I tightened my own rules, and including the one from a supplier whose generator is sloppier than mine. A strict parser would make the second document in that pipeline unreadable precisely when it matters most.

Writing: refuse anything that leaves a reader guessing

On the way out the posture inverts. Validate is called from exactly one place, the constructor, and it is unforgiving.

func (d *Document) Validate() error {
    var errs []error
    if strings.TrimSpace(d.Author) == "" {
        errs = append(errs, errors.New("document: author is required"))
    }
    if len(d.Statements) == 0 {
        errs = append(errs, errors.New("document: a document with no statements says nothing"))
    }
    for i, s := range d.Statements {
        if err := s.Validate(); err != nil {
            errs = append(errs, fmt.Errorf("statement %d: %w", i, err))
        }
    }
    return errors.Join(errs...)
}

Two details there are deliberate. Errors are joined rather than returned on the first failure, because somebody fixing a decisions file wants the whole list, not one problem per run. And each statement error is wrapped with its index, because "vulnerability name is required" is useless without knowing which of forty statements it belongs to.

not_affected is where the rule earns its keep

The statement-level check that matters most is the one on not_affected, since that is the verdict nobody rechecks:

case NotAffected:
    // One of the closed set of codes, always. Prose explains a decision to
    // the person reading this document; only the code lets a reviewer, a
    // diff or another tool compare it against every other not_affected.
    // An impact statement is welcome alongside it, never instead of it.
    switch {
    case s.Justification == "" && strings.TrimSpace(s.ImpactStatement) != "":
        errs = append(errs, fmt.Errorf(
            "not_affected has an impact_statement but no justification: prose does not stand in for one of %v",
            justifications))
    case s.Justification == "":
        errs = append(errs, fmt.Errorf("not_affected requires a justification, one of %v", justifications))
    case !slices.Contains(justifications, s.Justification):
        errs = append(errs, fmt.Errorf("justification %q is not one of %v", s.Justification, justifications))
    }

The first case took a real argument with myself to keep. A sentence like "the affected parser is only reached from the admin importer, which this build does not include" is genuinely more informative to a human than the code vulnerable_code_not_in_execute_path. It is also unusable by the diff above. A reviewer putting this quarter's document next to last quarter's can see that a code changed; nobody can tell whether two paragraphs of prose express the same conclusion. So the code is mandatory and the prose is encouraged next to it, and the error message lists the accepted codes rather than sending the author off to find the spec.

The symmetric rules follow from the same test: can you act on this without asking me a question. affected requires an action_statement, because telling a customer they are affected and nothing else gives them no move to make. A justification on any status other than not_affected is refused outright: a code attached to fixed looks like it means something and does not.

Note that none of this is enforced on read. A supplier document that breaks every one of these rules parses fine and diffs fine, and that is what splitting the paths buys.

The identifier comes from the statements

// New assembles a document and validates it. The identifier is derived from the
// statements, so the same decisions produce the same document id — a rebuild in
// CI does not look like a new document every time.
func New(author string, at time.Time, tooling string, statements []Statement) (*Document, error) {
    doc := &Document{
        Context:    Context,
        Author:     author,
        Timestamp:  at.UTC().Format(time.RFC3339),
        Version:    1,
        Tooling:    tooling,
        Statements: statements,
    }
    doc.ID = deriveID(statements)
    if err := doc.Validate(); err != nil {
        return nil, err
    }
    return doc, nil
}

deriveID is a hash of the marshalled statements, nothing else. A random UUID would have been one line shorter and would have made every CI run produce a document that looks new. The timestamp moves on every build and should; identity should not. If my decisions did not change, the @id does not change, and a recipient automating against these files can tell a reissue from a revision without reading the body.

That makes the placement of the validation matter in a way that is easy to miss: the ID is assigned before Validate runs, so a rejected document never escapes with a plausible-looking identifier attached. There is no partially-constructed Document lying around for a caller to grab and write out anyway.

No issuer, no document

The vex command refuses to run without -author. The library would refuse too, a few hundred milliseconds later, after reading the SBOM and the advisory directory, so the flag check is redundant in the strictest sense. I keep it because the reason is not about flags. A VEX document without an issuer cannot be trusted by its reader: an anonymous not_affected is a claim with nobody behind it, which is worth less than no claim at all. Checking it at the edge says that out loud, and checking it again in Validate means it holds no matter who calls the package.

What I would do differently

Leniency on read currently means silence on read. The information is available, since Validate is exported and I could run it against a parsed supplier document, but nothing in the read path surfaces it, so a document with a bare impact_statement and no code passes through unremarked. I would add that as a report rather than a refusal: parse, diff, and separately say "this document breaks three rules I would not have let myself emit." Accepting a flawed document and failing to mention the flaws are two different decisions, and I conflated them.

Two smaller things. Version is hardcoded to 1 while the @id moves with the statements, which leaves two identity-shaped fields following different rules. And because deriveID hashes only the statements, the same claims issued under a different author produce the same identifier, defensible enough since the ID names a set of decisions and the author is a separate field, but it is a property I arrived at rather than chose...

The general shape has held up, though, and I think it is not specific to VEX. Any format you both consume and produce wants the validator on one side only. Strict where you are the author and responsible for what the reader can do with it; permissive where somebody else already made their choices and all you can do is read them.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

Routing a withdrawal is a value, not an if inside the sender

polycratia — Wed, 21 Oct 2026 13:29:18 +0000

A withdrawal to an address you already custody does not have to touch a chain: crediting it is a ledger move, with no fee, no confirmations and nothing to wait for. A withdrawal to anyone else is a transaction you cannot take back. In most payout code I have been handed, the choice between those two lives inside the sender, as an if a few lines above the broadcast, which means that after the fact nobody can say which path a given withdrawal took, or why.

That is the whole problem. The branch itself is trivial. What is not trivial is that the branch decides the fee and the latency, and it also decides what the word settled means, and then throws that decision away.

The two rails do not disagree about states, they disagree about settlement

In the withdrawal pipeline I keep at https://github.com/polycratia/withdrawals, both rails walk the same state chart: requested, approved, sending, sent, confirmed, with rejected and failed as the two ways out. I did not fork the machine per rail, and I would push back hard on anyone who wanted to. What differs between the rails is not the shape of the lifecycle: it is what the reference on the request means, and when the request is allowed to confirm. On an internal route, the thing that moved is a ledger entry against an account I own, and it is done once the credit is booked. On an external route, sent means a transaction exists somewhere and confirmed means the chain agreed to keep it.

A boolean living in a local variable inside the sender cannot carry that. The settlement rule has to travel with the withdrawal, because the code that decides whether to wait for confirmations runs minutes or hours after the code that picked the rail.

Decide once, return the decision, log it before you hand it back

So routing is its own function, and it returns a value:

def route(request: WithdrawalRequest, custody: Custody) -> Route:
    """Decide which rail `request` travels on and log the decision."""
    account = custody.account_for(request.destination)
    if account is None:
        decision = Route(
            rail=Rail.EXTERNAL,
            destination=request.destination,
            reason="destination is not an address we custody",
        )
    else:
        decision = Route(
            rail=Rail.INTERNAL,
            destination=request.destination,
            reason=f"destination is custodied by account {account}",
            account=account,
        )
    logger.info(
        "withdrawal %s routed %s to %s: %s",
        request.id,
        decision.rail,
        decision.destination,
        decision.reason,
    )
    return decision

Two details in there are load-bearing, and the if is not one of them.

The first is reason. The rail alone tells you what happened; the reason tells you what the system believed at the time. Months later, when someone asks why a payout to a known customer address cost a network fee, the answer you need is not "it went external", since the fee already told you that. The answer you need is "custody did not recognise that address at 11:42", which is a statement about the state of your own address book, not about the chain.

The second is that the log line happens before the return, unconditionally, on both branches. That sounds like a style point and is not. An internal transfer leaves no trace on any chain, so if your only durable record of a payout is the explorer, every internal withdrawal you ever processed is invisible to you. Logging at the decision point, rather than inside whichever sender was eventually chosen, is what makes the two populations countable afterwards. I have had to reconstruct that population retroactively on a custodial system once, from ledger entries alone, and it is not an afternoon's work...

Each rail's rules belong in the value, not in the caller

The decision is a frozen dataclass that refuses to exist in a shape that makes no sense:

@dataclass(frozen=True, slots=True)
class Route:
    """The routing decision for one withdrawal, with the reason behind it."""

    rail: Rail
    destination: str
    reason: str
    account: str | None = None

    def __post_init__(self) -> None:
        if not isinstance(self.destination, str) or not self.destination.strip():
            raise ValueError("destination must be a non-empty string")
        if not isinstance(self.reason, str) or not self.reason.strip():
            raise ValueError("reason must be a non-empty string")
        if self.rail is Rail.INTERNAL:
            if not isinstance(self.account, str) or not self.account.strip():
                raise ValueError("an internal route must name the account it credits")
        elif self.account is not None:
            raise ValueError("an external route has no internal account")

    @property
    def is_internal(self) -> bool:
        """True when the transfer stays on our own books."""
        return self.rail is Rail.INTERNAL

    @property
    def needs_confirmations(self) -> bool:
        """True when the transfer only settles once the chain confirms it."""
        return self.rail is Rail.EXTERNAL

An internal route must name the account it credits. An external route must not carry one. That second rule is the interesting half: it is there because the bug I expect is not a missing account, it is a stale one, a decision that says external while still holding an account value that came from an earlier lookup, or a retry, or a copy-paste in a test fixture. A credit booked against that account is money created out of nothing. Making the combination unconstructible costs four lines and removes the whole class.

Note also that is_internal and needs_confirmations are derived from rail rather than stored beside it. There is exactly one field that can be wrong. Anything a later stage wants to know about settlement it asks the decision, so the sender, the confirmation watcher and the reconciliation job cannot drift apart on what external means.

The lookup is where the money actually goes wrong

Once routing is a value, the branch is boring and the risk moves entirely into the custody lookup. That lookup is asymmetric in a way worth saying out loud: a destination that is mine but fails to match sends real funds through a real rail and pays a real fee for something that should have been a free ledger entry, and you cannot undo it. The reverse error, matching an address that is not mine, credits an internal account for a payout that did not go anywhere. Neither is a crash. Both are quiet.

Which is why normalisation is the implementation's job, and it happens on both sides of the comparison:

class InMemoryCustody:
    """Custody backed by a mapping of deposit address to internal account."""

    __slots__ = ("_accounts",)

    def __init__(self, accounts: Mapping[str, str]) -> None:
        self._accounts: dict[str, str] = {}
        for destination, account in accounts.items():
            if not destination.strip() or not account.strip():
                raise ValueError("custody entries need an address and an account")
            self._accounts[_lookup_key(destination)] = account

    def account_for(self, destination: str) -> str | None:
        return self._accounts.get(_lookup_key(destination))


def _lookup_key(destination: str) -> str:
    return destination.strip().casefold()

A padded or mixed-case destination still finds its account, and an empty address or account is refused at construction rather than sitting in the map as a key that matches nothing. The test table is the specification here, and it is deliberately boring:

DESTINATIONS = [
    pytest.param(THEIRS, Rail.EXTERNAL, None, id="an-address-we-do-not-know"),
    pytest.param(THEIRS.upper(), Rail.EXTERNAL, None, id="case-does-not-make-it-ours"),
    pytest.param(OURS, Rail.INTERNAL, "acct-77", id="an-address-we-custody"),
    pytest.param(OURS.upper(), Rail.INTERNAL, "acct-77", id="upper-case"),
    pytest.param(f"  {OURS}  ", Rail.INTERNAL, "acct-77", id="surrounding-space"),
]

The case-does-not-make-it-ours row is the one I would keep if I could keep only one. Normalisation widens what matches. It must not widen whose it is.

One consequence of doing it this way: the decision records the destination exactly as it was asked for, spaces and all, while the match runs on the normalised key. The test asserts that directly. The log should show what the user submitted, because that is the string you will be comparing against a support ticket, not your internal key.

One place picks the sender

With the decision in hand, selecting a sender stops being a branch at all:

def withdraw(request, key, custody, holds, policy, rails):
    ensure_approved(request, policy)
    decision = route(request, custody)
    holds.place(Hold.on(request))
    send = rails[decision.rail]
    return submissions.submit(
        key, request, lambda pending: broadcast(pending, send, holds)
    )

rails[decision.rail] is the entire dispatch. The ledger sender contains no chain code and the chain sender contains no ledger code, so neither of them can take the wrong path under a bad condition: the worst they can do is fail at what they are for.

Custody itself is a one-method runtime-checkable Protocol, account_for(destination) -> str | None. In production that is a database query against the deposit-address table. In the tests it is a dictionary, and one test supplies a class that knows exactly one address, to pin down that any lookup will do. Routing has no opinion about where the address book lives, which is the only reason the decision can be unit-tested without a chain, a ledger or a clock.

What I would do differently

needs_confirmations is a boolean, and a boolean answers whether, not how deep. Confirmation depth is a per-asset number, so the boolean is probably best read as a floor ("this one waits"), and the depth still has to come from somewhere else. If I were starting again I would think harder about whether the route is the right carrier for that number, so that the settlement rule travels as one value rather than two.

reason is prose. That is correct for the reader I care about most, the person reading a log line at an awkward hour, and wrong for the other reader: you cannot aggregate over a sentence. The structured half of an internal decision is the account, and that is what I would group by if I had to report how much moved on my own books. I would not replace the sentence with a code. I would stop pretending the sentence is queryable.

And _lookup_key is a module-level helper, which quietly assumes one normalisation is right for every address space a withdrawal might target. The Custody protocol gets this right by leaving normalisation to the implementation. The in-memory implementation just happens to pick one, and that convenience is the kind of thing that later gets reused by a custody backend where it does not fit.

Close

The sender's job is to send. The decision about which sender, and the reason behind it, is one function call older than the send, and it needs to outlive it by the length of your log retention. Written down as a value, a branch nobody remembers taking turns into a fact you can answer questions about.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

A paywall is a table of what is sold

polycratia — Tue, 20 Oct 2026 13:29:17 +0000

Any HTTP payment gate has to answer one question: what happens to a request it has no price for. Most middleware answers that by accident rather than by design. Everything behind the gate is paid, so a path nobody priced gets a challenge built from some fallback amount, or a 500, or an empty accepts array that no client can act on. I built paygate402 the other way around. The price table is the entire statement of what is sold, and a route the table does not name is served free, with no challenge at all.

That is one of two defaults in this gate worth arguing about. The other one is an ordering: the free tier is spent before the X-PAYMENT header is even read. Five lines each. Both of them decide what the thing is.

An unpriced route is not an error

A gate sitting in front of a single resource can carry its terms directly in Accepts. A gate in front of a whole API declares a price per route instead:

gate := &paygate402.Gate{
    Prices: &paygate402.Prices{
        Terms: paygate402.Requirements{
            Scheme:            "exact",
            Network:           "base",
            PayTo:             merchantAddress,
            Asset:             usdcAddress,
            MaxTimeoutSeconds: 60,
        },
        Routes: []paygate402.Route{
            {Pattern: "/report", Amount: "10000", Description: "One generated report"},
            {Pattern: "/reports/{id}", Amount: "2000"},
            {Pattern: "POST /jobs", Amount: "50000"},
        },
        BaseURL: "https://api.example.com",
    },
    Facilitator: facilitator,
}

http.Handle("/", gate.Handler(api))

Terms holds what does not change from route to route (the scheme, the network, the asset, the recipient), and each route supplies the amount. The lookup is a compiled http.ServeMux of the priced patterns, so method matching, wildcards and precedence come from the standard library instead of a second dialect invented inside a payment package. /reports/summary beats /reports/{id} here for the same reason it would anywhere else in a Go server.

What happens on a miss is where the design decision sits:

accepts, err := g.accepts(r)
switch {
case errors.Is(err, ErrNoPrice):
    // The table is the whole statement of what is sold here, so a route
    // missing from it is free rather than closed.
    next.ServeHTTP(w, r)
    return
case err != nil:
    http.Error(w, err.Error(), http.StatusInternalServerError)
    return
}

ErrNoPrice is a sentinel the lookup returns for a path no Route declares, and the gate reads it as this costs nothing, not as this failed. A health check needs no line in the table saying it is free. Neither does /favicon.ico, nor the OpenAPI document, nor whatever the load balancer probes.

The opposite default (fail closed, charge for anything not explicitly exempt) sounds like the safe one, and it is the one I would expect a security reviewer to ask for. I think it is wrong for this component, for a reason that has nothing to do with safety: it stops the price table from being the statement. You would need a second table, of exemptions, and the two would be disagreeing inside a month. Every route added to the API would have to land on one list or the other before the service could answer a probe, and the failure mode of forgetting is a 402 on an endpoint whose caller has no wallet and no idea what a facilitator is. Ask a paywall what it sells, and it should answer by printing one list. It can only do that if the list is exhaustive by construction.

The cost of this default is honest and should be stated: a typo in a pattern makes a paid route free rather than broken. That is a real risk, and it is why the two failures that can be caught are caught loudly. A malformed or repeated pattern makes http.ServeMux panic, and registration turns that panic into an error the gate reports as a 500, like the rest of its misconfiguration. Gate.validate refuses a gate with both Accepts and Prices set, because those are two ways of saying the same thing, and the alternative is a precedence rule nobody would remember at three in the morning.

The free tier is spent before the payment is read

The second default is nothing but the order of two statements in Handler:

// The free tier is spent before the payment is even read: a client that
// does not have to pay yet is not asked to, and a payment it sent anyway
// is left unspent rather than taken.
if client, left, granted := g.Free.take(r); granted {
    g.serveFree(w, r, next, client, left)
    return
}

payment, err := DecodePayment(r.Header.Get(PaymentHeader))

Turn those around and nothing visibly breaks. The gate still serves five free requests, and the tests that count responses still pass. What changes is the client that sends a payment it did not have to send: an agent that attaches one to every request, because that is simpler than tracking an allowance. Decode first, and that payment gets matched, verified, recorded against replay and settled, and the free request is consumed on top of it. The client paid for something advertised as free and has no way to tell, because the response looks exactly like the paid one.

With the check in this order, the payment header is not read at all. The facilitator is not called. The test asserts the absence rather than the outcome, which is the only way to assert it:

func TestAGateWithFixedTermsCanAlsoHaveAFreeTier(t *testing.T) {
    facilitator := &StaticFacilitator{}
    g := gate(facilitator)
    g.Free = &Allowance{Requests: 1}

    if first := request(t, g, ""); first.Code != http.StatusOK {
        t.Fatalf("status = %d, want the free request served", first.Code)
    }
    if second := request(t, g, ""); second.Code != http.StatusPaymentRequired {
        t.Errorf("status = %d, want 402 once the allowance is gone", second.Code)
    }
    if facilitator.Trail() != "" {
        t.Errorf("calls = %q: a free request was put to the facilitator", facilitator.Trail())
    }
}

Every free response also carries how many are left:

X-PAYMENT-ALLOWANCE: 4

AllowanceHeader is this package's own: x402 does not define it. A client that can watch the allowance running down can have a payment ready, instead of finding the boundary by being refused and having to replay the request.

A free request that produced nothing is given back

The paid path in this gate does not charge for work it did not deliver: a handler that answers non-2xx gets the payment released rather than left spent. The free path owes the client the same courtesy, for the same reason. serveFree runs the handler on the house, and when the handler answers outside the 2xx range it hands the request back through Allowance.give.

The in-process counter does that with a decrement, and with the floor that matters:

// Return gives one free request back.
func (c *AllowanceCounter) Return(client string) {
    c.mu.Lock()
    defer c.mu.Unlock()
    if entry := c.used[client]; entry != nil && entry.used > 0 {
        entry.used--
    }
}

The entry.used > 0 guard is not defensive clutter. Without it, a duplicated return (a retry, a buggy caller, a store that saw the release twice) drives the count negative and quietly hands out an unbounded free tier.

The tally resets over Every, and the sweep runs on write instead of in a goroutine:

// sweep drops the clients whose period has closed. It runs on write, on the
// requests that matter, so no background goroutine has to exist. A lifetime
// allowance has no period and is never swept — that is what makes it a
// lifetime one.
func (c *AllowanceCounter) sweep(now time.Time) {
    for client, entry := range c.used {
        if !entry.resets.IsZero() && now.After(entry.resets) {
            delete(c.used, client)
        }
    }
}

Every == 0 means the allowance does not reset, and that falls out of the same branch rather than from a special case: an entry with no reset time is never swept, so it is a lifetime allowance by omission.

Who counts as a client

Clients are told apart by ClientIP, which reads RemoteAddr and nothing else:

func ClientIP(r *http.Request) string {
    host, _, err := net.SplitHostPort(r.RemoteAddr)
    if err != nil {
        return r.RemoteAddr
    }
    return host
}

No forwarded headers. Anyone can set one, and a free tier that counts a header the client controls counts nothing. The consequence is that behind a proxy the default is useless, since every request arrives from one address and the whole world shares five requests, so Allowance takes a Client function: you give it the header your own proxy sets, or you key the tier on something a client states on purpose:

g.Free = &Allowance{
    Requests: 1,
    Client:   func(r *http.Request) string { return r.Header.Get("authorization") },
}

This is a courtesy and not a rate limit, and it should not be sold as one. Before anyone has paid, the only thing a request carries is the address it came from, and a new address is a new allowance, so the tier is worth exactly what a fresh address costs. It exists so an agent can try an endpoint before paying for it. The tally also lives in one process unless Counts is set, which is why it is an interface: several replicas mean several allowances.

type AllowanceStore interface {
    Take(client string, limit int, window time.Duration) (bool, int)
    Return(client string)
}

What I would change

The inconsistency I would fix first is about trusting the request. The allowance refuses to read a forwarded header because the client controls it. But when neither BaseURL nor a route Resource is set, the resource named in the 402 gets built from the request's own scheme and host, which are just as much the client's to assert and are wrong behind a proxy that terminates TLS. Two parts of one package answer the same question in opposite directions. The free tier is the side that got it right. The resource fallback should be a configuration error instead of a guess.

The rest I would keep. A paywall whose table is the whole statement of what is sold, and a free tier spent before any money is read: those are choices you get to make once, because every later line of code assumes one answer or the other.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

The ledger core takes no lock, and the protocol says whose job it is

polycratia — Mon, 19 Oct 2026 13:22:41 +0000

Placing a hold is a read followed by a write: you ask what is available on an account, refuse if the amount does not fit, store the hold. Two callers doing that at the same moment can both read the same figure, both find room, and both write, and the account now reserves more than it holds. A ledger core cannot close that gap on its own, because it owns no records to lock. So you put the gap in the interface and name the side responsible for closing it.

That is the shape I settled on in ledger-core, a double-entry core for custodial balances. The core promises the arithmetic. The store promises that a check and the write it guards land as one step. Both halves are written down, and the second one is a method on a protocol rather than a paragraph in a README, which (as I will get to) is the difference between a contract and a wish.

The two steps

Here is the whole of it, unedited:

    def place(
        self,
        *,
        hold_id: str,
        account: Account,
        amount: MoneyLike,
        placed_at: datetime,
        memo: str = "",
        metadata: Mapping[str, str] | None = None,
    ) -> Hold:
        """Reserve funds. Refused if they are not there to reserve."""
        with self._storage.atomic():
            if self._storage.hold(hold_id) is not None:
                raise DuplicateHold(f"hold {hold_id} is already placed")
            hold = Hold(
                hold_id=hold_id,
                account=account,
                amount=amount,
                placed_at=placed_at,
                memo=memo,
                metadata=metadata or {},
            )
            available = self.available(account)
            if hold.amount.amount > available:
                raise InsufficientFunds(
                    f"hold {hold_id} reserves {amount}, "
                    f"{available} {account.currency} is available on {account.account_id}"
                )
            self._storage.put_hold(hold)
            return hold

Two guarded writes are stacked in that method, not one. The duplicate check reads a hold by id, then writes a hold under that id. The funds check reads available(account) and then writes. Strip the with line and the method still reads as correct to a reviewer going top to bottom, and it is wrong in the only sense that matters in production.

Holds.spend has the same shape for a movement that was never reserved:

    def spend(self, entry: Entry, *, account: Account) -> Entry:
        """Write a movement out of an account, refused if the funds are not free.

        An entry answers to its own rules only, and balancing says nothing about
        whether what it takes is still there once open holds are counted. This
        asks that question first and appends only if the answer holds.
        """
        with self._storage.atomic():
            taken = -effect_on(entry, account)
            available = self.available(account)
            if taken > available:
                raise InsufficientFunds(
                    f"entry {entry.entry_id} takes {taken} {account.currency} from "
                    f"{account.account_id}, {available} {account.currency} is available"
                )
            return self._journal.append(entry)

The arithmetic the core actually owns

What the core guarantees with no help from anyone is narrow, and worth stating precisely: people assume it covers more than it does.

An entry is checked for balance when it is built, so a one-sided movement never becomes an object. Journal.append asks whether the entry balances and nothing else: it has no opinion about whether the account can afford it. A balance is folded from postings rather than kept as a number, so there is no stored total for two writers to read and write back over each other, and that whole class of lost update does not exist here. available is balance less held, both derived.

None of that says anything about two spend calls racing on one account. Each one balances. Each one gets a truthful answer from available() at the moment it asks. The arithmetic is sound in both and the account still ends up overdrawn. Correct arithmetic over an unstable read is still a wrong answer, and no amount of invariant checking inside the core fixes it, because the instability sits in the store.

The seam has a name

So the seam is declared:

@runtime_checkable
class Storage(Protocol):
    def atomic(self) -> AbstractContextManager[None]:
        """One unit of writing: everything under the block, or none of it.

        A guarded write is two steps — asking whether the funds are free or the
        key already settled, and then writing — and a refusal in the second must
        not leave the first behind. Writes made under the block land together;
        leaving it by an exception undoes them. A block opened inside another
        joins it rather than settling on its own, so a guarded write called from
        inside a wider one widens to it.
        """

The reason this is a protocol method and not prose is mechanical: you cannot implement Storage without writing atomic. A type checker raises it, and isinstance(InMemoryStorage(), Storage) is asserted in the test suite. Whoever adapts this core to their database gets forced to answer the question at the moment they are best placed to answer it, since they are the only person who knows what their store can promise.

An earlier shape of this package had no atomic at all. Storage declared append_entries,put_hold,put_snapshot and put_operation, and the README said the atomic check-then-write pair belonged to the backend. True words, and useless ones. The core performed the check and the core performed the write; the backend was handed two unrelated method calls with no indication that they belonged together. There was nothing for a transaction to attach to. The split was documented and unimplementable at the same time, which is the specific failure I want to name, because it reads like diligence. A division of responsibility that gives the other side no place to stand has not been divided. It has been dropped.

What the reference store does

The in-memory store has no transaction of its own, so it builds one:

    @contextmanager
    def atomic(self) -> Iterator[None]:
        """Writes under this block land together, or not at all."""
        with self._lock:
            if self._depth:
                self._depth += 1
                try:
                    yield
                finally:
                    self._depth -= 1
                return
            # records are values, so a shallow copy of each shelf is a whole one

The outermost block keeps a copy of each shelf and puts it back if the block is left by an exception, and the RLock admits one writer at a time. Rollback is cheap for the same reason balances cannot drift: records are values, so copying the dicts copies everything that could change.

For a dict-backed double this is exactly right. As a mental model for production it misleads you in one direction: it serialises every writer in the process, not the writers touching one account. Which brings me to the part that actually bites.

A transaction is not the whole promise

For a database-backed store,atomic is the transaction you already have, and that is usually where people stop. Not enough. The read inside the block is available(account): the account's balance folded from the journal, resumed from a snapshot, minus the sum of its open holds. Under read-committed isolation, two transactions can both read that pre-state, both insert their hold row, and both commit. No constraint is violated; a hold is just a row, and nothing in the schema knows it was meant to fit inside a figure that was true a moment ago. The block was atomic. It was never serialised against the other writer.

So the promise a backend owes is more specific than "wrap it in a transaction": the read the core makes inside the block and the writes that follow must not interleave with another writer on the same account. Two ordinary ways to get there: serialisable isolation with a retry on conflict, or an explicit lock on the account row taken before the read. Neither is exotic. What takes the work is noticing you need one, and that is what a declared method with a docstring, sitting in front of the implementer, buys you.

One thing you do not have to guard: the snapshot cache. A snapshot is only ever replaced by one that reaches further, and one claiming more entries than the journal holds is refused, so a stale snapshot costs you the replay of what it is missing and never a wrong answer. The guarded set is holds and entries. Effort spent locking the cache is effort not spent on the thing that can actually go wrong.

Blocks nest, and that is why capture is one step

Holds.capture appends the settling entry and then writes the settled hold: two writes, and Journal.append opens a block of its own for the first of them. If that inner block committed independently, a refusal in the second half would leave the entry in the journal with the hold still open, and the ledger would claim the money both moved and is still reserved. The joining rule is what lets me describe capture as a single step rather than a sequence that is usually fine.

It works outward too. An application that wants "settle the hold and mark the payout row paid" in one commit opens its own transaction and calls capture inside it; the core's block widens to the caller's instead of closing early. Width belongs to whoever has the broader unit of work, and the core does not assume it is that party.

Prove the rollback, do not assert it

The second half of a guarded write is the half nobody exercises, so the suite has a store you can tell to refuse:

class Refusing(InMemoryStorage):
    """A store that can be made to refuse one kind of record."""

    def __init__(self) -> None:
        super().__init__()
        self.refuses: str | None = None

    def put_hold(self, hold: Hold) -> None:
        self._check("hold")
        super().put_hold(hold)

    def put_operation(self, operation: Operation) -> None:
        self._check("operation")
        super().put_operation(operation)

    def _check(self, kind: str) -> None:
        if self.refuses == kind:
            raise RuntimeError(f"the store refused the {kind}")

That is how you get to assert the property that matters, a store that refuses the hold leaves no entry behind, instead of hoping it holds. There is a tests/test_races.py next to it for the overlapping-call side of the same question.

What I would do differently

Ship atomic in the first version. The release that documented the split without providing it was worse than one that said plainly "this core is single-writer, do not run two processes against it": an honest limitation is actionable, a contract the other side cannot fulfil is not.

Write the isolation requirement into the protocol in verifiable terms. "How wide that promise is belongs to the store" is true and leaves the sharp edge unnamed. "The read made inside the block and the writes that follow must not interleave with another writer on the same account" is something a backend author can check their isolation level against.

And I would probably be more careful about the reference store setting expectations. A test double stricter than production is how a race survives a green suite: the in-memory store's one-writer-at-a-time lock makes the concurrency tests pass for a reason no database will reproduce. The double proves the rollback semantics. It cannot prove the isolation, and the suite should say so where someone will read it.

A core that takes no lock is a reasonable core. A core that takes no lock and does not say so is a race condition with documentation.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

A post-quantum migration plan should print its arithmetic

polycratia — Sun, 18 Oct 2026 13:22:37 +0000

A cryptographic inventory lists what you have. A migration plan claims what to do first, and that claim is an opinion wearing a number. If you cannot recompute the score sitting next to a line, you have two moves available: accept the ordering, or ignore the tool. In my experience you ignore the tool.

I hit this building cbomscope (https://github.com/polycratia/cbomscope), which reads Go source and a live TLS handshake and writes a CycloneDX cryptography bill of materials. The inventory part was uncontroversial. A posture says what a quantum computer would do to an algorithm, which is a property of the algorithm, and nobody argues with a list. The plan command was different: it ranks the same assets by the work they imply, and the first version printed a sorted list with a score column and nothing else. It fell over the first time someone disagreed with it. They could not tell me which part they disagreed with, only that an RSA call in a test helper should not be sitting above the signature algorithm on a public endpoint. That is not feedback about the result. It is feedback about a weight, and the output gave them no way to say so.

The constraint

A migration plan gets read by people who have the authority to reorder it and information the tool does not have. This service is being decommissioned next quarter. That endpoint carries data with a ten-year confidentiality requirement. This dependency is pinned because the upgrade breaks an API nobody here controls. None of that is visible to a program that parses syntax trees and completes handshakes.

So the goal is not to be right about the order. It is to be legible enough that you can overrule one factor without throwing away the whole ordering. A hidden score forces an all-or-nothing judgement on the tool's credibility. A printed one shrinks the argument into something much smaller and much more productive: you think exposure should outweigh lifetime here, and now the conversation is about a number instead of about trust.

Small scales, declared

The weights are not literals buried in a scoring function. Each one is a named Factor (a label, a weight, and a reason), and the scoring functions do nothing but choose a row.

// postureFactor reports the weight of a posture, and false where the posture
// implies no migration at all. Anything it does not recognise — including an
// asset that never reached the classifier — is ranked as unjudged rather than
// settled: silence is not a clean bill of health.
func postureFactor(p asset.Posture) (Factor, bool) {
    switch p {
    case asset.QuantumSafe, asset.Hybrid, asset.NotApplicable:
        return Factor{}, false
    case asset.Broken:
        return brokenNow, true
    case asset.QuantumVulnerable:
        return shorFalls, true
    case asset.QuantumReduced:
        return resizable, true
    }
    return unjudged, true
}

func exposureFactor(a asset.Asset) Factor {
    switch {
    case a.Location.Host != "":
        return atEndpoint
    case a.Module != "":
        return inDependency
    default:
        return firstParty
    }
}

func lifetimeFactor(a asset.Asset) Factor {
    switch a.Primitive {
    case asset.KEM, asset.KeyAgree, asset.PKE:
        return keyExchange
    case asset.Signature:
        return signature
    }
    return storedData
}

Three questions, three scales, and the scales are small on purpose. Key exchange weighs 3 and a signature weighs 1, because the two run on different clocks: what is recorded off the wire today can be decrypted years from now, while a signature is forged at the moment it is verified. That is a deadline rather than a leak, and a deadline you can still move. A one-to-three range is a thing you can hold an opinion about. A zero-to-a-hundred range is not. It only looks more precise, and the extra digits are where fabricated confidence hides.

Why posture multiplies and the rest adds

exposure, lifetime := exposureFactor(a), lifetimeFactor(a)
report.Items = append(report.Items, Item{
    Asset:    a,
    Score:    posture.Weight * (exposure.Weight + lifetime.Weight),
    Posture:  posture,
    Exposure: exposure,
    Lifetime: lifetime,
})

The shape of the expression is itself a claim, and it should be one you can defend out loud. Exposure and lifetime add because they are independent reasons to move sooner: being reachable from the internet and protecting long-lived secrets are separate problems, and an asset with one of them is still work. Posture multiplies because it scales both. A classically broken primitive makes every other property worse, and a sound one makes them moot. Addition there would let a pile of mild factors out-rank something already broken, which is the failure mode people notice and do not forgive.

The expression goes next to the line

This is the whole argument in one method:

// Arithmetic is the score written out, so the number in a column can be checked
// against the line that carries it.
func (i Item) Arithmetic() string {
    return fmt.Sprintf("%s (%d) × [%s (%d) + %s (%d)]",
        i.Posture.Name, i.Posture.Weight,
        i.Exposure.Name, i.Exposure.Weight,
        i.Lifetime.Name, i.Lifetime.Weight)
}

Every ranked line carries the expression that produced its number, in named terms rather than bare integers. You do not have to find the source, and you do not have to take my word for the ordering: you can see that this line scored where it did because it was found at an endpoint and because it is a key agreement, and you can decide that in your deployment the second term is wrong. The factors also survive the JSON output, along with the formula itself on the report, so a plan that travels into a ticket arrives with its reasoning attached instead of a column of integers somebody has to reverse-engineer later.

The same instinct runs one layer down. The classification table prints on its own, with the published source each verdict was read from, because a reviewer who wants to check a posture should not have to read the source to find out where the verdict came from. And a family missing from that listing is the honest shape of a gap.

Settled, not dropped

The first version filtered out the assets with nothing to do. That is the natural implementation and it is a quiet lie:

// Settled are the assets with no migration work: post-quantum, hybrid, or
// carrying no posture of their own. They are counted rather than dropped,
// because a report that omits what it checked reads exactly like one that
// never looked.
Settled []asset.Asset `json:"settled"`

A plan listing four items and a plan listing four items out of sixty are different documents, and the difference matters most to you when you are deciding whether to believe the first number. Post-quantum algorithms, hybrid key exchange, and protocol versions with no posture of their own all go into Settled with their locations intact. The hybrid line earns its place in particular: on a deployment that has already moved its key exchange, the fact that something was checked and found to need nothing is the single most useful sentence in the report.

The weight I expect to defend

Anything the classifier did not recognise, including an explicit unknown posture, ranks at 2, above the Grover-reduced assets at 1. People read that as a bug. It is the one weight I most want printed, because it is the one most worth arguing with: an asset nobody has judged is not a settled asset, and a ranking that sorts unknowns to the bottom turns a gap in the table into a clean bill of health. The same refusal shows up elsewhere in the tool. A key size of zero means not determined rather than small, the classical security level returns zero rather than invent a number, modulus sizes between the published pairs are left out rather than interpolated, and a citation stays empty on an unknown so that a printed source does not make a gap look checked.

If you decide unjudged should sit below resizable for your codebase, you are now making an informed local decision. That is a better outcome than agreement, I think, and it is only reachable because the number is shown in parts.

What I would do differently

The facts the tool cannot see still have nowhere to go. Today someone who knows a service is being decommissioned has to apply that knowledge in their head, after the sort. I would rather it entered as an exposure override on the input side, so the adjustment shows up inside the printed expression like every other factor. A plan corrected by a human should still be a plan whose arithmetic you can read, not one with a silent post-processing step bolted to the end.

I would also hold the line hard on scale size. Every request to make the ranking smarter arrives as a request for one more factor, and three factors with reasons beat seven with none. The moment the expression stops fitting beside the line it describes, it stops being checkable, and an unchecked score is back to being an opinion.

The product is the expression. The number is just what it evaluates to.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

A date is not a version: strict parsing under OSV ranges

polycratia — Sat, 17 Oct 2026 13:22:37 +0000

A vulnerability range is a claim about ordering, and it is worth no more than the version parser sitting underneath it. Walk an OSV range's events in whatever order they happen to sit in the file, compare them against leniently parsed versions, and you get two silent failures: verdicts that flip when the advisory is re-serialised, and a component calling itself 2023-08-01 that lands outside every affected range forever.

Neither failure announces itself, and both land on the one verdict nobody rechecks. An affected finding gets read, argued about, and then either fixed or justified in writing. not_affected gets counted. So a comparator that quietly produces not_affected out of an ordering it invented is worse than one that refuses to answer. A good part of the design work in the internal/match package of vexdesk (the tool I wrote to turn an SBOM and a pile of OSV advisories into an OpenVEX document) goes into making the refusal the cheap path rather than the embarrassing one.

not_affected is version arithmetic and nothing else

Worth pinning down what the verdict means before I defend it. When the matcher says not_affected, it says this version sits outside every range the advisory gives. It is not saying the vulnerable code is unreachable in the product. That is an engineering judgement, it belongs to a person, and it arrives later as a decision file: anything still undecided leaves the document as under_investigation, not as not_affected.

The narrowness is deliberate. A claim about arithmetic over an ordering is the kind of claim you can get exactly right, and the kind that is indefensible to get wrong, because there is nothing subjective left to hide behind. So the ordering has to be real, and where you cannot establish it, the honest output is unknown with the reason attached.

A lenient parser invents an ordering

vexdesk's parseSemver demands exactly major.minor.patch and refuses leading zeros. 1.2 is an error. 2023-08-01 is an error. Both come out as unknown with the parse error as the reason, and the first reaction you hear is that this is pedantry inflating the unknown count. It runs the other way: this is the one place where pedantry carries load.

Think about what a tolerant parser does with a date. Split on non-digit separators, coerce the parts, and 2023-08-01 turns into something that orders as 2023.8.1, a version above every real version anyone will ever publish. Now measure it against a perfectly ordinary range, introduced: 0, fixed: 2.1.4. The date sits above the fix, so it falls outside the interval, so the component is not_affected. Permanently, for every advisory, for a version string that carries no ordering information at all. The tolerant parser did not fail to compare: it succeeded at comparing a number it made up.

The leading-zero refusal is the same argument at smaller scale. Accept 1.08.0 and read it numerically, and now two spellings of one version exist, equality and ordering stop agreeing with each other, and whether a component sits inside a range starts depending on which spelling reached you. Cheaper to have no answer than an answer that depends on that.

Everything the comparison cannot establish ends up in one list, and the list is kept whole:

    var unknown []string
    for _, r := range aff.Ranges {
        switch r.Type {
        case osv.RangeSemver:
            // fall through to comparison
        case osv.RangeEcosystem:
            if !eco.semverOrdered {
                unknown = append(unknown, fmt.Sprintf(
                    "ECOSYSTEM range for %s needs that ecosystem's own version ordering, which is not implemented",
                    eco.name))
                continue
            }
        case osv.RangeGit:
            unknown = append(unknown, "GIT range: commit ranges cannot be compared against a version string")
            continue
        default:
            unknown = append(unknown, fmt.Sprintf("range type %q is not recognised", r.Type))
            continue
        }

        hit, err := inSemverRange(version, r)
        if err != nil {
            unknown = append(unknown, err.Error())
            continue
        }
        if hit {
            return Affected, "version falls inside the advisory's affected range"
        }
    }

    if len(unknown) > 0 {
        // Every reason survives: the person clearing this finding needs the
        // whole list, not the first item of it.
        return Unknown, strings.Join(unknown, "; ")
    }

The ECOSYSTEM branch is the same discipline applied to a whole ecosystem instead of one string. An ECOSYSTEM range means "use this ecosystem's own version ordering", and only a few ecosystems in the tool are marked semverOrdered: golang, npm, cargo. Anything else leaves the range unknown, carrying the reason, rather than quietly compared with semver rules that do not apply to it. Here is what a PyPI component looks like on the way out:

$ vexdesk match -sbom sbom.cyclonedx.json -advisories ./advisories
4 component(s) compared against the advisory set

STATUS    ADVISORY      COMPONENT        VERSION  REASON
affected  FIXTURE-0001  widget           1.2.3    version falls inside the advisory's affected range
affected  FIXTURE-0004  cogwheel         4.0.0    version is in the advisory's affected version list
unknown   FIXTURE-0003  Example_Fixture  3.1.2    ECOSYSTEM range for PyPI needs that ecosystem's own version ordering, which is not implemented

File order is not a sort order

The second failure is subtler, because nothing about it looks like a parsing problem.

An OSV range is a JSON array of events: introduced opens an interval, fixed and last_affected close it. JSON preserves array order, so walking the array and flipping a boolean as you go feels natural. The trouble: the schema's meaning does not live in the array order. A well-formed range can list its events in an order that has nothing to do with their version order, and any tool that reads, filters, merges or re-emits an advisory feed is free to hand you the same claim with the array arranged differently.

This range is the same claim whichever way round you print it:

{
  "affected": [
    {
      "package": { "ecosystem": "Go", "name": "github.com/example/widget" },
      "ranges": [
        {
          "type": "SEMVER",
          "events": [
            { "introduced": "1.5.0" },
            { "fixed": "2.0.0" },
            { "introduced": "0" },
            { "fixed": "1.2.0" }
          ]
        }
      ]
    }
  ]
}

Walked in array order, 1.1.0 opens at 1.5.0, closes at 2.0.0, opens again at 0, closes at 1.2.0, and whether it comes out inside depends entirely on how you carried that state. Shuffle the four lines and the same advisory gives you a different answer. Not a wrong answer: an unstable one, which is harder to notice and much harder to explain to someone who has already shipped a document built on it.

So inSemverRange sorts the events before walking them. Two rules do the work. First, "introduced": "0" sorts ahead of everything: it is the schema's sentinel for "from the beginning", it is not a parseable major.minor.patch triple, and it needs its own case rather than a parser loose enough to swallow it. Second, at equal versions the closing event sorts before the introduced event that reopens. That pair is a real shape and not a hypothetical: a branch gets closed at a version and a new line of releases opens at the same version. Sorted that way the walk closes and then reopens, so the interval is open at that version, and the verdict holds regardless of the order the file used. internal/match/range_order_test.go exists to hold that property down: the same range, shuffled, one verdict.

The strongest verdict wins across entries

One more place where order would otherwise leak in. Advisories routinely split a single claim across several affected entries: one per branch, one per range type, one per distribution. Stating each separately would put several statements in the document for one claim, and a claim-by-claim reader keeps only the last one it sees, which puts the output back at the mercy of entry order. So the entries fold into one verdict, and the verdict that asks most of a person wins:

    for _, aff := range a.Affected {
        if !samePackage(aff.Package.Ecosystem, aff.Package.Name, eco, name) {
            continue
        }
        s, reason := evaluate(version, aff, eco)
        switch {
        case !stated || rank(s) > rank(status):
            stated, status, reasons = true, s, []string{reason}
        case rank(s) == rank(status) && !slices.Contains(reasons, reason):
            reasons = append(reasons, reason)
        }
    }

// rank orders verdicts by how much they ask of a person.
func rank(s Status) int {
    switch s {
    case Affected:
        return 2
    case Unknown:
        return 1
    default:
        return 0
    }
}

unknown outranking not_affected is the thesis of this whole post compressed into one integer. An entry that could not be compared does not get cleared by a sibling entry that could. And because match exits 1 when anything needs attention, an unknown stops a pipeline instead of sitting as a line in a report nobody opens.

Where leniency does belong

Strictness is not a general virtue here, and it is worth being precise about where it stops.

An explicit affected version list is exact set membership, not ordering, and there I do apply one normalisation: the leading v gets trimmed on both sides, because Go components carry it and OSV lists mostly do not, and a string mismatch there would clear a version that is plainly on the list. Safe, because nothing is being ordered, and deliberately nowhere near the comparator.

The same split runs through the document side of the tool. openvex.New validates what it builds, and Statement.Validate is severe about it: a not_affected with an impact_statement and no justification code gets refused, because prose cannot be compared against every other not_affected the way a code can. But openvex.Parse never calls Validate, so documents already issued are read leniently, including ones that break the rules the tool enforces on write. Refusing them would hide statements someone needs to read.

The rule is not "be strict everywhere". It is: never let leniency manufacture a fact. A tolerant read of a document's shape loses nothing. A tolerant read of a version string produces an ordering that was not in the input, and then a verdict that was not in the input either.

What I would do differently

I would stop treating the unknown count as a defect to engineer down. My first instinct here was the wrong one: a long unknown list reads like the tool failing, and the obvious fix looks like a cleverer parser that gets an answer out of 2023-08-01 somehow. That instinct comes from the wrong domain. In payment systems I have spent years on the opposite reflex: an unsettled transfer is not a successful one, and the unknown state is the expensive one precisely because you cannot collapse it into either neighbour without lying. Vulnerability matching has the same shape. unknown with a reason attached is a work item for a named person; not_affected derived from a guess is a liability with a signature under it.

And I would write the order-independence test before the comparator, not after it. The property (same range, any permutation, one verdict) runs about ten lines, it fails immediately against a naive in-order walk, and it is probably the cheapest way to find out that your evaluator's answer depends on something as fragile as how a feed happened to serialise an array.

The parser is not a detail underneath the range. It is the range.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

Stuck withdrawals: wait, bump or cancel as a pure function

polycratia — Fri, 16 Oct 2026 13:22:37 +0000

A withdrawal that has sat unconfirmed for an hour is not in one state. It is in one of three: too early to touch, worth replacing at a higher fee, or past the point where the only useful move left is to cancel it and hand the money back. Code that models this as a single timeout has to pick one of those three, and it will be wrong about the other two.

I keep running into the same shape in payout systems. A job wakes up, loads everything that is sent, compares a timestamp against a constant, and rebroadcasts. Small constant: it replaces transactions that were about to confirm on their own, and pays the fee twice for nothing. Large constant: support answers the customer by hand hours before the code does anything. Either way the interesting part, what should happen to this one transaction right now, is smeared across a cron expression, a config value and a branch inside a worker that is also holding an RPC connection.

The fix is not a better constant. You want a decision, and the decision should be a value that a function returns.

Three outcomes need two thresholds

One threshold gives you two answers. Three answers need two thresholds, plus a budget for how often you are willing to pay for a replacement.

from datetime import timedelta

from withdrawals import StuckPolicy

policy = StuckPolicy(
    bump_after=timedelta(minutes=10),
    cancel_after=timedelta(hours=2),
    bumps=3,
)

bump_after is where patience ends. cancel_after is where hope ends. bumps is how many replacements the transaction is worth before you stop paying for them.

The constructor refuses an incoherent pair:

StuckPolicy(bump_after=timedelta(hours=2), cancel_after=timedelta(minutes=10))
# ValueError: cancel_after cannot come before bump_after

That check earns its line. An inverted policy does not crash anything: with the cancel condition tested first, it quietly makes the bump path unreachable, and nobody notices until someone asks why no transaction has ever been replaced. The same constructor refuses a policy that allows fewer than one bump. A pipeline that never bumps is spelled by choosing cancel_after, not by hollowing out the middle action.

The decision is a value, not an effect

from datetime import datetime, timedelta, timezone
from decimal import Decimal

from withdrawals import Action, WithdrawalRequest, review

sent_at = datetime(2026, 9, 14, 12, 0, tzinfo=timezone.utc)

request = (
    WithdrawalRequest(
        id="w-1041",
        amount=Decimal("25.00"),
        currency="USDC",
        destination="0x5c69bee701ef814a2b6a3edd4b1652cb9cc5aa6f",
    )
    .approve()
    .start_sending()
    .mark_sent("0xdeadbeef")
)

decision = review(
    request, sent_at=sent_at, policy=policy, now=sent_at + timedelta(minutes=5)
)

assert decision.action is Action.WAIT
assert decision.waited == timedelta(minutes=5)

review takes the request, the moment it went out, a policy and the current time, and returns a Recovery: an action, the reason behind it, and how long the transaction has waited. It opens no connection, reads no clock it was not given, and writes nothing down. now is optional and defaults to the real clock. Tests pass it, so every branch is reachable without sleeping.

The reason is not decoration. It is prose carrying the numbers that produced the decision: unconfirmed for this long, against that deadline. A Recovery has two readers. The dispatch reads action, and the person who gets paged at three in the morning reads reason. Both are served by the same value.

Dispatch then fits in one function with no policy in it:

from withdrawals import bump, return_funds

def act_on(request, *, sent_at, policy, holds, replacement):
    decision = review(request, sent_at=sent_at, policy=policy)
    if decision.is_wait:
        return request
    if decision.action is Action.BUMP:
        return bump(request, replacement)
    return return_funds(request, decision.reason, holds)

The list of replaced references is already the counter

review counts bumps as len(request.replaced). There is no bump_count field, and I would not add one.

bumped = bump(request, "0xfeedface")

assert bumped.reference == "0xfeedface"
assert bumped.replaced == ("0xdeadbeef",)
assert bump(bumped, "0xfeedface") is bumped

You keep the replaced references anyway, for a reason that has nothing to do with counting: any one of them can be the one that gets mined. When a confirmation finally arrives carrying a reference that is no longer the current one, you still have to recognise it as yours. Once that list exists, a separate integer is a second source of truth: you can increment it without replacing anything, or replace without incrementing, and the two will disagree exactly when the transaction is already in trouble.

The same list makes the replacement step total. Bumping to the reference that is already current gives back the same request. Bumping back to one that has been replaced is a conflict, not a no-op:

bump(bumped, "0xdeadbeef")
# InvalidTransition: withdrawal w-1041 already replaced transaction 0xdeadbeef

And the request itself refuses to exist in a state where a reference is current and replaced at once, so the counter cannot be inflated by resubmitting the same hash.

Two budgets that run out independently

Time and attempts are not the same resource, and either one running out means the same thing: stop.

policy = StuckPolicy(
    bump_after=timedelta(minutes=10), cancel_after=timedelta(hours=2), bumps=1
)

decision = review(
    bumped, sent_at=sent_at, policy=policy, now=sent_at + timedelta(minutes=30)
)

assert decision.action is Action.CANCEL

Thirty minutes is comfortably inside the two hours. This transaction is cancelled because the attempt budget is gone, not the time budget.

sent_at comes from the caller, because the package stores nothing and owns no clock. That makes the definition of "waited" a design decision at the call site, so it is worth saying out loud: I pin it to the first broadcast, not to the latest bump. Reset it on every bump and a transaction that gets bumped becomes immortal. Each action the system takes pushes its own deadline out, and the only requests that ever reach cancel_after are the ones nobody touched. The customer's clock does not restart when you rebroadcast.

Cancelling twice has to release nothing twice

The cancel path is the one that moves money back, so it is the one that gets retried by a crash, a redelivered queue message, or an operator clicking twice.

failed = return_funds(request, "unconfirmed past its deadline", holds)
again = return_funds(failed, "unconfirmed past its deadline", holds)

assert again is failed
assert holds.held("w-1041") is None

Two things make that hold. Failing a request that has already failed with the same reason goes through the replay path and returns the identical value; failing it with a different reason raises InvalidTransition instead of quietly overwriting the first explanation, which is the one that is true. And releasing a hold that is not there reports that nothing was released rather than inventing a credit.

The packaged cancel puts the replacement in front of the stuck transaction and then returns the funds. Because bumping to an unchanged reference hands back the same request, running that sequence twice does not manufacture a second replacement or a second release.

What review refuses to answer

Anything that is not a transaction waiting on the chain raises InvalidTransition. A request still in sending has no reference and an unknown outcome, so there is nothing to replace and nothing to measure. A confirmed or failed one is terminal.

That is deliberately a refusal and not a WAIT. WAIT means the chain still deserves the benefit of the doubt. A request that may never have reached the chain is a different problem with a different owner, usually reconciliation, and answering it with "wait" is how it stays invisible for a day.

What I would do differently

In the payout systems I built before I wrote any of this down, the recovery rule lived inside the worker, and the first production surprise was the same every time: someone needs different patience for a different asset or a different rail, and the threshold is global. The signature here allows a policy per call, which I think is the right shape. It only takes the discipline not to add one module-level default policy and reintroduce the problem in a nicer font.

The other boundary I would keep exactly where it is: nothing in the decision function knows what the replacement should cost. Fee strategy belongs to the rail. review says only that a replacement is due, and leaves the caller to produce one.

The decision function is the smallest piece of a withdrawal pipeline, and the first one I would transplant into a new system. It lives in recovery.py in https://github.com/polycratia/withdrawals, next to the holds and the broadcast path it answers to.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

Ledger errors should name the invariant they protect

polycratia — Thu, 15 Oct 2026 13:22:37 +0000

A payout worker catches an exception from the ledger and has to decide, with no human in the loop, whether to retry the request, park it, or escalate. If the refusal says "invalid entry", it cannot decide any of those, so the request goes to a dead-letter queue and the string goes to the logs, where a person reads it an hour later and learns exactly as little. A refusal is part of a ledger's interface, and in most systems it is the one part nobody designed.

I have been building payment and custodial systems since 2018, and the error taxonomy is reliably the piece written last and read most. The balance code gets reviewed twice. The exception that stops a write from happening gets a string literal and a shrug. That is backwards: the write that succeeds is self-documenting, since there is an entry you can read. The write that was refused leaves nothing behind except whatever the message said.

Two readers, one string

The message has two audiences with incompatible requirements. The machine wants something narrow and stable: a key it can branch on, which does not move when someone improves the wording. The human wants the opposite, enough context to know whether money is at risk, and whether refusing was the right call.

"Insufficient funds" serves the machine and leaves the human guessing what the ledger was protecting. "The customer's available balance would have gone below zero, so the write was refused" serves the human and is a terrible thing to pattern-match. The usual compromise is a short code plus a wiki page that explains it, and the wiki page is stale within two quarters because nothing fails when it drifts.

In ledger-core (https://github.com/polycratia/ledger-core) I split it differently. The exception class is the branch key. The promise it protects is a class attribute on that class. The message is the two of them joined at raise time, and the bare reason stays reachable on its own.

class InvariantViolated(ValueError):
    """A refused write, carrying the promise that refused it."""

    protects: ClassVar[str] = "a rule of the ledger"

    def __init__(self, detail: str) -> None:
        super().__init__(f"{detail} \u2014 protects: {self.protects}")
        self.detail = detail


class InsufficientFunds(InvariantViolated):
    """Raised when a reservation or a movement takes more than is available."""

    protects: ClassVar[str] = (
        "available balances stay at or above zero, so the same funds are never spent twice"
    )

There are three of these: UnbalancedEntry, InsufficientFunds, ClosedAccount. Each overrides protects with one sentence written from the ledger's point of view, not the caller's. What reaches a log line is the specific reason and the general promise, separated by an em dash so the two halves stay legible as two halves. What reaches code is the class, plus .detail holding the reason without the promise appended.

That last part matters more than it looks. The promise sentence is editorial copy. Someone will reword it, because it is the sort of thing that gets reworded in review. If callers are storing the full message as a rejection reason on a request row, every rewording changes historical data and any accumulated string matching. .detail is the half that does not carry the editorial text, so it is the half you persist and forward.

Enforced at the one place it can be broken

A promise is only worth printing if it holds where the caller forgot to check. So each of the three is enforced at the single point where it could be violated, and nowhere else pretends to cover it.

Journal.append asks only whether an entry balances. It is not a funds check and does not imply one. The funds check lives in Holds.spend, which computes -effect_on(entry, account), compares that against available(), raises InsufficientFunds if it exceeds it, and only then appends. Closure is enforced where a closed account can actually be touched: in Account._posting, and in Hold.__post_init__.

    def _posting(self, amount: MoneyLike, side: Side) -> Posting:
        if self.closed_at is not None:
            raise ClosedAccount(
                f"account {self.account_id} was closed at {self.closed_at}, "
                f"{side.value} of {amount} is refused"
            )

Putting it there has a consequence worth stating plainly: an Entry.transfer against a closed account raises before the journal appears anywhere in the call stack. The refusal happens while the write is still a value. Same for a one-sided movement: an unbalanced entry never becomes an object, so it cannot be discovered later by something reading the journal. And closing an account refuses new movement without erasing anything, since a closed account's balance still folds from the entries it already has.

Branch on the class, store the detail

With the taxonomy on the classes, the worker's decision is a try block and nothing else:

from ledger_core import ClosedAccount, InsufficientFunds


def settle(holds, entry, account, requests):
    try:
        holds.spend(entry, account=account)
    except InsufficientFunds as refused:
        requests.park(entry.entry_id, reason=refused.detail)
        return "retry_after_funding"
    except ClosedAccount as refused:
        requests.escalate(entry.entry_id, reason=refused.detail, alert=str(refused))
        return "manual"
    return "settled"

The two branches differ because the underlying facts differ. InsufficientFunds is a statement about a quantity that changes: a deposit lands and the same instruction succeeds, so retrying is meaningful. ClosedAccount is a statement about a terminal state, so retrying is guaranteed waste and a human needs to look. The alert carries str(refused), which includes the promise, because the person being woken up has not read this module. The parked row carries refused.detail, because repeating the same promise sentence across ten thousand rows is noise.

Not every refusal is an invariant

The holds module also raises CaptureMismatch, DuplicateHold, HoldNotOpen and UnknownHold. None of them subclass InvariantViolated, and that is deliberate.

Holds.capture skips the funds check entirely: the funds were reserved when the hold was placed, and the reservation is what the balance has been standing behind since. What capture checks is that the amount taken equals hold.amount.amount, raising CaptureMismatch if it does not. That is not a money problem. It is the caller's instruction disagreeing with the caller's own reservation, and collapsing it into InsufficientFunds would make the retry decision wrong: one is worth re-reading a balance for, the other will fail identically forever.

So the dividing question is this: does this refusal protect a promise the ledger makes to everyone, or does it report that the instruction disagrees with state the caller named? The first gets a protects sentence. The second gets a precise message and a plain exception type. That is the only taxonomy rule I have found that keeps working when a new error case shows up six months later.

The test that keeps it honest

Because protects has a default, a subclass that forgets to override it inherits "a rule of the ledger", which is worse than silence, since it looks like a promise and says nothing. One parametrised test closes that:

@pytest.mark.parametrize("violation", [UnbalancedEntry, InsufficientFunds, ClosedAccount])
def test_every_violation_names_what_it_protects(violation: type[InvariantViolated]) -> None:
    assert issubclass(violation, InvariantViolated)
    assert violation.protects != InvariantViolated.protects
    assert violation.protects in str(violation("refused"))

It asserts the override exists and that the sentence actually reaches the message. Since protects is a class attribute, it is readable without raising anything, which means documentation can be generated from the classes rather than maintained beside them.

What I would do differently

First, the default. A ClassVar default makes the base class convenient and makes forgetting silent, and a test is a weaker guard than a definition-time failure. I would check the override in __init_subclass__ so a new invariant without a promise fails at import rather than in CI.

Second, I would write down, in the base class docstring, that the message is for humans and the class is for code. Without that sentence, someone eventually matches on the full string, and then the promise sentence is load-bearing text that cannot be edited.

Third, the gap a refusal cannot close. Holds.place and Holds.spend read available() and then write, as two steps, and the Storage protocol has no transaction, so the atomic pair is left to the backend. InsufficientFunds is then a true statement about what available() was a moment before the append, not a guarantee about what it was during it. Where more than one writer posts to one account, making that pair a single step belongs to the store underneath. A message that names its invariant is honest about the rule, and it cannot be honest about a race the layer below allows. That belongs in the exception's own docstring, next to the promise, so nobody reads the promise as stronger than it is.

A ledger earns trust by what it refuses, and a refusal nobody can act on is barely better than a write nobody checked. Making the promise a class attribute and putting it in the message costs about four lines in one module. What you get for that is that the retry logic and the person on call are reading the same refusal and reaching the same conclusion about it.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

A 402 is a bill: what an x402 gate owes a client when things fail

polycratia — Wed, 14 Oct 2026 13:21:09 +0000

An HTTP 402 gate sits between a client's money and a service that can fail on either side of it. The handler can fail. The payment network can fail. Those two failures need opposite answers: a failed handler must charge nothing, while a failed facilitator must tell the client the gate is broken, not that it owes more. Nearly every bug I have hit in paid-request plumbing is one of those two answers given in the other's place.

In ordinary HTTP a status code is a statement about the request. On a paid request it is also a statement about money, and that changes what you are allowed to say. A 402 means: you have not paid for this, here are the terms, come back with a payment. Return it because your own facilitator stopped answering and you have just told a client to sign a second authorisation to cover your outage. Settle before the handler has finished and you have billed someone for a 503.

I wrote paygate402 (https://github.com/polycratia/paygate402) around that single asymmetry. The protocol part of x402 is short: unpaid request, 402 with terms, repeat with an X-PAYMENT header, verify, settle, serve. The happy path is not where the work is. It is the four or five places where something fails and the gate has to decide whose problem it is.

Two failures, two answers

The handler side is the easy half, as long as you commit to it early. The client's money must only move for work that was delivered. So settlement cannot happen before the handler has run, and the handler's output cannot reach the client before settlement has succeeded. Both constraints point at the same implementation: the response goes into a buffer.

The payment side is the half that gets written wrong, because the wrong version looks like idiomatic Go.

The facilitator answers on two channels

A facilitator is the seam where the chain lives: signature recovery, allowance checks, the actual transfer. From the server's point of view it is an HTTP call that can fail, and that gives it two distinct ways of saying no:

// Facilitator verifies payments and settles them. This is the seam where the
// chain lives: signature recovery, allowance checks and the actual transfer all
// happen behind it.
type Facilitator interface {
    // Verify reports whether a payment is good for the given terms. A returned
    // error means the question could not be asked; a Verification with
    // Valid=false means it was asked and answered no.
    Verify(ctx context.Context, payment Payment, terms Requirements) (Verification, error)
    // Settle moves the money and reports the transaction.
    Settle(ctx context.Context, payment Payment, terms Requirements) (Settlement, error)
}

type Verification struct {
    Valid  bool   `json:"isValid"`
    Reason string `json:"invalidReason,omitempty"`
    Payer  string `json:"payer,omitempty"`
}

The error is "the question could not be asked". The Valid field is "it was asked and answered". Those are not degrees of the same thing, and the line that treats them as one is the bug:

// Wrong, and it reads fine.
if v, err := g.Facilitator.Verify(ctx, payment, terms); err != nil || !v.Valid {
    g.challenge(w, accepts, "payment was not accepted")
    return
}

That || converts a network timeout into a demand for money. The client, especially an automated one, does exactly what the status code tells it to: signs another payment and retries into the same outage. The correct shape keeps the channels apart, and the comment is in the repository because it is the reason the branch exists at all:

verification, err := g.Facilitator.Verify(r.Context(), payment, terms)
if err != nil {
    // The facilitator is unreachable. This is the server's problem, not
    // the client's: answering 402 would tell them to pay again.
    http.Error(w, "payment verification is unavailable", http.StatusBadGateway)
    return
}
if !verification.Valid {
    g.challenge(w, accepts, orDefault(verification.Reason, "payment was not accepted"))
    return
}

One more detail about Verification.Payer: inside the handler, Payer(r.Context()) returns the address the facilitator said paid. It is the facilitator's claim, not the web layer's finding. A web layer cannot honestly check a payer, an amount, an asset or a recipient, so it should not pretend to. It should record whose claim it is relaying.

Settlement comes after the handler

The handler runs into a recorder, and the buffered body is only written out once money has actually moved:

recorder := &recorder{header: http.Header{}, status: http.StatusOK}
next.ServeHTTP(recorder, r.WithContext(withPayer(r.Context(), verification.Payer)))

if recorder.status < 200 || recorder.status >= 300 {
    // Nothing was delivered, so nothing is charged.
    recorder.flush(w, "")
    return
}

settlement, err := g.Facilitator.Settle(r.Context(), payment, terms)
switch {
case err != nil:
    g.log("settlement call failed", "error", err)
    http.Error(w, "payment settlement is unavailable", http.StatusBadGateway)
    return
case !settlement.Success:
    g.log("settlement declined", "reason", settlement.ErrorReason)
    g.challenge(w, accepts, orDefault(settlement.ErrorReason, "payment could not be settled"))
    return
}

Read the two cases of that switch next to each other. A settlement call that errored is a 502 and a log line. A settlement the facilitator declined is a 402 carrying the facilitator's own reason. Same function, same return site, opposite audiences: the first is for an operator, the second is for a client who can act on it.

The non-2xx branch is the part that earns the buffer. A 503 from the upstream the handler depends on, a 404 for a report that does not exist, a panic turned into a 500: none of those produce a charge, because none of them produced a response. The cost of that guarantee is real and worth saying plainly. A buffered response is a response held in memory, and you cannot stream a paid body while also promising it was never delivered unsettled. Buffering is choosing the promise over the streaming. For priced API responses that is the right trade; for large downloads it is not, and that boundary is a design decision rather than an implementation detail.

The log line matters too. Settlement failures are the events an operator most needs to see, because they are the only ones where the work was done and the money did not arrive. Everything else you can read off the status code distribution.

Where the rule bends, and why

There is one place where "charge nothing for failed work" does not fully hold, and pretending otherwise would be worse than admitting it. The replay key for a payment is recorded after verification passes, and once recorded the payment stays spent even if settlement then fails:

Recording at first sight would burn a payload the facilitator rejected, because a client that fixes its allowance will legitimately resend the same signed payment. But once settlement has been attempted, that attempt may already have reached the chain. The server does not know. The ambiguous case is resolved against the possibility of a double charge, which means the client gets a 502 and the old payload is refused if it comes back.

So a client that hits a settlement outage has to produce a fresh payment instead of retrying the stale one. That is friction, and it is deliberate: the alternative is a path where the same authorisation can be settled twice because a timeout looked like a failure.

The same "whose failure is it" question decides a smaller case. When receipts are enabled and a receipt fails to sign, the failure is logged and the paid response still goes out. The money moved, the service was rendered, and withholding the body would punish the client for a problem with the server's signing key.

Free requests follow the same logic

The free tier is spent before the X-PAYMENT header is even read:

if client, left, granted := g.Free.take(r); granted {
    g.serveFree(w, r, next, client, left)
    return
}

A client with free requests remaining is not charged, even if it sent a payment anyway: the payment is left unspent rather than taken. And serveFree calls Allowance.give when the handler returns a non-2xx, giving the free request back. Work that was not delivered is not billed, whether the unit of billing is a stablecoin transfer or a slot in a courtesy allowance. A route that the price table does not mention is served unpaid for the same kind of reason: the table is the whole statement of what is sold, so a health check needs no line saying it costs nothing.

Testing the refusals, not just the success

The distinction is only real if there is a test that fails when someone collapses it. Two fakes exist for that: StaticFacilitator, which answers from a fixed script and records its calls, and MockFacilitator, which keeps balances in memory and exposes VerifyErr and SettleErr to fail the way an unreachable facilitator does.

func TestFacilitatorOutageIsNotTheClientsProblem(t *testing.T) {
    facilitator := &StaticFacilitator{VerifyErr: errors.New("dial tcp: connection refused")}
    g := &Gate{Accepts: accepts, Facilitator: facilitator}

    w := httptest.NewRecorder()
    r := httptest.NewRequest(http.MethodGet, "https://api.example.com/report", nil)
    r.Header.Set(PaymentHeader, encoded)
    g.Handler(paid).ServeHTTP(w, r)

    if w.Code != http.StatusBadGateway {
        t.Errorf("status = %d, want 502 when the facilitator cannot be reached", w.Code)
    }
    if facilitator.Trail() != "verify" {
        t.Errorf("calls = %q: settlement ran after verification could not be asked", facilitator.Trail())
    }
}

The Trail assertion is the one I would not drop. A status code test tells you the client was answered correctly. The call trail tells you the gate did not quietly attempt to move money on a question it never got an answer to.

What I would do differently

The two unavailability responses are prose: "payment verification is unavailable" and "payment settlement is unavailable". For a human operator reading logs that is fine. For the agent clients this protocol is aimed at, it means string matching to tell a transient server fault from a permanent one, and I would rather hand them a small machine-readable body with a stable code. Related: x402 defines a shape for "you must pay" and no shape at all for "come back shortly", so a 502 from this gate carries no retry guidance beyond what HTTP already gives you. A Retry-After on the facilitator-outage path costs nothing and is the only hint a client can act on without guessing.

I would also make the buffer's ceiling explicit instead of leaving it implicit in the host's memory, and I would surface the settlement ambiguity in the response rather than only in the log. A client that is told "this payment is spent, sign a new one" can act correctly, while a client that is told only "unavailable" will reasonably retry the payload that will now be refused.

None of this changes the rule the gate is built around. A 402 is a bill, and a bill is only honest when the service was delivered and the payment was actually judged and declined. Everything else the server has to own in its own status code.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.