DEV Community: polycratia

Refusing a snapshot a deposit watcher cannot fully read

polycratia — Sat, 14 Nov 2026 13:22:08 +0000

Credit an on-chain deposit exactly once and you have to remember which payments you already reported, and that memory has to outlive the process. Once the memory becomes a serialised value, the restart path picks up a failure mode worse than a crash: a snapshot that is almost readable, read anyway, and a watcher that notifies a second time without noticing.

I keep running into this shape in custodial wallet and on-ramp work. Crediting is a one-way door: the balance moves, and the customer can spend it. A chain carries no matching obligation and may change its mind about what happened a few blocks back. Everything between those two facts sits in the watcher's bookkeeping, and that bookkeeping is all a restart carries. In chain-watch it is a value called WatcherState, and the decision worth writing about is not what it stores: it is what its loader refuses to accept.

A partial restore is a silent double-credit

WatcherState holds five things: start_height, last_tip, the pending transfers that are not deep enough yet, the reported deposits still inside the reorg window, and settled, the keys of deposits buried below it. Each field fails differently if you load it loosely.

Drop settled and the watcher comes back believing it never reported anything. The chain source gives you a view of the chain as it looks right now, not a log of what was ever seen, so the next poll hands back every transfer still visible at or above the read floor, and the watcher reports all of them again. Nothing errors. The customer gets credited twice.

Misread last_tip or start_height as something other than the integer they were written as, and the read window moves. last_tip exists so a resumed watcher reads from the block after the one it stopped at, however far the chain moved on in the meantime. A height that comes back wrong does not break the loop: it widens or narrows what the next poll looks at, which is the same double-report or missed-deposit bug with a different trigger.

Accept a non-positive amount and the watcher reports a deposit for it. A credit of a negative amount is a debit. Also not a crash.

None of this is exotic. It is what you get when a loader does its best with whatever it was handed. A watcher that starts with no snapshot at all is honest about it: it starts cold, and whoever operates it knows the cost. A watcher that starts from a snapshot it understood most of is not honest about anything, and the guarantee it exists to provide is gone before the first poll.

So the loader is strict on purpose, and the strictness is all-or-nothing: WatcherState.from_dict either builds a state it fully read or raises.

Name the field, not the file

Strictness has to buy a usable error first. A KeyError out of a nested dict comprehension tells you a key was missing somewhere in a snapshot. At three in the morning that is not enough to act on.

def _field(data: Mapping[str, object], name: str, label: str) -> object:
    if name not in data:
        raise InvalidState(f"{label} is missing {name!r}")
    return data[name]

Every read goes through it, and the label travels with the nesting level, so a transfer missing its amount says so as a transfer rather than as a dict. InvalidState subclasses ValueError, which matters for the callers who load a snapshot inside a startup path that already catches value errors around configuration parsing.

Type every scalar, including the bool that is an int

JSON is not a type system. A value that went out through to_dict as an int can come back as a string, a float, a list, or True, depending on what touched the file in between. So every scalar gets a checked accessor:

def _as_int(value: object, label: str) -> int:
    if isinstance(value, bool) or not isinstance(value, int):
        raise InvalidState(f"{label} must be an int: {value!r}")
    return value


def _as_sequence(value: object, label: str) -> Sequence[object]:
    if isinstance(value, (str, bytes)) or not isinstance(value, Sequence):
        raise InvalidState(f"{label} must be a list: {value!r}")
    return value

The two explicit refusals are the interesting lines, and both are there because Python's type hierarchy is more generous than the data model here.

bool is a subclass of int. Without the first clause, a confirmations field holding True passes an isinstance(value, int) check and becomes a depth of one. A deposit reported with one confirmation when it really had twelve is the kind of record that reads as plausible in a support ticket and is wrong.

str is a Sequence. Without the first clause of _as_sequence, a string where a list belongs gets accepted as a list, then decomposed character by character, and the failure surfaces somewhere further in with a message about something unrelated. Refusing it at the door keeps the error pointing at the field that is actually wrong.

The deposit keys stored in settled are written as two-item lists, so they get a shape check on top of the element checks:

def _key_from_list(value: object) -> DepositKey:
    pair = _as_sequence(value, "deposit key")
    if len(pair) != 2:
        raise InvalidState(f"deposit key must hold two items: {value!r}")
    return (_as_str(pair[0], "tx_id"), _as_int(pair[1], "output_index"))

A DepositKey is the (tx_id, output_index) pair naming the transaction output a payment landed on. It is the identity the watcher deduplicates by, and the most load-bearing thing in the snapshot. A key that loses its second element is not a key.

Rebuild values through the door a chain source uses

The other half of the design is that from_dict does not construct raw tuples of fields. It rebuilds Transfer, BlockRef and Deposit through the same constructors a live chain source feeds, so each one runs its own __post_init__ validators on the way back in:

def _transfer_from_dict(value: object) -> Transfer:
    data = _as_mapping(value, "transfer")
    return Transfer(
        tx_id=_as_str(_field(data, "tx_id", "transfer"), "tx_id"),
        output_index=_as_int(
            _field(data, "output_index", "transfer"), "output_index"
        ),
        address=_as_str(_field(data, "address", "transfer"), "address"),
        amount=_as_str(_field(data, "amount", "transfer"), "amount"),
        asset=_as_str(_field(data, "asset", "transfer"), "asset"),
        block=_block_from_dict(_field(data, "block", "transfer")),
    )

That is what keeps a snapshot from being a back door around the invariants. Transfer.__post_init__ runs the amount through _as_amount:

def _as_amount(value: Decimal | int | str) -> Decimal:
    if isinstance(value, float):
        raise InvalidTransfer("amount must not be a float; use Decimal, int or str")
    if isinstance(value, Decimal):
        amount = value
    else:
        try:
            amount = Decimal(value)
        except (ArithmeticError, TypeError, ValueError) as exc:
            raise InvalidTransfer(f"amount is not a number: {value!r}") from exc
    if not amount.is_finite():
        raise InvalidTransfer("amount must be finite")
    if amount <= 0:
        raise InvalidTransfer(f"amount must be positive: {amount}")
    return amount

Note that the loader reads amount with _as_str, not as a number. to_dict renders it with str, and reading it back as text means the Decimal gets reconstructed from the digits that were written, not from whatever a JSON parser decided a float should be. A deposit of 0.10000000 survives the round trip with its trailing zeros intact. Same reason the type refuses a float outright on the way in: a payment that is rounded once is rounded forever, and the restart path is a particularly undignified place to lose a satoshi.

Because those validators raise InvalidTransfer, the loader catches it and re-raises it as InvalidState saying the snapshot holds unusable data. The caller gets one error type for the whole operation. Anything else forces every startup path to catch two exceptions that mean the same thing, and someone will eventually catch only one of them.

The version is a refusal, not a hint

to_dict writes version as STATE_VERSION, currently 2. from_dict compares it and raises InvalidState on any other value instead of attempting a best-effort read.

This is the part people argue about, so I will put the trade plainly. A tolerant loader that ignores an unknown version keeps the process starting after a rollback or a downgrade, and it buys that with exactly the bug this whole design exists to prevent: an older layout whose settled list is absent or shaped differently loads as an empty set, and the watcher replays. The strict loader refuses to start, loudly, with the version in the message. Refusing to start is an outage. Replaying is a double-credit. One of those a human reverses in a few minutes, and the other you reverse with an apology.

The discipline the version buys is also what keeps the format changeable later. Because the loader has never accepted anything it did not fully understand, a future layout arrives by bumping the number and handling exactly the two shapes that ever existed, instead of guessing which of several historical half-formats a file in the wild belongs to.

What the strictness buys at the boundary

Once the snapshot is known to be complete, the constructor adopts it directly, with no defensive code of its own:

        if state is not None:
            self._last_tip = state.last_tip
            self._pending.update((t.key, t) for t in state.pending)
            self._reported.update((d.key, d) for d in state.reported)
            self._settled.update(state.settled)

And the property that produces a snapshot writes pending, revertible and settled sorted, so equal bookkeeping renders as one dict. The round trip turns into something a test can state as an equality instead of as a set of spot checks:

    snapshot = json.loads(json.dumps(watcher.state.to_dict()))
    resumed = DepositWatcher(
        chain,
        ["addr-1"],
        policy=2,
        reorg_depth=3,
        state=WatcherState.from_dict(snapshot),
    )
    assert resumed.state == watcher.state

The json.dumps and json.loads in the middle are not decoration. Testing from_dict(to_dict(state)) on live Python objects proves close to nothing, because the objects never degrade into the string-keyed, loosely typed data the loader is actually written to defend against. The serialisation step is what makes the test exercise the real boundary.

What I would do differently

Two things in this area are not settled.

The first is the ambiguity of a defaulted argument. start_height and state are mutually exclusive, but zero is the default, so a start_height=0 passed alongside a state cannot be told apart from one that was never passed at all. It is not refused, and the snapshot's own start_height wins (the alternative would drop the floor of a resumed watcher to zero and widen every read below the height the snapshot was taken from). I think that is the right behaviour, and it rests on an argument about defaults rather than on anything the type system enforces. A sentinel would make the exclusivity real instead of documented...

The second is that settled only grows, which is what the exactly-once guarantee costs. A watcher running long enough on a busy address set accumulates keys forever, and a growing snapshot eventually becomes slow to write and large to read. Pruning it means deciding when a key is old enough that the chain can no longer surface the transfer, which is a claim about the chain and not about the watcher. I would rather have the strict loader and an unbounded set today than a clever pruning rule I cannot defend.

Strict loading looks like pedantry in a diff. It is the only place in the system where the exactly-once property can actually be enforced, because by the time the first poll runs, the watcher has already decided what it remembers.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

Reserving a length prefix and its body in one bounds check

polycratia — Fri, 13 Nov 2026 13:21:36 +0000

A serialiser that writes a count and then writes the payload it counted has two failure points, and the second one is the dangerous one: the count has landed in the buffer, the payload has not, and whatever reads that buffer back trusts a length nothing backs. You don't close that gap with a better error path after the fact. You close it by reserving the prefix and the body in a single bounds check, then refusing the whole field before a single byte of it goes down.

I have written length-prefixed framing for card-rail payment messages, for on-chain payloads, and for service-to-service protocols that were never supposed to meet hostile input and eventually did. The partial-write bug has the same shape every time, and it does not look wrong in review.

The second failure point is the one that escapes

Here is the shape I am arguing against. It is not how bytespan does it; it is how almost everything else does it.

// Two calls, two bounds checks, two chances to refuse.
bool write_blob(std::span<std::byte> out, std::size_t& at,
                std::span<const std::byte> payload) {
    if (!write_u16(out, at, static_cast<std::uint16_t>(payload.size()))) return false;
    return write_bytes(out, at, payload);   // the failure that matters
}

If out has room for the count but not for the body, this returns false and the buffer now ends with a count and nothing behind it. The failure gets observed on the writing side. The damage is on the reading side, later, quite possibly in another process. A reader that trusts the count reads past the end of the record and into whatever follows it: the next frame, the next record, uninitialised memory. The caller that logged the write error has no idea it produced a readable lie.

There is a second defect in those four lines, and it is the cast. A payload of 65536 bytes becomes a count of 0. Truncating a count is worse than refusing, because the result is well-formed: nothing downstream can tell it apart from an empty field. A payload too large to be counted in its prefix has to fail the writer.

One check, then two writes that cannot refuse

In bytespan both prefix widths go through one private template, and length_prefixed_u16 and length_prefixed_u32 are the two instantiations:

template <typename T>
bool length_prefixed(std::span<const std::byte> payload, endian order) noexcept {
    if (!ok()) return false;
    const std::size_t size = payload.size();
    if constexpr (sizeof(std::size_t) > sizeof(T)) {
        if (size > static_cast<std::size_t>(std::numeric_limits<T>::max())) {
            poison({offset_, reason::too_long, size,
                    static_cast<std::size_t>(std::numeric_limits<T>::max())});
            return false;
        }
    }
    // Prefix and body are reserved in one check, so a refusal cannot leave
    // a lone count behind. Subtracting from the room left, rather than
    // adding the two, keeps the sum from wrapping on a 32-bit size_t.
    const std::size_t room = out_.size() - offset_;
    if (size > room || sizeof(T) > room - size) {
        poison({offset_, reason::no_room, detail::saturating_add(sizeof(T), size), room});
        return false;
    }
    put(static_cast<T>(size), order);
    return bytes(payload);
}

The order is the design. First the poisoned-writer guard, so a field after an earlier refusal is a no-op instead of a write into a buffer whose offset nobody trusts any more. Then the counting question, decided entirely on the payload size and the prefix type: a payload larger than std::numeric_limits<T>::max() is reason::too_long, with the payload size as wanted and the prefix maximum as available. That test sits inside if constexpr, so it disappears where std::size_t and T have the same width and the comparison could not fail. A 32-bit size_t with a u32 prefix has nothing to check.

Only then the room question, and you ask it once for both pieces. After it passes, the two writes are not failure points any more. They go through the same private helper every other write uses:

bool take(std::size_t n) noexcept {
    if (!ok()) return false;
    if (n > out_.size() - offset_) {
        poison({offset_, reason::no_room, n, out_.size() - offset_});
        return false;
    }
    offset_ += n;
    return true;
}

Both take calls are bound to succeed, because the room for sizeof(T) + size was established a line earlier. That is why put has its return value ignored there, the one place in the writer where ignoring it is defensible. What you get out of it is the property worth having: written() either moves past the whole field or does not move at all. There is no state in which a count exists for a reader to trust without its body.

The arithmetic is part of the guarantee

The natural way to write that bounds check is sizeof(T) + size > room, and on a 32-bit size_t it is a vulnerability. A payload size near the top of the type plus two bytes of prefix wraps to something small, the comparison passes, and the writer goes ahead and places a count it cannot back. Subtract instead, size > room || sizeof(T) > room - size, and every term stays bounded by room; the second comparison is only reached once the first has established that room - size does not underflow. Same question, no sum, nothing to wrap.

The refusal record carries the same hazard for a different audience. wanted is meant to name what the field needed, which is prefix width plus payload, so computing it means adding exactly the two numbers the check just refused to add. detail::saturating_add folds them together and clamps instead of wrapping; a diagnostic that lies about the size it refused is worse than no diagnostic at all. The reader side is symmetric: a header claiming 65535 bytes of payload in a 4-byte input reports wanted 65537 and available 4, at the offset of the prefix rather than somewhere inside the field.

And the record is the first refusal. poison keeps only what it is given first:

void poison(refusal record) noexcept {
    if (!refusal_) refusal_ = record;
}

Once a writer is poisoned, every later call fails as a consequence. Letting those consequences overwrite the record would bury the one failure that explains the output.

When the length is not known until the body is written

Reserving both halves requires knowing the size of the body before you write it, and sometimes you do not: the body gets assembled by nested calls that are themselves writers of fields. For that there is the slot API. open_u16 and open_u32 reserve the prefix, zero it, and hand back a length_slot, and close fills it in with what was written since.

const auto slot = w.open_u16();
if (!slot) return false;      // the one call you do have to check
// ... write the body through w ...
w.close(*slot, bytespan::endian::big);

This path writes the prefix first, so the invariant has to be weaker, and it is made explicit rather than left to luck. open zeroes the reserved bytes, because the caller's buffer need not have arrived zeroed and an unclosed field has to read as zero length instead of as whatever was sitting there. close computes body = offset_ - (slot.offset + slot.width), refuses a body too large for its width as too_long at the field's own offset, and leaves the zeros alone when it does. A slot closed out of order, closed twice, or opened on another buffer is bad_slot, recorded at the offset the writer had reached: such a slot's own offset may name a position in a buffer that has nothing to do with this one.

A varint written whole

The same rule applies where the field has no prefix at all. varint_u64 emits unsigned LEB128, seven bits per byte, low group first, and a half-written one is not merely short: the last byte it placed has its continuation bit set, pointing the reader at whatever happens to come next. Since the reader accepts only the canonical encoding and judges each byte as it reads it, a truncated varint followed by unrelated bytes can decode cleanly to a value the writer never meant.

So the writer builds the groups into a std::array of detail::varint_u64_max_bytes, ten, the most a 64-bit value can need, and hands them to bytes in one call. One bound, one write, all or nothing. The property, as a test:

void a_varint_that_does_not_fit_writes_nothing() {
    std::array<std::byte, 2> buffer{};
    writer w{buffer};

    CHECK(!w.varint_u64(0xFFFFFFFFFFFFFFFFull));
    CHECK(!w.ok());
    CHECK(w.written() == 0);
}

What I would do differently

Two things. The first is that length_prefixed takes a std::span<const std::byte>, which means the body has to exist contiguously in memory before the field can be written. That is what makes the single bounds check possible, but it splits the API: a body you can measure takes the strong path that refuses before writing, and a body you cannot takes the slot path that leaves zeros behind. I spent a while trying to collapse those into one call and concluded the split is honest, it falls exactly where the knowledge falls, but it is still two failure stories for one concept, and a caller has to know which one they are in.

The second is smaller and bothers me more: put(static_cast<T>(size), order); with a discarded bool is correct for a reason that lives three lines above it. Correct code that reads like a missing check is a maintenance hazard, and I would rather the discard were loud than rely on the comment surviving the next edit...

A count is a promise about the bytes that follow it. Under a failing write, the only way to keep that promise is to decide you can keep it before you make it.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

One weight estimator for script types and size hints

polycratia — Thu, 12 Nov 2026 13:21:36 +0000

A coin selector prices an output before anyone tells it what kind of output it is. A caller that tracks script types wants the witness discount to land in the fee; a caller holding nothing but a byte count still wants a number. So you write two fee calculators, one per description, and the two of them disagree by a byte at the edges. A byte at the edges is how you ship a transaction that will not relay.

I have been building custodial wallet services and on-chain payment systems since 2018, and fee estimation is the part of that work that rots quietly. It rots because the estimate lives in several places (a selection routine, a dust check, a candidate ranking) and each place rounds a little differently. In utxo-select (github.com/polycratia/utxo-select) I collapsed the two description paths into a single weight estimator. The mechanism that makes it possible fits in one sentence: a size hint gets priced as a legacy input or output stretched to that size.

Two descriptions of the same output

A Utxo states what it will cost in one of two ways.

from utxo_select import ScriptType, Utxo

hinted = Utxo(txid="a" * 64, vout=0, value=200_000, confirmations=12)

typed = Utxo(
    txid="b" * 64,
    vout=0,
    value=200_000,
    confirmations=12,
    script_type=ScriptType.P2WPKH,
)

script_type names the script kind, and the input is priced by weight. script_size is the older hint: a virtual size for the input's signature script, defaulted to DEFAULT_INPUT_SCRIPT_SIZE, which is 107, the scriptSig of a legacy P2PKH input. A caller gets the hint without asking for anything, and that is the entire reason it stays supported. Target and the change description on SelectionRequest carry the same pair, with DEFAULT_OUTPUT_SCRIPT_SIZE of 25.

There is no third state. _check_script_type raises TypeError for anything that is neither a ScriptType nor None, and once a type is given the hint goes unread. The precedence sits in one place instead of getting argued about at every call site.

The arithmetic behind the hint is the standard bill: roughly 10 virtual bytes of transaction overhead, then 41 vbytes plus the scriptSig per input, then 9 vbytes plus the scriptPubKey per output. With the defaults of 107 and 25, an input comes to 148 vbytes and an output to 34. At fee_rate=12_000 (12 base units per vbyte, expressed per 1000 vbytes so that finer rates stay in integer arithmetic) that is 1_776 for every input and 408 for every output, whatever they hold.

The hint is a stretch, not a second model

The whole of the translation is this:

def input_weight(script_type: ScriptType | None, script_size: int) -> int:
    """Weight one input adds, from its script type or from its size hint.

    The hint is read only when no script type is given, and then as the
    unlocking script of a legacy input stretched to it.
    """
    if script_type is not None:
        return script_type.input_weight
    return (INPUT_OVERHEAD_VSIZE + script_size) * WITNESS_SCALE_FACTOR


def output_weight(script_type: ScriptType | None, script_size: int) -> int:
    """Weight one output adds, from its script type or from its size hint."""
    if script_type is not None:
        return script_type.output_weight
    return (OUTPUT_OVERHEAD_VSIZE + script_size) * WITNESS_SCALE_FACTOR

The hinted branch does not return a vsize. It returns weight, scaled by WITNESS_SCALE_FACTOR, because every byte of a legacy shape is non-witness data and so gets scaled in full. From the estimator's point of view there is one currency now.

The tables it meets on the other branch are written out the same way. An input's non-witness part is a 36-byte outpoint, the scriptSig with its length varint, and 4 bytes of sequence; the witness bytes are counted once:

_INPUT_WEIGHT: dict[ScriptType, int] = {
    # scriptSig 1 + 72 signature + 1 + 33 pubkey, so 148 bytes in total.
    ScriptType.P2PKH: 148 * WITNESS_SCALE_FACTOR,
    # 41 bytes with an empty scriptSig; witness 1 + 1 + 72 + 1 + 33.
    ScriptType.P2WPKH: 41 * WITNESS_SCALE_FACTOR + 108,
    # 41 bytes with an empty scriptSig; witness 1 + 1 + 64 Schnorr signature.
    ScriptType.P2TR: 41 * WITNESS_SCALE_FACTOR + 66,
}

The two paths meet in one place, and at the defaults they agree exactly. A hinted input is (41 + 107) * 4, which is 592; a declared ScriptType.P2PKH input is 148 * 4, the same 592. On the output side (9 + 25) * 4 and 34 * 4 are both 136. I would rather not be grateful for that as a coincidence: it is a property worth a test, and it has one:

Selection	`vsize`	`fee` at 12_000
Default hints, one input, one target, one change	226	2_712
Same shape declared `P2PKH` throughout	226	2_712
Same shape declared `P2WPKH` throughout	141	1_692

The third row is what the hint cannot express at all. A size in vbytes has nowhere to put the witness discount.

Only the whole transaction knows whether it carries a witness

Two more things reach the fee that no per-part number can see: a segwit transaction carries a marker and flag once, and a legacy input sitting alongside witness inputs still pays for an empty witness stack. Both belong to the assembled transaction, so both get decided in the estimator and nowhere else.

def estimate_shaped_weight(
    inputs: Iterable[tuple[ScriptType | None, int]],
    outputs: Iterable[tuple[ScriptType | None, int]],
) -> int:
    weight = _OVERHEAD_WEIGHT
    witness = False
    legacy = 0

    for script_type, script_size in inputs:
        weight += input_weight(script_type, script_size)
        if script_type is not None and script_type.is_segwit:
            witness = True
        else:
            legacy += 1

    for script_type, script_size in outputs:
        weight += output_weight(script_type, script_size)

    if witness:
        weight += _MARKER_WEIGHT + legacy * _EMPTY_WITNESS_WEIGHT

    return weight

Note which side of the else the hinted input falls on. A hint is a legacy shape by construction, so it increments legacy and leaves witness alone: it gets billed for the empty witness stack if some other input brings a witness, and it cannot conjure a marker on its own. The stretch rides through as scaled weight, and the witness decision is made about the transaction the parts will form rather than about each part in isolation.

Mix one P2WPKH input with one legacy input paying two legacy outputs and the difference shows up in a single base unit. Price the parts apart (overhead 10, plus each input rounded to vbytes on its own, plus two outputs) and you get 294. Price them together: 40 + 272 + 592 + 272 weight, plus the marker and flag counted once and one empty witness stack, is 1_179, and -(-1179 // 4) is 295. Pricing the parts apart understates this transaction, because nothing in it charged for the marker. Three P2TR inputs paying a P2TR target with P2TR change go the other way: 270 apart, 269 together, because rounding each 230-weight input up to 58 vbytes charges three quarters of a byte each time for size that does not exist.

One rounding, at the end

So the selector builds shapes and hands the whole set over once.

def _tx_vsize(
    inputs: Sequence[Utxo], request: SelectionRequest, *, with_change: bool
) -> int:
    """Virtual size of the transaction these inputs and the request form.

    The weight of every part is added up first and divided once, so a witness
    input is not charged for the three quarters of a byte that rounding it on
    its own would add.
    """
    outputs = [
        (target.script_type, target.script_size) for target in request.targets
    ]
    if with_change:
        outputs.append(
            (request.change_script_type, request.change_script_size)
        )
    return weight_to_vsize(
        estimate_shaped_weight(
            [(utxo.script_type, utxo.script_size) for utxo in inputs], outputs
        )
    )

weight_to_vsize is a ceiling division by four, -(-weight // WITNESS_SCALE_FACTOR), applied exactly once per transaction, which matches what consensus and relay policy do.

Where per-input rounding survives, and why that is safe

One place still rounds a single input alone:

def _input_vsize(utxo: Utxo) -> int:
    """Marginal cost of one more input, rounded on its own."""
    return weight_to_vsize(input_weight(utxo.script_type, utxo.script_size))


def _effective_value(utxo: Utxo, fee_rate: int) -> int:
    """What an output is worth once the fee for spending it is taken off."""
    return utxo.value - estimate_fee(_input_vsize(utxo), fee_rate)

That is the number deciding whether a candidate is worth spending at all and how candidates rank against each other. It is allowed to be slightly pessimistic, and the ScriptType.input_vsize docstring says so in as many words: rounding each input on its own overstates a transaction slightly, so it is "a safe marginal cost to compare candidates with, not a term to sum into a total".

That sentence is the rule I care about most in this module. The two roles look like the same quantity and are not. One is a ranking key, where erring high drops a marginal output instead of including an unaffordable one. The other is money a miner will take, where erring high by a byte per input adds up across a consolidating spend. Keeping them in separately named functions with that distinction written down is what stops someone (me, in six months) from summing the cheap one because it was already there...

What I would do differently

The asymmetry in the public surface bothers me. estimate_weight is exported and accepts only ScriptType members, validated by _as_script_types. The function that actually prices a real mixed transaction, estimate_shaped_weight, is not in __all__. A caller who wants to measure a partially built transaction the way the selector does has to reach past the package's own front door. The typed-only helper is the convenient one and should stay, but the shaped one is the one that prices honestly, and it should probably be exported alongside it.

I would also reconsider pinning the hint to a legacy shape forever. Today it is the right default: the hint exists for callers who have a byte count and nothing else, and reading those bytes as non-witness data overcharges rather than undercharges, which is the correct direction for a fee estimate to fail. But a caller who knows the input is segwit and still only has a size gets billed out of the witness discount they are entitled to. If that case turns out to be common, the fix is not a second estimator: it is a hint that carries its own witness flag, so the stretch still rides through the one weight sum.

It comes down to one estimator that makes the witness decision once and divides by four once. Everything else in the library sits downstream of that, including the parts that are deliberately allowed to be wrong.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

Why a Markdown scanner should blank code spans, not delete them

polycratia — Wed, 11 Nov 2026 13:21:36 +0000

A link into a repository is worth reading only if it names a commit that cannot move underneath it. Checking that is the easy half: read the file, find the links, print a file:line for every ref that is not forty lowercase hex characters. The hard half starts when the same tool has to turn a branch link into a pinned one. Now it needs to write forty characters into a byte range of the file it just read, and every shortcut the reader took with positions is a bug waiting in the writer.

That pressure is what shaped srcproof, a small Go tool I wrote to keep repository links in prose honest. Two subcommands sit on one extractor. check reports links whose ref is not a full commit. pin resolves the branch and tag links of an article against a local clone and, with --write, rewrites them in place and records what it resolved in srcproof.lock. Both commands read the same Markdown through the same code. Only one of them edits, so the shared extractor has to serve the stricter consumer.

Masking preserves length; deleting does not

A link inside a code span is not a link. `https://github.com/owner/name/blob/main/go.mod` is text you are being shown, not a reference you are expected to follow, and a tool that refuses branch links has to keep the right to print the form it refuses. So code spans must not be scanned.

The obvious way to not scan them is to cut them out of the line before looking for URLs. For a tool that only reports, that works perfectly. It also quietly destroys every column after the span: remove thirty bytes from the middle of a line and everything to the right of it now sits thirty bytes earlier than it does in the file. A rewriter that trusts those offsets writes into the wrong place.

So the span is overwritten instead of removed, with exactly as many spaces as it occupied:

// stripCodeSpans blanks out code spans without moving anything after them, so
// the column of a link is the column it has in the line as written.
func stripCodeSpans(line string) string {
    var out strings.Builder
    i := 0
    for i < len(line) {
        if line[i] != '`' {
            out.WriteByte(line[i])
            i++
            continue
        }
        open := run(line, i)
        close := -1
        for j := i + open; j < len(line); {
            if line[j] != '`' {
                j++
                continue
            }
            if length := run(line, j); length == open {
                close = j
                break
            } else {
                j += length
            }
        }
        if close < 0 {
            out.WriteString(line[i : i+open])
            i += open
            continue
        }
        out.WriteString(strings.Repeat(" ", close+open-i))
        i = close + open
    }
    return out.String()
}

func run(line string, start int) int {
    length := 0
    for start+length < len(line) && line[start+length] == line[start] {
        length++
    }
    return length
}

The opening run is measured, and the span closes only on a later run of exactly the same length. Three backticks do not close a single-backtick span, which is why the inner loop advances by length rather than by one when it finds a run of the wrong size. The replacement is close+open-i spaces, which is precisely the width of the span from its first backtick to its last. The output string has the same length as the input, byte for byte, and every offset to the right of the span is the offset it has in the file.

The unclosed case is the one I like most. A run of backticks with no matching partner is written back unchanged, because that is both what CommonMark says it is (literal backticks, not a span) and the only thing that keeps the length intact. The two requirements happen to agree, and when correctness and layout agree, you take it.

Fenced blocks are skipped whole, and that costs nothing

Fences are a different problem with an easier answer. A fenced block is a run of entire lines, so the extractor does not need to preserve anything inside it: it produces no links from those lines at all, and a line that yields no link has no column to protect.

func fenceMarker(line string) (byte, int, bool) {
    i := 0
    for i < 3 && i < len(line) && line[i] == ' ' {
        i++
    }
    if i >= len(line) {
        return 0, 0, false
    }
    char := line[i]
    if char != '`' && char != '~' {
        return 0, 0, false
    }
    length := run(line, i)
    if length < 3 {
        return 0, 0, false
    }
    return char, length, true
}

func closesFence(line string, char byte, length int) bool {
    start := strings.IndexByte(line, char)
    return strings.TrimSpace(line[start+length:]) == ""
}

A fence opens on a line that starts, after at most three spaces, with three or more backticks or tildes, and closes only on a run of the same character at least as long with nothing but whitespace after it. Everything in between is skipped. Notice that this fenceMarker returns the character and the run length and throws the indent away. It does not need the indent, because it is discarding the lines. The quoting side of the tool has its own fenceMarker that also returns the indent, because it keeps the body lines and has to strip that indentation off them. Same syntax, two readers, two signatures. Trying to share one was worse than writing both.

The practical payoff is that srcproof's own README passes srcproof's own check while containing a table of the refs it refuses (main, v1.4.0, refs/heads/main) and a worked example of a link it would reject. Documentation that cannot show the bad form is documentation that cannot teach.

From a column to a splice point

With the masked line in hand, finding URLs is unremarkable except for one detail:

func findURLs(line string) []candidate {
    locations := urlPattern.FindAllStringIndex(line, -1)
    candidates := make([]candidate, 0, len(locations))
    for _, at := range locations {
        url := strings.TrimRight(line[at[0]:at[1]], ".,;:!?")
        if url == "" {
            continue
        }
        candidates = append(candidates, candidate{url: url, col: at[0]})
    }
    return candidates
}

A URL at the end of a sentence swallows the full stop, so the match is trimmed, but trimmed from the right only, and the recorded column is at[0], the start of the match. Sentence punctuation can shorten a URL. It cannot move one. That is the kind of invariant worth being deliberate about, because the alternative spelling (trim both ends, record the trimmed start) looks identical in every test that does not involve a leading character.

That column lands in the Col field of the link. A second offset lands next to it: when the URL is parsed, the position of the ref within the URL is computed by walking the segments that precede it.

func segmentOffset(start int, segments []string, index int) int {
    for _, segment := range segments[:index] {
        start += len(segment) + 1
    }
    return start
}

For a github.com link the ref is the fourth segment, after owner, name and the blob, tree or commit keyword. For raw.githubusercontent.com it is the third segment, or three segments joined when that segment is refs, so that refs/heads/main is read as one ref rather than as refs. The parse drops the fragment and the query to decide these things, but the URL is kept exactly as it was written, and the offset is into that original string. Column plus ref offset is the byte range of the ref in the file.

The rewrite distrusts the position it was given

An offset computed earlier is a claim about a string that may since have changed, so both ends check it before they use it.

Pinning a link refuses outright unless the ref offset is positive and the URL really does hold the ref at that offset. Only then does it splice the commit in and leave the rest of the URL, fragment included, as written. And when the rewrites are applied to the file, the same suspicion shows up again:

// apply rewrites a line from its right edge inwards, so the column of a link
// is still the column it was read at after an earlier link on the same line
// has grown into a 40-character commit.
func apply(source string, rewrites []Rewrite) string {
    if len(rewrites) == 0 {
        return source
    }
    grouped := make(map[int][]Rewrite)
    for _, rewrite := range rewrites {
        grouped[rewrite.Line] = append(grouped[rewrite.Line], rewrite)
    }
    lines := strings.Split(source, "\n")
    for number, group := range grouped {
        line := lines[number-1]
        for i := len(group) - 1; i >= 0; i-- {
            rewrite := group[i]
            end := rewrite.Col + len(rewrite.From)
            if end > len(line) || line[rewrite.Col:end] != rewrite.From {
                continue
            }
            line = line[:rewrite.Col] + rewrite.To + line[end:]
        }
        lines[number-1] = line
    }
    return strings.Join(lines, "\n")
}

Two things are happening here. The loop runs backwards through the rewrites of a line, so replacing a nine-character branch name with a forty-character commit cannot invalidate the column of a link further right on the same line: the right edge is edited first, and the left-hand columns are still the columns they were read at. The bytes at the recorded column are also compared against what the rewrite expected to find there. If they do not match, or the range runs past the end of the line, the rewrite is skipped and the line stays as it is. A stale position produces a link that was not pinned, reported as unresolved. It does not produce forty hex characters spliced into the middle of a sentence.

It is also why no column ever reaches srcproof.lock. A link directive records the article, the repository, the commit and the ref it was resolved from, and nothing about where in the file that happened. A byte offset is valid against one exact revision of one file, so persisting it would mean persisting something that the next edit silently falsifies.

What I would do differently

The length-preservation contract lives in a comment above stripCodeSpans, which is probably the weakest place to put an invariant that three other packages depend on. It belongs in a test: for every line, the masked line and the original have the same length. That property is cheap to check, it fails loudly the first time someone replaces a span with the empty string, and it says the thing the comment only hints at.

The scanner is also line-by-line, and CommonMark lets a code span cross a newline. A URL inside such a span is read as a link today, which is wrong. The fix is not a different strategy but the same one at document scope: mask the span across the lines it covers, keeping the newlines, and offsets still hold. Indented code blocks have the same shape of gap and the same shape of answer.

The general lesson I keep relearning is that a position is not a diagnostic. The moment a tool grows from reporting to editing, every offset it computed becomes an address somebody will write to, and the transformations it performed on the way to finding that offset become part of its correctness. Masking rather than deleting is a one-line decision that keeps the two halves of the tool talking about the same file...

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

Why a ring buffer's capacity must divide 2^64

polycratia — Tue, 10 Nov 2026 13:28:19 +0000

The simplest single-producer single-consumer ring keeps two counters that are never reset and masks them on every access. That shape turns the full and empty tests into one subtraction and takes the modulo off the hot path, but it also pulls the counters' own 64-bit overflow into the correctness of the queue: when an index wraps, the slot it addresses after the wrap has to continue the sequence from before it. That holds only when the capacity divides 2^64. So a power-of-two capacity is a correctness condition rather than a performance preference, and the difference shows up exactly once in the life of a process.

I keep a small header-only version of this queue with the reasoning written down next to the code, spscring. The README opens by listing the three ways these queues are usually wrong, and the third one is this: a counter that wraps at a non-multiple of the capacity, so the queue silently reorders once, years in. This post is that item in full.

The shape that makes the overflow matter

The indices only grow. No branch sends them back to zero, no wrapped flag, no modulo anywhere on the hot path.

// Indices count forever and are masked at use. Subtraction is correct
// across the unsigned wrap, and because the capacity is a power of two the
// wrap does not shift the alignment.
alignas(cache_line) std::atomic<std::size_t> write_{0};
alignas(cache_line) std::atomic<std::size_t> read_{0};

A push addresses slots_[write & (Capacity - 1)] and a pop addresses slots_[read & (Capacity - 1)]. Fullness is write - read == Capacity. A pop proceeds while read != write. The teardown path walks the same way:

void destroy_queued() noexcept {
    const std::size_t read = read_.load(std::memory_order_relaxed);
    const std::size_t write = write_.load(std::memory_order_relaxed);
    for (std::size_t index = read; index != write; ++index) element(index)->~T();
}

Every one of those expressions is unsigned arithmetic modulo 2^64, and every one of them stays correct when the counter rolls over: the subtraction because the true difference never exceeds the capacity, so the mod-2^64 result is the true difference, and the loop because it compares for inequality rather than ordering. size() is the one place where a third thread can be fooled, and it is handled by load order and a clamp instead of by pretending the wrap does not happen:

[[nodiscard]] std::size_t size() const noexcept {
    const std::size_t read = read_.load(std::memory_order_acquire);
    const std::size_t write = write_.load(std::memory_order_acquire);
    return std::min(write - read, ring_capacity());
}

So the arithmetic survives. What does not survive on its own is the mapping from index to slot.

What a capacity of 1000 actually does

Say the counters are the same but the capacity is a round number and the slot comes out of a modulo. This is the shape I am not shipping:

// The common wrong version: a capacity that does not divide 2^64.
static constexpr std::size_t capacity = 1000;
T* slot(std::size_t index) { return &slots_[index % capacity]; }

The hardware has already folded the counter before your % runs, so the slot for the n-th operation is (n mod 2^64) mod 1000, not n mod 1000. Those two functions agree until the fold does something, and then they diverge, because 1000 does not divide 2^64. The leftover is concrete: 2^64 is 18446744073709551616, so 2^64 mod 1000 is 616.

Follow one slot through that. Before the rollover, slot 0 was last written by index 2^64 - 616, since that index is the largest multiple of 1000 below 2^64. After the rollover, index 0 also wants slot 0. The two indices are 616 apart, and the queue's own rule allows up to 1000 indices to be live at once. If the queue is holding 616 or more elements at the instant the write index crosses zero, the producer builds a new element on top of one the consumer has not taken yet. The consumer then pops that position and gets the wrong element, pops forward, and finds the newer element still sitting there. One lap later the aliasing is gone and the queue is correct again, for the next 2^64 operations.

Two things make this the worst failure class I know of in this kind of code.

It is depth-dependent. Both threads compute the slot the same way from the same index, so they never disagree with each other; what breaks is the invariant that the live window of indices occupies distinct slots. Whether anything is lost depends on how deep the queue happens to be at one particular instant. A queue that is usually near-empty crosses the rollover untouched and you never learn that it was a coin flip.

It is also out of reach of brute force. The counter is 64 bits. At one operation per nanosecond, 2^64 operations take about 584 years; at a hundred million operations per second, about 5.8 years of continuous traffic. The demo in that repository pushes a million messages through a ring<message, 4096> and the threaded test moves two hundred thousand std::uint32_t through a heap_ring of 1024 slots. Neither of those gets within an astronomical distance of the condition. Throughput tests cannot find this, a fuzzer cannot find this, and a code review that reads % capacity as an innocuous spelling of & (capacity - 1) will not find it either. I have spent most of the last several years on payment and crypto systems, where a fault that fires once and then clears itself is the expensive kind: somebody has to explain a single mismatched record, and the code that produced it looks fine when they go read it.

The fix is not to detect the rollover. It is to pick a capacity that divides the counter's modulus. For a 64-bit counter that means a power of two, and once the capacity is a power of two, the modulo is an AND and the hot path gets faster for free. The correctness decision and the performance decision are the same decision, which is why I think the performance framing is dangerous: it makes the requirement sound negotiable.

A condition the compiler checks

If it is a correctness condition, it should not be a sentence in a README. For the fixed-capacity ring it is a static_assert:

static_assert(detail::is_power_of_two(Capacity), "capacity must be a power of two");

with the predicate written the plain way:

constexpr bool is_power_of_two(std::size_t n) noexcept {
    return n != 0 && (n & (n - 1)) == 0;
}

ring<int, 1000> is a compile error instead of a queue that works until the counter wraps.

A capacity that is only known at start-up cannot be checked that way, so the heap variant moves the check into a factory that is allowed to fail. heap_ring<T>::create(n) returns a null pointer rather than a queue it cannot honour, and the test pins down what it refuses:

void create_refuses_a_capacity_it_cannot_honour() {
    CHECK(spscring::heap_ring<int>::create(0) == nullptr);
    CHECK(spscring::heap_ring<int>::create(3) == nullptr);
    CHECK(spscring::heap_ring<int>::create(1000) == nullptr);  // not quietly 1024
    // 2^63 slots of two bytes each is 2^64 bytes, which is not a size_t.
    CHECK(spscring::heap_ring<std::uint16_t>::create(std::size_t{1} << 63) == nullptr);
}

The comment on the third line is the part I care about. Rounding 1000 up to 1024 is the friendly-looking choice and it is wrong twice over: it silently changes the caller's memory footprint, and it hides the fact that the number in their configuration file was never a capacity this queue could have. The honest answers are the capacity that was asked for, or none, and the caller gets to decide what to do about it.

Exercising a wrap you cannot reach

An invariant that no test touches is a comment. To test the rollover you have to put the indices near it, which means reaching inside the queue, and the usual ways of doing that all leave something behind in the shipped library. Public indices invite a caller to write them. An #ifdef means the thing you tested was not the thing you shipped. A named friend class drags the test's identity into the header.

What the header does instead is declare a template and define it nowhere:

namespace detail {
template <typename Ring> struct test_access;  // declared here, defined nowhere
}

ring names detail::test_access<ring> as a friend. A translation unit that supplies the definition gets at the indices; a build that does not define it has no way to reach them, because there is nothing to instantiate. The whole hook costs the library one declaration and one friend line, with no macro and no build variant. The definition lives in the test file:

namespace spscring::detail {

template <typename Ring>
struct test_access {
    // Called before any thread touches the ring, so relaxed stores are enough.
    static void start_indices_at(Ring& queue, std::size_t index) {
        queue.write_.store(index, std::memory_order_relaxed);
        queue.read_.store(index, std::memory_order_relaxed);
    }
};

}  // namespace spscring::detail

With that, the wrap becomes a short test. It starts both indices six short of the end, runs 32 elements through bursts of four, three, two and one, and checks every value out, the refusal of a fifth push into a full queue on both sides of the rollover, and the count in the state where the write index has passed zero and the read index has not:

using small_ring = spscring::ring<int, 4>;
constexpr std::size_t start = SIZE_MAX - 5;

small_ring q;
spscring::detail::test_access<small_ring>::start_indices_at(q, start);
CHECK(q.empty());

The burst sizes are there so the queue sits at four different depths when the counter crosses zero, and that depth is the variable that decides whether a broken capacity loses anything. The test ends by asserting that the run actually reached the rollover with the queue non-empty where it mattered, because a wrap test that never reaches the wrap passes cheerfully and proves nothing. That is a failure mode of the test rather than of the queue, and no less real for it.

What I would do differently

I would write the wrap test before the threaded one. The two-thread test that pushes two hundred thousand elements through a 1024-slot ring is the test that feels like proof, it is the one that catches a missing release store on hardware that does not forgive it, and it exercises the index arithmetic only in the region where the arithmetic is boring. Writing it first gave me a green suite that said nothing about the one bug in this class that survives a green suite.

I would also be slower to add the run-time capacity. The fixed-capacity ring has this problem solved by the compiler, with a message a caller can read. Every refusal create has to make, zero, a non-power-of-two, a byte size that overflows size_t, exists because a number arrived too late for the static_assert. That is a reasonable price when the capacity genuinely comes from configuration, and pure cost when it does not.

The general lesson is narrower than it looks. It is not that unsigned overflow is subtle; the subtraction here is correct across the wrap and the comments say why. It is that when you remove a reset, the counter's modulus becomes a parameter of your data structure, and you now owe an argument that your capacity and that modulus agree. For a 64-bit index there is exactly one family of capacities that satisfies it, and the compiler is perfectly capable of insisting on it.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

The 401 loop: when discovery and audience binding disagree

polycratia — Mon, 09 Nov 2026 13:28:21 +0000

An MCP server that checks audience properly refuses every token that was not minted for it. That makes it safe and unreachable in a single move: a client has to learn what "for it" means before it can ask for anything. All it has to go on is one discovery document and the path that document is served at. Get a field in there wrong and nothing fails loudly. The client fetches a token for whatever the document named, the server refuses it as the wrong audience, the client fetches another one that fails the same way, and the loop lands in the client's logs looking like a client bug.

I keep the resource-server half of MCP authorization in a small Go package, mcpaudience. What it leads with is audience binding, because that is the rule that stops an MCP server from becoming somebody else's deputy. But the strictness is the cheap half. Discovery is the half that decides whether a strict server is usable at all, and it is where the bugs live, because every one of its failures is quiet.

The loop, and what each field in it carries

Four hops, and a client that follows the specification makes all of them without asking a human anything:

A request arrives with no token. The server answers 401 and points at a document.
The client fetches that URL and gets back the protected resource metadata document of RFC 9728.
The document names a resource and a list of authorization_servers. The client asks one of those issuers for a token whose audience is that exact resource.
The client retries with the token, and the server checks the audience against its own canonical URI.

Hop one looks like this:

--- no token ---
status: 401
WWW-Authenticate: Bearer resource_metadata="https://mcp.example.com/.well-known/oauth-protected-resource"

Now look at what a wrong value does at each hop. A resource that is not the string the verifier binds to produces tokens minted for something this server does not answer to, a permanent invalid_token where the token is perfectly valid and the server is perfectly correct to refuse it. An issuer the server does not trust produces tokens signed by keys it does not hold. A document served at the wrong path produces a 404 at hop two, which leaves the client holding a challenge it cannot follow. And a bearer_methods_supported entry other than header invites the client to put the token in a query string, which is how a bearer token ends up in every access log between the client and here.

None of that returns an error saying "your metadata is wrong". That is the whole problem, and the reason the document deserves more care than a hand-written JSON literal.

The wiring is small:

guard := &mcpaudience.Guard{
    Metadata: mcpaudience.NewMetadata("https://mcp.example.com", "https://auth.example.com").
        WithScopes("files:read", "files:delete"),
    Verifier: &mcpaudience.JWTVerifier{
        Resource: "https://mcp.example.com",
        Issuers:  []string{"https://auth.example.com"},
        Keys:     publicKeysByKeyID,
    },
}

mux := http.NewServeMux()
guard.Metadata.Mount(mux) // /.well-known/oauth-protected-resource
mux.Handle("/", guard.Handler(tools))

NewMetadata sets BearerMethods to header and nothing else, because that default is not a preference: it is the one method that keeps the token out of logs and browser history.

A document the server would reject is never published

Validate is the gate, and it refuses a document a client could not act on:

// Validate refuses a document a client could not act on.
func (m Metadata) Validate() error {
    if err := canonicalURI(m.Resource); err != nil {
        return fmt.Errorf("resource: %w", err)
    }
    if len(m.AuthorizationServers) == 0 {
        return fmt.Errorf("authorization_servers: at least one issuer is required")
    }
    for _, issuer := range m.AuthorizationServers {
        if err := canonicalURI(issuer); err != nil {
            return fmt.Errorf("authorization_servers: %w", err)
        }
    }
    for _, method := range m.BearerMethods {
        // The other two methods RFC 6750 allows put the token in a query
        // string or a form body. Both leak it into logs.
        if method != "header" {
            return fmt.Errorf("bearer_methods_supported: %q puts the token where it will be logged", method)
        }
    }
    return nil
}

The same canonical-URI check runs over the resource and over every issuer, and it asks four things of each: non-empty, absolute, https unless the host is localhost or 127.0.0.1, and carrying neither a fragment nor a query string. Each of those maps to a concrete way the loop breaks. A relative URI cannot be an audience value, because on the client's side there is no base to resolve it against. A fragment or a query string makes the identifier unstable under comparison: the audience check works on canonical form, and the more syntax an identifier carries, the more ways two spellings of the same server stop matching. Plain http puts the token on the wire in clear text, and the localhost carve-out exists only to permit that while you are developing. Zero issuers points the client at exactly one place for a token: nowhere...

Validation happens once, when the handler is built. If it fails, every request to the well-known path answers 500 with the validation message. That is deliberate. A malformed document is better withheld than served, because a served-but-wrong document produces the silent retry loop, while a 500 with the reason in the body produces a stack trace in the one place that can fix it. On success the body is marshaled once and served with cache-control: public, max-age=3600; methods other than GET and HEAD get 405 with an allow header.

The well-known segment goes in front of the path, not after it

This is the detail that catches people, and I think it costs an afternoon when it does:

// WellKnownPath is the path the document has to be served at, per RFC 9728
// §3.1: the well-known segment goes between the origin and the resource's path,
// not after it. A server at https://mcp.example.com/mcp publishes at
// /.well-known/oauth-protected-resource/mcp, and appending instead would leave
// clients fetching a URL nothing serves.
func (m Metadata) WellKnownPath() string {
    parsed, err := url.Parse(m.Resource)
    if err != nil {
        return MetadataPath
    }
    return MetadataPath + strings.TrimSuffix(parsed.Path, "/")
}

An MCP server mounted at /mcp (which is most of them, since the transport wants one endpoint) does not publish at /mcp/.well-known/oauth-protected-resource. It publishes at /.well-known/oauth-protected-resource/mcp. Insertion, not suffixing.

The reason this is a function rather than a constant is that two places need the same answer and they must not drift: Mount registers the handler at WellKnownPath, and URL builds the absolute form that goes into the resource_metadata parameter of the challenge. Derive both from the resource and the 401 cannot promise a location the server does not serve. Write either one by hand and eventually it will.

Checked against the verifier, on every request

There are now two independent copies of the same string in the configuration: the metadata's Resource, which clients act on, and the verifier's Resource, which tokens are checked against. Nothing structural keeps them equal. So the guard checks, before it reads any token at all:

if err := g.checkBinding(); err != nil {
    http.Error(w, "mcpaudience: "+err.Error(), http.StatusInternalServerError)
    return
}
token, err := BearerToken(r)

And the check itself:

func (g *Guard) checkBinding() error {
    binder, ok := g.Verifier.(ResourceBinder)
    if !ok {
        return nil
    }
    bound := binder.BoundResource()
    if strings.TrimSpace(bound) == "" {
        return ErrNoResource
    }
    if advertised := g.Metadata.Resource; advertised != "" && !sameResource(advertised, bound) {
        return fmt.Errorf("the metadata advertises %q but tokens are bound to %q", advertised, bound)
    }
    return nil
}

Two things here matter more than the code does.

The first is the status. Both failures answer 500, not a challenge. A 401 would hand the client a document to go fetch, and a client that followed it would come back with a token for the advertised resource that this server then rejects: the loop again, now manufactured by the server's own error handling. ErrNoResource says it plainly: no resource identifier configured: audience binding cannot be skipped. An empty resource is not a permissive mode. It is a server that cannot do the one job it exists to do, and blaming the caller for it wastes everyone's time.

The second is the comparison. sameResource works on canonical form: scheme and host case-folded, a default port dropped, a trailing slash dropped, and everything else exact. So HTTPS://MCP.Example.com:443 and https://mcp.example.com are the same server, which they are by every rule the URI syntax has. A prefix match or a host-suffix match, the shortcuts that suggest themselves first, would accept a token minted for a neighbour. The mismatch message names both values instead of saying "mismatch", because the whole difficulty of this class of bug is that the two strings look identical until you put them side by side.

What I would change

Two things, and they are the same thing seen twice.

checkBinding opens with a type assertion, and a Verifier that does not implement ResourceBinder returns nil: no check performed, silently. For the built-in JWT verifier that is fine. For a custom verifier it means the one guard against advertised-versus-bound drift quietly does not exist, which is the exact failure mode the function was written to prevent. Folding BoundResource into Verifier itself would close that, at the cost of making every verifier declare a resource.

And both the metadata validation and the binding check surface on the first request rather than at boot. Guard is a struct with exported fields, so a composite literal is the construction, and there is no constructor to hang a check on. That is pleasant to write, and it means a misconfigured server deploys green and starts answering 500 only once a client shows up. If I were doing it again I would give Mount an error return and check the binding there, so a bad document fails the deploy instead of the first user.

The lesson I would keep from this is narrow and reusable. When a protocol has the server publish the terms a client must satisfy, those terms are code, and they belong under the same scrutiny as the check they feed. A refusal is easy to get right. A refusal that nobody can satisfy is also a refusal, and it will be read as the client's fault for as long as it takes someone to compare two strings by hand.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

One disarm override per pattern, not per key

polycratia — Sun, 08 Nov 2026 13:21:51 +0000

gitdisarm reads the git configuration that arrived with a repository, and for every key that would make git run a program it prints the invocation that neutralizes it. The obvious store for those invocations is a map from key name to override string, and that map fails on the very first key worth reporting: the keys worth reporting are patterns, a pager for one command, a filter driver, a credential helper for one URL. So there is no finished override to keep around. Only a template, plus the entry that fills it.

The second half of the job is quieter and it bit me later: once the templates are filled, the results are not interchangeable. A -c pair goes to any git command. An option like --no-ext-diff belongs to git diff alone. Put both on one command line and git rejects it, which is about the worst thing that can happen to a tool whose whole output is a command you are supposed to paste. The code is in gitdisarm; what follows is the reasoning behind the two halves.

The key is a pattern, so the override is a template

Each row in the catalog is a Rule. It carries the family it belongs to, the earliest git version that reads the key, and a Disarm string written against the row's Pattern. Inside that string, <name> stands for whichever part of the key varies. Filling it takes three lines:

// DisarmFor returns the row's override with <name> replaced by the part this
// entry varies in.
func (r *Rule) DisarmFor(e gitconfig.Entry) string {
    return strings.ReplaceAll(r.Disarm, "<name>", r.nameOf(e))
}

func (r *Rule) nameOf(e gitconfig.Entry) string {
    if r.key == "" {
        return e.Key
    }
    return e.Subsection
}

The condition is the interesting line. A row that does not pin a key matches on the section alone, so what varies between two matching entries is the key itself: pager.<name> covers pager.log and pager.diff, and <name> is the entry's Key. A row that does pin a key (clean, program, helper) is matching a family of drivers, so the subsection is what varies, and <name> is the entry's Subsection.

row pattern	row pins a key	`<name>` comes from
`core.pager`	yes (`pager`)	nothing varies
`pager.<name>`	no	entry's `Key`
`filter.<name>.clean`	yes (`clean`)	entry's `Subsection`
`gpg.<name>.program`	yes (`program`)	entry's `Subsection`

One field decides the direction, and the row already needed that field in order to match at all. That is the part I would defend: the fill rule comes out of the match rule instead of being configured beside it, so a row cannot match one way and fill another.

Matching is also where the value gets a vote, since some of these keys are dangerous only depending on what they hold:

    switch r.value {
    case boolValue:
        _, ok := parseBool(e.Value)
        return ok
    case bangValue:
        return strings.HasPrefix(strings.TrimLeft(e.Value, " \t"), "!")
    }

A core.fsmonitor that git reads as a boolean selects either the monitor built into git or none; anything else is a hook command. An alias.<name> is a program only when its expansion begins with !. Both rows answer inert, and an inert row fills no template: Classify sets Finding.Disarm only when the row's Class is Executes.

One template, sometimes two pairs

Clearing the key is frequently not enough, and the template is where that gets said, because the extra pair belongs to the row and not to the entry. Three cases in the catalog carry two overrides inside one string. The table-driven tests show the filled results directly, which is also how I keep the templates honest:

{"credential.<name>.helper", entry("credential", "https://example.com", "helper", "sh -c id"), Executes, "-c credential.https://example.com.helper= -c credential.helper="},
{"gpg.program", entry("gpg", "", "program", "sh -c id"), Executes, "-c gpg.program=false -c log.showSignature=false"},
{"gpg.<name>.program", entry("gpg", "ssh", "program", "sh -c id"), Executes, "-c gpg.ssh.program=false -c log.showSignature=false"},

A scoped credential helper does not displace the unscoped one, so emptying credential.https://example.com.helper leaves credential.helper standing, and the template clears both. The gpg rows do not empty the key at all: they name false as the program and add -c log.showSignature=false, so git is neither pointed at a program of the repository's choosing nor asked for the signature check to begin with. The filter rows carry a second pair of their own, -c filter.<name>.required=false, because a cleared filter that is still marked required changes which way the command fails.

All of it sits in one string per row. I considered modelling it as a list of pairs and decided against it, for a reason I still like: a row's override is text a human reads, pastes and checks against git's own documentation. Keep it as the literal text that gets printed, and the test above compares exactly what you will see, with no assembly step between the catalog and the screen to get wrong.

Keys the catalog has no row for still get a filled override when they sit inside a family whose drivers are nothing but the programs their keys name:

    name := Name(e)
    if family, ok := familyOf(e); ok {
        return Finding{
            Entry:  e,
            Class:  Executes,
            Family: family,
            Disarm: "-c " + name + "=",
            Reason: "no row covers " + name + ", and a " + string(family) + " driver is defined by the programs its keys name",
        }
    }

That is the template-free path: no pattern to fill, so the override gets built from the entry's own full name. A new filter.<driver>.* key invented by some future git gets an override anyway, instead of quietly being called safe.

The split is the printable half

diff.external and diff.<name>.command are neutralized by --no-ext-diff, and diff.<name>.textconv by --no-textconv. Those are options of git diff, not of git. They cannot go where the -c pairs go. So the report sorts each filled override as it arrives, on the one distinction that matters for placement:

func (r *report) addDisarm(disarm string) {
    dst := &r.config
    if !strings.HasPrefix(disarm, "-c ") {
        dst = &r.flags
    }
    for _, have := range *dst {
        if have == disarm {
            return
        }
    }
    *dst = append(*dst, disarm)
}

Two decisions worth naming here. The deduplication is a linear scan over a slice rather than a map, because the order is part of the output contract: each override appears once, in the order it was first found, which is the order the keys were declared in the file. A set would make the invocation unstable between runs on the same input, and an unstable invocation is one you cannot diff or paste into a review. Second, both lists are append-only and never sorted, so the printed command reads in the same sequence as the findings above it.

Then the summary prints each list as the thing you do with it:

    if len(r.config) > 0 {
        fmt.Fprintf(out, "\nRun git with every override at once:\n\n  git %s <command>\n", strings.Join(r.config, " "))
    }
    if len(r.flags) > 0 {
        fmt.Fprintf(out, "\nThese belong to the command that honours them rather than to git:\n\n  %s\n", strings.Join(r.flags, " "))
    }

One invocation to run, one short list to place by hand. printFinding still shows each entry's own filled Disarm line next to the key that caused it, because the two have different readers: the per-finding line answers "what does this one key do and how do I stop it", and the aggregate answers "what do I type before I touch this tree".

Neither one is a claim of safety, which is why the summary ends with catalog.NotCovered: include.path, core.gitProxy, a template directory, the executables already sitting in .git/hooks. A short list of findings must not read as a clean repository, and an exit status of 0 says only that no covered key named a program.

What I would change

<name> is a single placeholder and strings.ReplaceAll fills every occurrence with the same string. For every row in the catalog today that is correct, because no row varies in two independent places. The first row that does will be filled wrongly, and quietly. A check that a row's Pattern and its Disarm agree on how many distinct names they mention would turn that into a test failure instead of a bad override.

The other soft spot is the split itself. addDisarm decides placement from the "-c " prefix of a string the catalog wrote. It works because one author writes both sides, and it is the kind of agreement that holds right up until someone adds a row whose override begins with something else and lands in the wrong list with no test noticing. The honest representation is probably a field on Rule saying which list the override belongs to, with the prefix becoming a consequence rather than the source of truth.

Those two are the same mistake in two sizes: structure derived from text after the fact, when the structure was known at the moment the row was written. The template idea itself holds up, since a pattern cannot produce a per-key answer, it has to produce a filled one. Where I cut corners was in letting the filled string stand in for facts the row already had...

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

A resumed acceptance window is measured by the rejection that stopped it

polycratia — Sat, 07 Nov 2026 13:22:57 +0000

Most contracts hand a delivery ten business days to be accepted and then go quiet about what a rejection does to that count. Does the redelivery get a fresh ten days, or only what the first window had left? On a long engagement that term is worth real money, so code that replays a contract has to read it out of the contract rather than choose for it. Resuming is the harder half: a clock you resume needs a measurement of how much of it went already, and in an append-only journal that measurement can only live in the rejection that stopped it.

This comes out of engageledger, a Go library I maintain. It holds the money terms of a contracting engagement as a signed append-only journal, then folds that journal into the current terms and the acceptance state of every milestone. Acceptance there is a business-day clock: a delivery opens a window, an acceptance or a rejection closes it, and a window nobody closed runs out into deemed_accepted. The retest rule itself is small, one switch when the head is read and one branch when the next delivery arrives. Almost everything interesting about it sits in what happens when the number those two lean on is missing or wrong.

The term lives in the contract head

readConfig reads the optional acceptance object out of the contract.opened body:

func readConfig(body map[string]any) (int64, Retest, error) {
    v, ok := body["acceptance"]
    if !ok {
        return 0, RetestRestart, nil
    }
    obj, ok := v.(map[string]any)
    if !ok {
        return 0, "", errors.New("has an acceptance value that is not an object")
    }
    window := int64(0)
    if raw, ok := obj["window_days"]; ok {
        var err error
        if window, err = readCount(raw, "window_days"); err != nil {
            return 0, "", err
        }
    }
    retest := RetestRestart
    if raw, ok := obj["retest_window"]; ok {
        s, _ := raw.(string)
        switch Retest(s) {
        case RetestRestart, RetestResume:
            retest = Retest(s)
        default:
            return 0, "", fmt.Errorf("names retest_window %v, want %q or %q", raw, RetestRestart, RetestResume)
        }
    }
    return window, retest, nil
}

Two defaults are doing real work here. An absent acceptance object gives you a window of zero days and RetestRestart: a contract that never said how long acceptance runs does not get a window invented on its behalf. An unrecognised retest_window refuses the head itself, which in this fold is about as loud as failure gets. acceptance.Fold does not stop at a refused event, it records the refusal and keeps reading, and because the head never applied, f.opened stays false. Every later milestone event is then refused in turn for arriving before contract.opened. What you end up with is an empty contract next to a list of refusals, not a contract whose retest rule was quietly guessed. A typo in a negotiated term should not decay into a default.

The fold has no clock, so the rejection carries the count

The remaining days cannot simply be computed at redelivery time, because the fold owns no clock on purpose. Deadlines are fixed once, at delivery: acceptance.Deliver computes the close of the n-th business day on the contract's calendar and writes that UTC instant into the event as deadline_utc, next to the calendar_hash it was computed on. The fold takes the instant as recorded. Acceptance.Verify can recompute every recorded deadline and report a Drift for each one that now lands elsewhere, but it writes none of them back, so a zoneinfo update cannot shorten a window that is already running.

A fold that subtracted "days used so far" at replay time would break that property in the most annoying way available: the same journal would fold to different windows depending on when you replayed it. So the elapsed days have to be a recorded number, and the rejection is the event in a position to record them:

func (f *folder) reject(e journal.Event, i int) error {
    m := &f.milestones[i]
    if _, open := m.Open(); !open {
        return fmt.Errorf("rejects milestone %q, whose acceptance window is not open", m.ID)
    }
    defects, err := readDefects(e.Body, m.ID, m.Criteria)
    if err != nil {
        return err
    }
    used := int64(0)
    if v, ok := e.Body["elapsed_days"]; ok {
        if used, err = readCount(v, "elapsed_days"); err != nil {
            return err
        }
    }
    m.Status = StatusRejected
    w := &m.Windows[len(m.Windows)-1]
    w.Closed, w.ClosedBy, w.Used, w.Defects = true, e.Seq, used, defects
    return nil
}

readCount is the same reader the head uses for window_days: it takes a json.Number, an int64 or a plain int, so a body built in memory and the same body read back off disk behave identically, and it refuses a negative count. With the key absent it leaves used at zero.

The redelivery side is symmetric and short. A delivery is refused while a window is open; otherwise it appends a Window whose Days are the window the head configured. If the milestone's Status is StatusRejected, that window is marked as a retest, and under RetestResume its Days become the Remaining of the window that was just closed: Days minus Used, or zero once Used has reached Days.

In journal terms, a resume contract and a rejection four days in look like this:

{"acceptance": {"window_days": 10, "retest_window": "resume"}}
{"milestone": "m1", "defects": [{"criterion": "c1", "note": "export omits the tax column"}], "elapsed_days": 4}

The next milestone.delivered opens a retest window of six business days, and the deadline for those six days gets computed and recorded at that moment, on the calendar version the delivery names.

The two ways a recorded count goes wrong

The design keeps the fold pure, and the price is that elapsed_days is load-bearing. It fails in exactly two ways, and neither of them is an error.

Omission. A rejection with no elapsed_days leaves Used at zero, so Remaining equals Days, so the retest runs the full term. Under retest_window: "restart" you cannot see it, since the count is decoration there: the next window is the full term anyway. Under "resume" the contract quietly delivers restart semantics, and nothing in the journal is inconsistent, because a rejection written on the delivery day genuinely spends zero days. The default is the most generous value available to the contractor, and it reads exactly like a true same-day rejection.

Overstatement. A rejection that records more days than the window had leaves Remaining at zero, and the retest opens a window of zero days. That is not refused either, and it is not a void window: it has a deadline like any other, the close of the delivery day itself:

spentWindow, open := milestoneOf(t, spent, "m1").Open()
if !open {
    t.Fatal("the retest opened no acceptance window")
}
if spentWindow.Days != 0 {
    t.Errorf("a retest after a window spent over runs %d days, want 0", spentWindow.Days)
}
if stamp := spentWindow.Deadline.Format(time.RFC3339); stamp != "2026-03-06T16:00:00Z" {
    t.Errorf("a retest of no days runs to %s, want the close of the delivery day", stamp)
}

That is the behaviour I want, and it falls out of the calendar instead of a special case. Deadline for a window of zero business days is the close of the day the delivery landed on, and a delivery past close counts from the next business day. Deem then moves the milestone to deemed_accepted once that instant has passed, and its guard is the window the head configured, not the days of the particular window that happens to be open. A contract that configured no acceptance window at all has no clock to run out. A window that resumed down to zero days does have one, and it closes at close of business.

So the failure is quiet at both ends. One direction hands the contractor a free full window, the other hands the client an acceptance that is over by the end of the delivery day. Both read as ordinary journal lines.

What I would do differently

I would make elapsed_days mandatory whenever the head says resume, and refuse the rejection without it.

The fold already has that instinct elsewhere. A milestone.rejected event has to name at least one criterion of the milestone, each with a non-empty defect note; a rejection that names none fails no criterion, so it is refused and the milestone stays delivered with its clock still running. The reasoning is that a rejection which does not say what is wrong has not stopped anything. Under resume semantics, a rejection that does not say when it arrived has not measured anything either, and unlike the missing defect it does not leave the state untouched, it moves the window in the contractor's favour.

The second change is on the writing side. Deliver computes the deadline so that a caller cannot hand-type one. There is no equivalent helper for a rejection, so elapsed_days is whatever the caller counted. A Reject that derives the count from the recorded deadline and the instant of the rejection would make the number a consequence of the same calendar that produced the deadline, instead of a second opinion about it. The count would still be recorded in the event, and that part is not negotiable, because the fold has to stay replayable, but it would stop being the one field in the acceptance path that is produced by hand.

The general shape is probably worth keeping in mind beyond this library. When you move a clock out of your state machine and into recorded events, you buy determinism and you take on a new obligation: every recorded quantity now has a default, and that default is a business rule whether you chose it or not. Zero is rarely the neutral value it looks like.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

A wei must not print as 1E-18: fixed-point money formatting

polycratia — Fri, 06 Nov 2026 13:21:22 +0000

A money type that is exact in memory and lossy the moment it becomes text has not removed the rounding bug. It moved the bug to the last line of the call stack. Python's Decimal renders a single wei as 1E-18, a hand-rolled units / 10**8 renders one satoshi as 1e-08, and a round(amount, 2) dropped into a template picks a rounding mode that nobody in the system ever chose. In cryptomoney I treat the output path as part of the type: text is fixed point, shortening is allowed only inside the asset's own precision, and an explicit rounding mode is required as soon as a nonzero digit would disappear.

Exactness ends where the string starts

The arithmetic side of this library is careful. A float is refused at construction and every amount is bound to one asset, and anything that might not fit the asset's grid is a method with a mandatory rounding argument rather than an operator. I wrote about that split in an earlier post: Python's operator protocol is fixed-arity, so there is physically nowhere to put a rounding mode, which is why / raises and divide(..., rounding=...) does not.

None of that care survives str() on the underlying Decimal. The decimal spec says to switch to scientific notation once the adjusted exponent drops below -6, and ETH has eighteen decimal places:

from decimal import Decimal

str(Decimal('1E-18'))        # '1E-18'
f"{Decimal('1E-18')}"        # '1E-18', __format__ with an empty spec is str
f"{Decimal('1E-18'):f}"      # '0.000000000000000001'

Decimal is not at fault here. Exponent notation is exact, and it even round-trips through this library's own parser, since the number pattern in parsing.py accepts an exponent. Everything outside this library is the problem: you read a receipt, a spreadsheet opens a CSV export, someone diffs yesterday's balance dump against today's, a support agent holds a number on a screen next to a number in a log. 1E-18 and 0.000000000000000001 are one amount and two strings, and a system that emits both has turned its own output into a parsing problem.

So Money.format builds its text from the f presentation type (f"{magnitude:f}"), which is fixed point by definition. One wei prints as one wei:

from cryptomoney import from_wei

str(from_wei(1))             # '0.000000000000000001 ETH'

__str__ is just format with its defaults, so the cheapest thing a caller can do is also the correct one. That matters more than it sounds: the formatting rules below only hold if nobody has a reason to reach past them.

Shortening is a different operation from rendering

Once rendering is settled, the interesting question is what a caller gets when they want fewer decimal places than the asset has. Eighteen zeros behind a round ETH amount are noise, and a fee table with eight columns of trailing zeros is unreadable. But trimming noise and dropping digits look the same at the call site, and only one of them is lossy.

The distinction format draws is between zeros and significant digits. The helper that decides it is small enough to read in full:

def _significant_decimals(amount: Decimal) -> int:
    """The decimal places the amount needs once its trailing zeros are dropped."""
    digits = amount.as_tuple().digits
    exponent = int(amount.as_tuple().exponent)
    if not any(digits):
        return 0
    while exponent < 0 and digits[-1] == 0:
        digits = digits[:-1]
        exponent += 1
    return max(0, -exponent)

Every amount inside Money has already been quantized to the asset's exponent at construction, so equal amounts carry the very same digit tuple: 0.5 and 0.50000000 and fifty million satoshi are one value with one hash and one text. That padding is what makes the question answerable, because the digits actually present tell you how many of them mean anything.

From there the rules fall out. A requested decimals must be an int and not a bool, or you get a TypeError. It has to sit between zero and the asset's own decimals, or you get a ValueError, because asking BTC for nine places is asking for a digit that cannot exist. If the request is at least the significant count, the quantize that follows cannot lose anything: it only pads or drops zeros, and the ROUND_HALF_UP default has no decision to make. If the request is below that count, the caller's mode is the only thing standing between a display helper and a silently altered number, so there is no default at all:

from decimal import ROUND_DOWN

from cryptomoney import BTC, Money

fee = Money('0.00012500', BTC)

fee.format()                                 # '0.00012500 BTC'
fee.format(trim=True)                        # '0.000125 BTC'
fee.format(decimals=6)                       # '0.000125 BTC', zeros only
fee.format(decimals=4, rounding=ROUND_DOWN)  # '0.0001 BTC', a digit was chosen away
fee.format(decimals=4)                       # ValueError: pass rounding
fee.format(decimals=9)                       # ValueError: above BTC's precision

Money('0.5', BTC).format(decimals=8)         # '0.50000000 BTC'
Money('0.5', BTC).format(decimals=2)         # '0.50 BTC'

trim=True is the unconditional version of the same idea: it strips trailing zeros from the fraction and cannot touch a significant digit, so it needs no mode and no bound. Padding and un-padding are free, rounding is not. The rounding argument runs through the same validator as everywhere else in the library, which raises TypeError for a non-string and ValueError for a mode outside the eight the decimal module defines, so a typo like 'ROUND_SOMEHOW' fails at the call and not three releases later, when someone wonders why a report is off by a satoshi.

The code is at github.com/polycratia/cryptomoney if you want to read the whole path.

Signs that do not lie

Sign handling is where display code usually collects its small untruths, so I wrote the rules out instead of leaving them to be inferred. The leading minus is stripped before the integer and fraction are split on the point, which means grouping and padding operate on the magnitude and do not have to reason about a sign character. The sign then goes back on deliberately: a minus for a negative amount, a plus only when the caller asked for plus and the amount is strictly above zero, and nothing at all for zero.

That last clause is the one worth saying out loud. A delta column that renders +0.00 makes a claim about direction that the number does not support, and a negative zero is an artifact of arithmetic rather than a fact about a balance. Construction already folds negative zero to zero, so the two agree:

from decimal import Decimal

Money(Decimal('-0.00000000'), BTC).format(symbol=False)       # '0.00000000'
Money.from_base_units(0, BTC).format(trim=True, symbol=False) # '0'

Grouping that survives the trip back

Thousands separators are the point where most money formatters stop being reversible. Here the integer part is routed through a grouping helper with a single module-level separator, a comma, and the parser's separator class accepts that comma along with spaces, underscores, apostrophes and the two non-breaking spaces. The output of the formatter is valid input to the parser:

from cryptomoney import USDT, Money, parse_money

text = Money('1234567.5', USDT).format(group=True)
text                                     # '1,234,567.500000 USDT'
parse_money(text) == Money('1234567.5', USDT)   # True

Money('1234567.5', USDT).format(group=True, trim=True)  # '1,234,567.5 USDT'

The separator is deliberately not configurable and deliberately not locale-aware. A locale that groups with a dot produces 1.234.567,5, which this parser rejects, and rightly so: a dot is already the decimal point, and a string where one character means two things cannot be read back unless you know which locale wrote it. Presentation for a specific audience is a layer above a money type, not a feature of it. What this type guarantees is narrower and more useful: what you see is what the system can read.

What I would do differently

Two things nag at me.

First, format(decimals=..., rounding=...) turns a display call into a rounding decision. The mandatory mode keeps it honest, but a stricter design would keep formatting purely textual and force the caller to produce a rounded Money first, so the lossy step shows up in the data flow instead of in a template argument. I chose against that because the check would then live further from where people look, and a developer wiring up a fee column is exactly the person who needs to be stopped. I still think that was right, and I am less sure of it than of anything else here...

Second, trim, group, symbol and plus are four booleans on one call. Four reads fine as keywords, six would not, and the honest fix at that point is a small presentation value rather than another flag. Four is not yet a reason to add a type.

The general lesson carries past crypto amounts. A value type is only as exact as its widest boundary, and for money that boundary is usually not the database, it is the string. If the type refuses a float at the front door and then hands you 1E-18 at the back, it has only relocated the failure to the place where nobody writes tests.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

One path, many spellings: normalizing a robots.txt match

polycratia — Thu, 05 Nov 2026 13:21:22 +0000

A URL path has more than one legal spelling, and a robots.txt matcher that compares a rule to the path as written gets some of them wrong in both directions. /%7Euser/notes and /~user/notes name the same resource, so a rule written Disallow: /~user/ has to catch both of them. /a%2Fb and /a/b do not name the same resource, so nothing in the matcher may quietly read %2F as a separator. A table of special cases per escape is not the way out. You rewrite both sides into one form before any comparison happens, and then you live with the fact that this moves the number which decides precedence.

Two failure modes, opposite directions

The first is a rule that slips. A site owner writes Disallow: /~user/, because that is what the URLs look like in a browser. A crawler sends GET /%7euser/notes: lowercase hex, encoded tilde, entirely within its rights, since a tilde is unreserved and a client may encode it anyway. Byte comparison says no rule applies. The path comes back allowed, and if you are auditing access logs to decide whether a verified crawler respected the policy, you have just recorded a compliant fetch of something the site refused.

The second is a rule that over-reaches. Decode everything before comparing and /a%2Fb becomes /a/b, which Disallow: /a/ now matches. But an encoded slash is one octet inside a single path segment, not a segment boundary. The rule was about a directory; the path was never in it.

Both failures come from one place: the two inputs arrive at the comparison in different alphabets. In the robots package of crawlwarrant (https://github.com/polycratia/crawlwarrant) the only comparison that happens is between two strings that have already been through the same function.

One form, applied to both sides

RFC 9309, Section 2.2.2, asks for exactly one transformation before matching: escapes of unreserved characters are decoded, every other escape is uppercased. Reserved octets stay escaped, and that is what keeps %2F from turning into a slash. The matcher does this to the path once, and to each pattern as it considers it:

func (g Group) Match(path string) (rule Rule, ok bool) {
    p := normalize(path)
    if p == "/robots.txt" {
        return Rule{}, false
    }
    best := -1
    for _, r := range g.Rules {
        pattern := normalize(r.Pattern)
        if pattern == "" || !matches(pattern, p) {
            continue
        }
        if n := len(pattern); n > best || (n == best && r.Kind == Allow && rule.Kind == Disallow) {
            rule, best = r, n
        }
    }
    return rule, best >= 0
}

Three things in those lines are worth naming.

The path is normalized before the /robots.txt exemption is checked, so /%72obots.txt gets the exemption too. The exemption belongs to the resource, not to one spelling of it.

A pattern that normalizes to the empty string is skipped. That is how an empty Disallow: value matches nothing: without the skip it would reach the prefix test as an empty prefix and match every path on the site, which is the opposite of what the line means.

And best holds the length of the normalized pattern, not of the pattern in the file. That is the whole cost of this design, and I come back to it below.

normalize

const hexDigits = "0123456789ABCDEF"

func normalize(s string) string {
    var b strings.Builder
    b.Grow(len(s))
    for i := 0; i < len(s); {
        c := s[i]
        switch {
        case c == '%':
            if v, ok := unescape(s[i:]); ok {
                if unreserved(v) {
                    b.WriteByte(v)
                } else {
                    writeEscape(&b, v)
                }
                i += 3
                continue
            }
            writeEscape(&b, '%')
            i++
        case c <= ' ' || c >= 0x7f:
            writeEscape(&b, c)
            i++
        default:
            b.WriteByte(c)
            i++
        }
    }
    return b.String()
}

func unreserved(c byte) bool {
    switch {
    case c >= 'a' && c <= 'z', c >= 'A' && c <= 'Z', c >= '0' && c <= '9':
        return true
    }
    return c == '-' || c == '.' || c == '_' || c == '~'
}

func writeEscape(b *strings.Builder, c byte) {
    b.WriteByte('%')
    b.WriteByte(hexDigits[c>>4])
    b.WriteByte(hexDigits[c&0xf])
}

unescape reads the two hex digits in either case, so %7e and %7E arrive at the same byte. From there the rule is short: if the byte is a letter, a digit, a hyphen, a period, an underscore or a tilde, it is written plainly, and anything else is re-emitted as a percent sign plus two uppercase digits. A percent sign that does not begin a valid escape is itself escaped, to %25. Any byte at or below the space character, or at or above 0x7f, gets the same treatment.

Two properties matter more than the details here.

It is idempotent. Run it twice and nothing moves: %7E becomes a tilde and stays a tilde, a stray percent sign becomes %25, and %25 decodes to a percent sign, which is not unreserved, so it is written straight back as %25. That matters because patterns and paths go through the same function, and because a caller may hand the matcher a path that some earlier layer has already canonicalized.

It leaves * and $ untouched. They are printable ASCII and not percent signs, so they fall through to the default branch unchanged. %2A does not: an asterisk is a reserved character, unreserved rejects it, and writeEscape puts the escape back. So an encoded asterisk stays a literal octet instead of becoming a wildcard, and an encoded dollar sign does not become an anchor.

Wildcards are read on the normalized text

func matches(pattern, path string) bool {
    anchored := strings.HasSuffix(pattern, "$")
    if anchored {
        pattern = pattern[:len(pattern)-1]
    }
    parts := strings.Split(pattern, "*")
    if len(parts) == 1 {
        if anchored {
            return path == pattern
        }
        return strings.HasPrefix(path, pattern)
    }
    if !strings.HasPrefix(path, parts[0]) {
        return false
    }
    pos := len(parts[0])
    for _, part := range parts[1 : len(parts)-1] {
        i := strings.Index(path[pos:], part)
        if i < 0 {
            return false
        }
        pos += i + len(part)
    }
    last := parts[len(parts)-1]
    if anchored {
        return len(path)-pos >= len(last) && strings.HasSuffix(path, last)
    }
    return strings.Contains(path[pos:], last)
}

The pattern is split on the asterisk. The first part must be a prefix of the path, each middle part is found leftmost after the previous one, and the last part must occur in what remains, or, when the pattern ended in $, be a suffix of the path beginning past the bytes already consumed. That last condition is what the length check enforces: without it an anchored suffix could overlap bytes a middle part already matched, and a pattern would match a path shorter than itself.

Because the split happens after normalization, the alphabet the splitter sees is fixed. No spelling of a path introduces a wildcard, and there is no spelling of a pattern where a literal asterisk hides inside an escape and then springs open during matching.

The cost: precedence is measured on the normalized form

RFC 9309 breaks ties by octet count: the longest matching rule wins, and an allow beats a disallow of equal length. The count is taken after normalization, which means it is not the count in the file. Consider:

User-agent: ExampleBot
Disallow: /%7Euser/
Allow: /~user/

As written, the disallow is eight bytes and the allow is seven, so the disallow is longer and would win. Normalized, both patterns are the same seven bytes, and the tie goes to the allow. /~user/notes is allowed, and the rule that decided it sits on line 3.

Normalization moves lengths the other way too. /a b/ is five bytes in the file and seven once the space is escaped, so it can take precedence over a competing rule it would have lost to on raw bytes. A rule can gain or lose precedence purely because of how its author spelled it.

I do not think a better answer is available. Ranking by the bytes in the file makes the outcome depend on exactly the authoring choice that normalization exists to erase, and it puts the first failure mode back at the tie-break instead of at the match. The specification already made this trade when it defined the comparison on normalized text, and measuring length anywhere else would be inconsistent with the comparison that produced the candidates.

What it does cost is explainability. The verdict chain in the identity package quotes the rule as the file wrote it (the kind, the pattern and the line), which is the right thing to show a site owner reading an audit. But when a rule loses on normalized length, a line naming /%7Euser/ and line 2 does not carry the reason it lost...

What I would change

Normalize patterns at parse time rather than inside Match. Today every call re-normalizes every rule of the group, and an audit replaying an access log calls it once per request. The reason it is not done yet: Rule.Pattern is public and reports what the file said, so the normalized form has to arrive as a second field rather than a rewrite of the first.

And once that field exists, put it in the explanation: report the pattern as written, then add the normalized form when it differs. A site owner who wrote two spellings of the same prefix should be able to read that off the output instead of reconstructing it.

One thing I would not change is the handling of query strings. The path handed to Match includes the query as the request carried it, and normalize treats those bytes like any other, so ? and & pass through as printable ASCII. Query strings have their own encoding conventions, and a matcher that started applying them would be deciding things RFC 9309 never asked it to decide.

Close

The discipline here is narrow and worth keeping narrow: a matcher should have exactly one place where its two inputs become comparable, and nothing downstream of that place should know anything about spellings. Once normalize is the only door, %2F cannot become a slash by accident, because nothing past the door is looking at escapes at all.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

Confirmation depth for an unlisted asset: one, not never

polycratia — Wed, 04 Nov 2026 13:27:49 +0000

A confirmation policy written as a mapping of per-asset depths is incomplete by construction: the mapping names the assets someone thought about, and a watched address is free to receive one nobody did. Something still has to decide how deep is deep enough for that asset. The answer that sounds safest, hold the payment until a human decides, is the one that loses payments quietly.

This came up in chain-watch, a small library I keep for the part of custodial deposit handling that is always the same: follow a set of addresses, count confirmations, report each deposit exactly once, hand it back if the chain takes the block away. Crediting is a one-way door. Once the balance moves, the customer can spend it, while a chain is under no obligation to agree with itself about what happened three blocks ago. The depth policy is where the caller states how much agreement it wants before that door opens.

Depth is a judgment about one asset

There is no correct number of confirmations. A chain with fast blocks and a chain that settles in minutes do not deserve the same depth, and a token can deserve a stricter number than the coin it rides on: a single stablecoin transfer is routinely worth more than a single native transfer, while both are secured by exactly the same work. Per-asset depth is not a configuration nicety. It is the only honest shape I can give the knob.

So the policy is a default plus overrides, and the watcher asks it about every transfer it holds:

from chain_watch import ConfirmationPolicy, DepositWatcher

policy = ConfirmationPolicy(default=6, per_asset={"ETH": 12, "USDT": 12})
watcher = DepositWatcher(source, ["addr-1"], policy=policy)

policy.depth_for("BTC")   # 6, the default
policy.depth_for("USDT")  # 12, the override

That is the complete case, and it is not the one that bites. The case that bites is the caller who writes only the overrides, because that is what people actually do: they know they care about two assets and they do not want to think about a third.

A mapping is a partial function

The watcher never takes a raw depth. Its policy argument accepts a policy, a bare integer or a mapping, and everything goes through one classifier:

@classmethod
def coerce(cls, spec: DepthSpec) -> ConfirmationPolicy:
    if isinstance(spec, ConfirmationPolicy):
        return spec
    if isinstance(spec, Mapping):
        return cls(per_asset=spec)
    if isinstance(spec, int) and not isinstance(spec, bool):
        return cls(default=spec)
    raise TypeError(
        f"expected a ConfirmationPolicy, an int or a mapping of depths: {spec!r}"
    )

The order matters, and so does the one thing the mapping branch does not do: it does not raise the default. cls(per_asset=spec) leaves default at its dataclass value of 1. Pass policy={"ETH": 12} and you have asked for twelve confirmations on ETH and one confirmation on every asset in existence, including the ones that do not exist yet.

Written down like that it looks careless. It is the deliberate part.

"Hold it until someone decides" is a silent failure

The alternative default is infinity: treat an unnamed asset as never deep enough and keep it in pending until an operator extends the policy. That one fails review in about a minute of thinking about operations.

A transfer waiting for a depth it can never reach produces nothing. poll() returns a PollResult that is falsy when both confirmed and reverted are empty, so if not watcher.poll() reads as "nothing happened," and that is exactly what it would say, poll after poll, while a real payment sits on a real address. No exception, no log line, no row in a queue anyone has a reason to look at. The pending map is an internal detail of the watcher, not an alerting surface, and nobody writes a dashboard for the thing they did not anticipate. The failure mode is a support ticket: the customer sent a token, the chain shows it arrived, the balance never moved, and the first person who can explain why is whoever reads the watcher's internals.

With the default at one confirmation, the same payment arrives as a deposit at the first block that contains it, carrying the depth it actually had. Depth counts the block itself, so a transfer sitting in the tip has one confirmation, and you see the whole truth in the object:

KNOWN_ASSETS = {"BTC", "ETH", "USDT"}

result = watcher.poll()
for deposit in result.confirmed:
    if deposit.asset not in KNOWN_ASSETS:
        # Reported, visible, and explicitly not credited.
        review.record(deposit.key, deposit.address, deposit.amount, deposit.asset)
        continue
    ledger.credit(deposit.address, deposit.amount, deposit.asset)

The unknown asset is now a record with an address, an amount, a confirmations count and a (tx_id, output_index) key, in a code path the caller wrote on purpose. Crediting it is still a decision a human makes. The difference is that the decision exists.

The trade this makes is worth stating plainly, because it is a real one. A shallow report is a weaker statement than a deep one, and if the caller blindly credits whatever confirmed contains, a mapping-only policy will credit an unknown token at depth one. That is a caller-side bug with a one-line fix. Infinite patience is a library-side behaviour with no fix at all, because there is nothing to fix: the system is working as designed and producing silence. Between a weak report the caller can gate on and no report the caller cannot see, I take the weak report every time.

Tightening is one call, and it does not mutate anything

The loud default is only acceptable because correcting it is trivial. The policy is a frozen dataclass, its overrides are wrapped in a MappingProxyType after validation, and tightening returns a new value rather than editing one some other watcher may be holding:

def with_asset(self, asset: str, depth: int) -> ConfirmationPolicy:
    """Return a copy that requires ``depth`` for ``asset``."""
    return ConfirmationPolicy(self.default, {**self.per_asset, asset: depth})

So the operational answer to a shallow unknown deposit is a new policy and a watcher that uses it, not a mutation applied to a running object mid-poll. One knock-on effect is worth knowing: reorg_depth defaults to policy.max_depth, which is the largest of the default and the overrides. A mapping-only policy therefore sizes its reorg window from the assets that were listed, and the unlisted asset, reported at depth one, stays revertible for as long as that window lasts. The shallow report is not final. It can still come back as reverted.

Zero and `True` are not depths

The same code that is permissive about an unnamed asset is strict about the number itself. Every depth, the default and each override, goes through one check:

def _validate_depth(depth: int, label: str) -> int:
    if isinstance(depth, bool) or not isinstance(depth, int):
        raise TypeError(f"confirmation depth for {label} must be an int: {depth!r}")
    if depth < 1:
        raise ValueError(f"confirmation depth for {label} must be at least 1: {depth}")
    return depth

A depth below one is refused because zero confirmations means crediting a payment that is not in any block. That is not a lenient policy, it is a different product: a claim about the mempool dressed up as a claim about the chain. There is no caller-side gate that recovers from it, because the deposit was reported as confirmed.

The bool rejection is the same reasoning one level down. True is an int in Python, and {"USDT": True} would quietly become one confirmation on the asset the caller most likely meant to protect. coerce refuses a bare True as a default for the identical reason. A config file that produced a boolean where a depth belongs is wrong, and the distance between wrong and a stricter-looking default of one is exactly the kind of gap that survives review.

Note the asymmetry: an asset nobody named is permitted and reported shallow, a depth nobody can justify is refused at construction. The unknown asset is the caller's ignorance about the chain, which is normal and recoverable. A zero depth is the caller's ignorance about what a confirmation is, which is not.

What I would do differently

Asset names are matched exactly as the chain source spells them, per_asset.get(asset, self.default), no normalisation, so "usdt" and "USDT" are two different assets and only one of them has an override. Combine that with a default of one and a mis-keyed override degrades silently to a shallow credit that looks exactly like a legitimately unknown asset. The policy cannot catch this: it has no idea which assets the source can report.

The check belongs one layer out, at startup, where the deployment does know its universe: refuse to boot if an override names an asset the configured source never spells, and refuse to boot if an asset the deployment credits has no override. That turns the typo into a failed deploy instead of a weak deposit, and it leaves the policy a pure value with no knowledge of any particular chain. I would not fix it by lowercasing keys inside depth_for; the chain's spelling is the chain's, and the library guessing at case is probably a worse bug than the one it prevents.

The rule I keep from this: when a configuration is a mapping, the behaviour for everything the mapping does not name is not a leftover, it is the most important branch in the file. Pick the one that produces a visible, cheap-to-correct outcome, and write down why, because the safe-sounding alternative produces nothing at all, and nothing is very hard to debug.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

Why a parser cursor has to stay failed after the first refusal

polycratia — Tue, 03 Nov 2026 13:21:21 +0000

A C++ parser for a binary format fails in one of two ways: it trusts a length field and reads past the end of its buffer, or it has a bounds check on every single read and nobody keeps that up. The second failure is the more interesting one, because people agree with the discipline right up until they skip it. A record header is ten reads, and honest per-read checking means you write the same conditional ten times to reach one return std::nullopt.

I have spent most of the last decade on payment and crypto systems, so most of my parsing work has been on framed messages arriving off a socket from something I do not control. The conclusion I keep arriving at is not more checks. It is a cursor that stays failed once it has refused anything, so the ten checks collapse into one question you ask at the end. That is the whole premise of bytespan, a header-only C++20 cursor I wrote for exactly this shape of problem.

The check everybody agrees with and then skips

Written out by hand, a correct parse of a three-field header looks roughly like this:

std::size_t offset = 0;

if (input.size() - offset < 1) return std::nullopt;
std::uint8_t type{};
std::memcpy(&type, input.data() + offset, 1);
offset += 1;

if (input.size() - offset < 2) return std::nullopt;
std::uint16_t version{};
std::memcpy(&version, input.data() + offset, 2);
offset += 2;
// ...and again, for every field

Nothing is wrong with this except that it does not survive contact with a real format. By the sixth field the conditional has been copied five times, and a copy is where input.size() - offset < n turns into offset + n < input.size(), or where the offset += n goes missing, or where the whole guard gets skipped because the field three lines up already proved there were bytes left. I have inherited this code more than once during legacy modernisation work. The bug is never interesting: it is the same bug wearing a different field name.

So the design question is not how to check a read. It is how to make a parse of ten fields cost one check without being wrong.

Two early returns

Every consuming operation in bytespan's reader goes through one private method:

bool take(std::size_t n) noexcept {
    if (!ok()) return false;
    if (n > data_.size() - offset_) {
        poison({offset_, reason::short_input, n, data_.size() - offset_});
        return false;
    }
    offset_ += n;
    return true;
}

Those two early returns are the entire mechanism. The second one is the ordinary bounds check. The first is what changes the economics: once the cursor is poisoned, take refuses before it looks at anything, so every later read fails too. u8, u16, u32 and u64 all go through a read<T> that calls take(sizeof(T)) before its memcpy and returns std::nullopt on refusal; bytes and skip call take directly. No path advances the cursor without passing the check, and no path passes the check on a cursor that has already failed.

What you get at the call site is the shape from the repository's own example:

std::optional<record> parse(bytespan::reader& r) {
    record out{};
    const auto type = r.u8();
    const auto version = r.u16(bytespan::endian::big);
    const auto payload = r.length_prefixed_u16(bytespan::endian::big);

    if (!r.ok()) return std::nullopt;
    out.type = *type;
    out.version = *version;
    out.payload = *payload;
    return out;
}

The three dereferences after the check are safe, and not by convention. Stickiness makes "ok() is true" equivalent to "every read on this cursor succeeded", because the first failure would still be latched when the check runs. Without stickiness that equivalence does not hold and the single check is a lie: you would have to test each optional, which is the ten-conditional version again with worse ergonomics.

Note also that take does not move offset_ when it refuses. The cursor never ends up positioned past bytes it did not actually read, which matters if the caller wants to log offset() or resume. That decision has a consequence of its own, and it is the one people miss.

The loop that would otherwise spin

A cursor that refuses without advancing is a cursor whose position no longer changes. Drive a loop off the byte count and you get an infinite loop:

std::vector<std::span<const std::byte>> extensions;
while (r.remaining() > 0) {
    if (const auto body = r.length_prefixed_u16(bytespan::endian::big)) {
        extensions.push_back(*body);
    }
}
if (!r.ok()) return std::nullopt;

This terminates, and it terminates for a reason you cannot see from the loop body: remaining() returns 0 once the cursor has failed, instead of data_.size() - offset_. Sticky failure and a non-advancing cursor are each individually correct and together they produce a hang, so the third piece is mandatory rather than a convenience. It is also what lets the loop above contain no error handling at all: a malformed extension list ends the loop, and the single ok() afterwards says the parse is dead. A refused length-prefixed read additionally resets offset_ back to the start of its prefix, so there is genuinely nothing left in the cursor that could make progress.

The writer mirrors all three properties: its own take compares against out_.size() - offset_, its remaining() is zero once it has failed, and put and bytes refuse after the first overrun. An overrun fails the writer rather than the process, and stays failed.

The bound is a subtraction, and that is not cosmetic

n > data_.size() - offset_ and offset_ + n > data_.size() are the same statement in arithmetic and different statements in C++. The left-hand form subtracts; the invariant that offset_ only ever advances through a successful take means offset_ <= data_.size() holds, so the subtraction cannot underflow. The right-hand form adds, and n is frequently attacker-controlled: it comes out of the length field you just read. On a platform where std::size_t is 32 bits, a declared length near the top of the range makes offset_ + n wrap to something small, the comparison passes, and the memcpy runs for the full unwrapped count. The check becomes the exploit.

That is why the length-prefixed read can cast a 32-bit count straight to std::size_t and compare it:

const std::size_t wanted = static_cast<std::size_t>(*length);
if (wanted > data_.size() - offset_) {
    poison({start, reason::short_input, detail::saturating_add(sizeof(T), wanted),
            data_.size() - start});
    offset_ = start;
    return std::nullopt;
}
offset_ += wanted;

The prefix and the payload are one call because splitting them is where the second check gets forgotten, and the refusal is recorded against start (the first byte of the prefix) so it names the field rather than a position inside it. The demo output shows what that reads like: a four-byte input whose prefix claims 65535 bytes of payload reports refused at 3: short input (wanted 65537, had 3). The 65537 is the two-byte count folded into what the field as a whole needed.

Keeping the first refusal, not the last

ok() decides control flow; it does not write the log line. So the cursor keeps a record, offset, reason, bytes wanted, bytes available, and poison only writes it when there is nothing there yet:

void poison(refusal record) noexcept {
    if (!refusal_) refusal_ = record;
}

That ordering falls out of stickiness. After the first failure, every later call fails as a consequence, and those consequences are uninformative: a header that died on a truncated u32 must not end up reporting the single-byte read that followed it. Last-write-wins would bury the only refusal that explains the parse. The writer's tests pin the same property from the other side: an overrun at offset 2 wanting four bytes with one left keeps saying exactly that, even after a smaller u8 and an empty bytes have both failed behind it.

What I would do differently

One thing the sticky cursor makes harder, and I have not found a clean way around it: it does not distinguish "this message is truncated, wait for more bytes" from "this message is malformed, drop the peer". Both arrive as short_input, which is correct, the parser genuinely cannot tell, but a streaming caller has to make that call from the refusal's offset and sizes plus knowledge of its own framing. If I were starting again I would probably still refuse to guess inside the library, because a cursor that speculates about your transport is a cursor that is wrong on somebody else's transport. But the record is the only thing standing between the caller and a bad decision there, which is a good argument for it carrying sizes rather than just a reason.

The other cost is that [[nodiscard]] optionals plus a run-then-check style means binding values you will never read. const auto version = r.u16(...) exists to be dereferenced after the check, and in a parse where you only care about presence it is noise. I accepted that; the alternative is dropping [[nodiscard]], which removes the compiler's only warning about an ignored read on a cursor you forgot to check.

The discipline that survives review is the one that costs a single line. Everything above is in service of that line being if (!r.ok()), and of it meaning exactly what it appears to mean.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.

DEV Community: polycratia

Refusing a snapshot a deposit watcher cannot fully read

A partial restore is a silent double-credit

Name the field, not the file

Type every scalar, including the bool that is an int

Rebuild values through the door a chain source uses

The version is a refusal, not a hint

What the strictness buys at the boundary

What I would do differently

Reserving a length prefix and its body in one bounds check

The second failure point is the one that escapes

One check, then two writes that cannot refuse

The arithmetic is part of the guarantee

When the length is not known until the body is written

A varint written whole

What I would do differently

One weight estimator for script types and size hints

Two descriptions of the same output

The hint is a stretch, not a second model

Only the whole transaction knows whether it carries a witness

One rounding, at the end

Where per-input rounding survives, and why that is safe

What I would do differently

Why a Markdown scanner should blank code spans, not delete them

Masking preserves length; deleting does not

Fenced blocks are skipped whole, and that costs nothing

From a column to a splice point

The rewrite distrusts the position it was given

What I would do differently

Why a ring buffer's capacity must divide 2^64

The shape that makes the overflow matter

What a capacity of 1000 actually does

A condition the compiler checks

Exercising a wrap you cannot reach

What I would do differently

The 401 loop: when discovery and audience binding disagree

The loop, and what each field in it carries

A document the server would reject is never published

The well-known segment goes in front of the path, not after it

Checked against the verifier, on every request

What I would change

One disarm override per pattern, not per key

The key is a pattern, so the override is a template

One template, sometimes two pairs

The split is the printable half

What I would change

A resumed acceptance window is measured by the rejection that stopped it

The term lives in the contract head

The fold has no clock, so the rejection carries the count

The two ways a recorded count goes wrong

What I would do differently

A wei must not print as 1E-18: fixed-point money formatting

Exactness ends where the string starts

Shortening is a different operation from rendering

Signs that do not lie

Grouping that survives the trip back

What I would do differently

One path, many spellings: normalizing a robots.txt match

Two failure modes, opposite directions

One form, applied to both sides

normalize

Wildcards are read on the normalized text

The cost: precedence is measured on the normalized form

What I would change

Close

Confirmation depth for an unlisted asset: one, not never

Depth is a judgment about one asset

A mapping is a partial function

"Hold it until someone decides" is a silent failure

Tightening is one call, and it does not mutate anything

Zero and True are not depths

What I would do differently

Why a parser cursor has to stay failed after the first refusal

The check everybody agrees with and then skips

Two early returns

The loop that would otherwise spin

The bound is a subtraction, and that is not cosmetic

Keeping the first refusal, not the last

What I would do differently

Zero and `True` are not depths