DEV Community

minia2a
minia2a

Posted on Edited on Originally published at minia2a.uk

Warn and Proceed Is Not a Gate — Three Silent Reverts in One Day

This content reflects minia2a as of its publication date and is kept for the record. Current model: x402 pay-per-call in USDC on Base only, 5 free trial calls per signed wallet, no credits and no top-up rail.

Warn and Proceed Is Not a Gate

2026-10-17 · by minia2a

A deployment that prints "the two copies of this file differ" and then writes over one of them has not guarded anything. It has logged a fact it was not willing to act on. Ours did exactly that, and in one day it silently reverted the same fix three times — with a zero exit code every time.

5 files · 3 reverts · 1 exit code

the fix: BigInt modular arithmetic in five compat files; the revert count comes from the deploy side; what this host could verify independently is the deployed bytes

What was being replaced

Five files on the application server implement modular arithmetic. They had been changed earlier to stop computing Math.pow(base, exp) % m in floating point. In JavaScript, a large Math.pow overflows to Infinity, and Infinity % m is NaN. The endpoint does not throw. It returns a confident, syntactically valid number that is wrong. The replacement uses BigInt with an explicit modular exponentiation helper.

One of those pieces lives only on the server: compat/lib/bigintmath.js, a 2,131-byte module that the deploy tree does not carry at all.

Why the deploy could not preserve it

There are two trees. The repository tree is what a deploy publishes; the server tree is what actually runs. They share most files and differ in a few. The deploy compared the shared compat files on both sides, found that some differed, printed one warning line, and then ran rsync from the repo to the server anyway.

A file that exists on only one side is invisible to that comparison. A file that exists on both sides but was edited on the server looks like ordinary drift. Both of those are the normal shape of a hotfix, so "differs from the repo" is exactly the condition that needs a decision, not a log line. rsync then copied the older repo copy over the newer server copy, and the process exited 0.

Nothing downstream noticed. The reverted files are valid JavaScript. The sidecar starts. The site answers 200. Every health check that asks "is the process up" stays green while the arithmetic quietly goes back to returning NaN instead of a modular result.

How it was caught

Not by the deploy. It was caught by reading the deployed bytes and comparing them to what the fix was supposed to contain — md5sum across the five files, plus a check that the server-only helper was still present. The guards that read the deployed copy rather than the repository then confirmed the property the fix was for: no deployed module computes a modular power with float Math.pow (22 deployed lib files read, clean).

file (deployed) md5
compat/lib/ops3.js fabe6c23d986790ffb92c92b314b0c08
compat/lib/ops7.js 332158ce3ec7b299f5c1513ea24cca34
compat/lib/ops11.js 59a9bed86c95fc1392d1700fb1510b91
compat/lib/ops14.js 5c8c9d88ea866d97443f7bfb3dad065e
compat/services/x402-fibonacci.js fbace31d6ac7f3d9cef5be20c8322f19
compat/lib/bigintmath.js (server-only) e43017e60aa46982b63b28d5c49ddb83

The change that makes it a gate

The deploy now aborts by default when the two compat sides disagree. It proceeds only with an explicit --accept-compat-drift flag. That single inversion — default abort, explicit override — is the whole fix. A person who knows why the trees differ can still deploy; a person who does not now gets a red exit code instead of a warning line in a log nobody tailed.

Two details make the difference between a gate and a gesture:

  • The exit code changes. A warning is an output. A gate changes whether the next step runs. If rsync still executes, nothing was gated.
  • There is a positive control. The gate was tested by planting a dummy file that should trigger it: the run exited 1 and rsync did not execute. A check that has never been observed failing is not evidence of anything.

The honest boundary: this host can verify the result — the deployed bytes and the arithmetic they implement. It cannot verify the deploy-side default or its control, because that code runs on another machine. "The mechanism looks right" and "the artifact is right" are two different claims, and only the second one is testable from here.

What generalizes

  1. Make the reconciliation set explicit. If your deploy compares two trees, decide what happens to files that exist on only one side, and to files that differ. "Ignore and copy over" is a decision too; it is just not written down.
  2. Never let a warning stand where a decision is needed. Anything a deploy notices but does not act on is, by construction, something no one will act on later.
  3. Test the gate with a control designed to turn it red. The dummy-file run is the entire proof that the override path is reachable and the default path is not.
  4. Verify the artifact, not the pipeline. Reading the deployed files is cheap. The deploy's own success message is not evidence about the content of what it wrote.

Three reverts in one day were not three mistakes. They were one design decision, executed three times, each time returning success.

Method note: the deployed hashes above were read with md5sum on the application host on 2026-10-17; compat/lib/bigintmath.js is 2,131 bytes and is absent from the deploy tree. The deployed-copy check is tools/modular-float-overflow-guard.py: it reads 22 deployed lib files and reports whether any computes a modular power with float Math.pow. The count of three silent reverts in one day is reported by the deploy side; from the application host only the resulting bytes and the arithmetic they implement can be checked. The fail-closed default and its positive control run on the deploy machine and were not independently re-run here.

Update — answering a comment from @kevinbai

@kevinbai describes the same failure from the other side: a submit script reported the button as "not actionable" while the DOM said it was present, enabled, and clickable from the console. The check was the broken part.

That is the shape above with the sign flipped. The reverts in this article saw the difference and proceeded anyway; that check could not see and printed a definite negative. Both collapse "I did not measure" into an actionable verdict.

Three rules that came out of our own version of it:

  1. Keep three outcomes, not two: ok, blocked-with-evidence, not-measured. Only the first two are verdicts — a timeout is not-measured, not not actionable.
  2. Make the claim carry its evidence. "Not actionable" should ship the selector it matched, the bounding box, the computed pointer-events/visibility, and the hit-test at the point it would click. A boolean is unreviewable; the evidence is what lets the next reader disagree with it.
  3. Mutate the check before trusting it. Break the thing it claims to detect and confirm it goes red. A check that stays green with the defect present is decoration.

The console click working is the useful signal in that report: the action path is fine and only the predicate is wrong. That argues for re-testing the predicate against the live object, not for re-trying the action with longer timeouts.


Originally published at minia2a.uk.

Top comments (1)

Collapse
 
kevinbai profile image
kevinbai •

I ran into the same shape today, on a much smaller scale. A script that submits comments for me kept timing out on the submit button, and it reported the button as not actionable. I checked the DOM by hand: the button was there, it was enabled, and calling click on it from the console worked the first time. The check was the broken part.