I had twenty-two passing tests and two separate reviewers on a piece of code. None of it objected. Then I pointed it at a real API owned by somebody else and it broke on the first live read.
The mismatch fit in one sentence: my parser required ISO 8601, the documented API returned Unix seconds. The repair was not one line. It touched five files, 74 lines of parser and 52 lines of tests. The assumption was small; making it safe was not.
Here is why nothing caught it, and it is the part worth keeping:
My tests used ISO because my code used ISO, so they agreed with each other and never checked reality.
The fixtures were written by the person who wrote the parser. They encoded the same assumption. The suite confirmed internal behaviour without ever challenging the ISO assumption, because both halves of it came from one head. Internally consistent is not the same claim as right, and nothing in that suite could tell the difference.
Two separate reviewers missed it too. I cannot prove why, and I am not going to invent a reason. What I can show is that the parser and every fixture encoded the same ISO assumption, so none of the artifacts in front of anyone supplied the live contract that contradicted it.
The second one was worse
Working against a real system made redirect containment matter, so an independent breaker went at it. In Python 3.13 the default redirect handler rebuilds the redirected request from req.headers, dropping only content length and type. My X-API-Key sat in that header set, so the redirected request inherited it. Python has Request.add_unredirected_header() for exactly this, which marks a header as one that will not be added to a redirected request. I was not using it. The breaker reproduced it offline with a sentinel value and a cross-origin Location, and the sentinel crossed.
No live FIPSign credential was ever shown to have crossed an origin. The defect was real and unshipped. I did not find it by auditing my own code, and I did not find it myself — the live integration made the question worth asking, and a seat that was not mine answered it.
What it takes to actually get that
On 2026-06-07 I opened an issue on tacoda/keystone, an agent charter framework, asking whether a narrow adapter test was possible against a real signing API. FIPSign (mobydickfinance) said yes and then did something I did not expect: he handed over disposable API keys, rotated them on request, and stayed in that thread for ten weeks.
The collaboration produced a block I could not explain. POST /mandate came back with a Cloudflare 1010, browser_signature_banned. That stopped the cycle at issuance: no mandate was created, and CAPTURE, PATCH and EVALUATE never ran.
I could have guessed. He didn't let either of us do that. His reply:
Checked both questions directly, not from assumption.
Then he listed what he actually did. Reviewed the CORS and origin handling in the Worker source and confirmed server-to-server calls authenticated by API key never enter that logic at all. Checked Bot Fight Mode specifically. Compared the public guide field by field against the real request validation in the backend — length limits, scope counts, budget types, expiry bounds, required headers — and confirmed nothing was undocumented. Then ran POST /mandate himself from PowerShell on a residential Windows connection and got a clean signed mandate back.
Both of my hypotheses were dead. It was not the CORS or origin policy I suspected, and it was not a missing requirement in the docs.
What he asked me for next is the part I want other people to steal:
the exact timestamp · the HTTP client/library + version · whether the request went through any proxy or VPN · what kind of network you are running it from — home connection, cloud VM, CI runner, corporate network
With the note that hosting-provider IPs get scored very differently from residential ones, independent of the client library.
I gave him the client. Python-urllib/3.13.
that's the default, unmodified user agent for Python's
urllib, and it's one of the most commonly flagged signatures by automated bot detection precisely because so much low-effort scraping traffic uses it unchanged.
Then, before I spent another key:
if you set a real user agent on your client, I'd bet that alone resolves it. Worth trying first — cheap to test.
If you are hitting an endpoint from a script and getting bot-flagged for no visible reason, check what user agent your HTTP library is sending by default. You probably never set one. That is a five-second check and it is the most portable thing in this entire post.
The part where I refused the result
One seat added the header, an independent seat confirmed it was present in the request urllib assembled for transport, and I authorized a single controlled cycle that a separately assigned firing seat executed once. POST /mandate returned 201. No 1010. CAPTURE 200, PATCH 200, EVALUATE 200. One attempt per stage, no retry, no redirect follow.
That is the clean story: header missing, endpoint blocked, header added, endpoint opened.
I am not going to tell you that.
FIPSign's operator had confirmed nothing changed on his side — Bot Fight Mode off then and off now, never toggled, a Browser Integrity Check rule drafted during the investigation but never deployed. That statement is dated 2026-08-16 17:35Z. The firing seat ran the controlled cycle on 2026-08-19.
Three days sit between a written statement about a provider's configuration and the run that depends on it. He was not being careless and neither was I. But nobody controlled that window, so a success is attributable to the header only as far as that statement reaches. One run, one header, three days later, is not causation. It is a result with an uncontrolled gap in front of it.
And then the worse gap, which was mine
That run's issuance receipt contains no User-Agent field at all.
The entire point of the run was the header. A narrative receipt written afterwards names it. The artifact that binds the 201 does not. The two share a run id, so software can associate them, but the User-Agent assertion sits outside the sealed receipt chain rather than inside it. A reader holding the artifact that binds the 201 still cannot recover which user agent produced it.
We built the capture afterwards — it records the configured value, the mapping handed to the transport, and what urllib assembled after add_header, three witnesses at three different strengths, none of them TLS-level and none of them the edge's view. It exists because the previous run could not testify about its own independent variable, which is not a good reason for a thing to exist.
The route opened. The experiment did not establish why. The receipt did not bind the one variable that was supposed to explain the change. That is a worse failure than the block was.
The result carries CONTROLLED_EXTERNAL_API_FIXTURE on its face. The project authored the PATCH, so it is not independent lifecycle evidence and I will not present it as any.
What the ten weeks were actually worth
Not the 201. The 201 is a fixture with a disclosed ceiling.
What it was worth was two defects in my own code, found by two different paths, and collapsing them into one lesson would overclaim what the receipts show.
The timestamp mismatch survived twenty-two green tests and two separate reviewers because the parser and every fixture carried the same wire-format assumption. The live response is what contradicted it.
The redirect defect was different. It survived a green local suite and was found by an independent breaker attacking the adapter with a sentinel value, before any live credential could cross. One was reality contradicting my fixture. The other was an adversarial seat asking a question my tests had not asked. One of them was a cross-origin credential-forwarding path.
His closing note put it plainly, and he is right:
it says something good about testing against a live system instead of only mocks.
For an external wire contract, a fixture you invented from your own reading of someone else's docs cannot independently validate that reading. My suite could not challenge the ISO assumption because every fixture already carried it.
Reality is a better adversary than your fixtures. It is not a substitute for a receipt.
Thread is public: tacoda/keystone#4, opened 2026-06-07, forty-one comments when audited on 2026-08-30. Every quote above is from it. **The sanitized run receipt for 2026-08-19 is not currently public* — it is credential-adjacent and has not been through a publication scan, so the User-Agent gap described above is stated on my word rather than something you can open. I would rather say that than imply a receipt you cannot read. The keys used for the Mandate runs described here were disposable, delivered privately, reported spent, and revoked by the issuer.*
Top comments (0)