Every one of these ran green. No exception, no failed execution, nothing in the log. Each one was found by going and looking at something the system never told me about.
They are all the same bug, wearing four different costumes: the check reported on something adjacent to the thing that mattered.
1. The trigger that returned 1 of 18
I appended 17 test messages to a mailbox with an IMAP trigger listening, and the trigger returned exactly one.
Not an error. A successful execution with one item.
"Fetch Only New Emails" is on by default, and on recent versions of that node it pushes SINCE <today> onto the IMAP search. My corpus was backdated, so it matched nothing. The mailbox and an empty mailbox are indistinguishable from the outside.
Reading the shipped source, the search is built from two mutually exclusive branches:
if (staticData.lastMessageUid !== undefined) {
searchCriteria.push(['UID', `${staticData.lastMessageUid}:*`]);
} else if (node.typeVersion > 2 && options.trackLastMessageId !== false) {
searchCriteria.push(['SINCE', activatedAt.toFormat('dd-LLL-yyyy')]);
}
A stored UID watermark beats everything else, and it is never validated against the mailbox — the code's own comment notes that UIDs change if a mailbox is recreated, and that UIDVALIDITY is how you would detect it. It is not checked. A watermark that no longer matches produces zero results forever, with the trigger reporting perfect health.
The tell: "no new mail" and "my query excluded everything" produce identical output. If your trigger returns nothing, prove the query is right before you believe the mailbox is empty.
2. The folder that was created perfectly, in the wrong place
A workflow built one directory per client from the company name. Client two was Acme/West Coast Consulting.
mkdirSync with recursive: true did exactly what it is documented to do: created clients/Acme/, then West Coast Consulting/ inside it, and wrote all six files two levels deep. Exit status zero, green run, confirmation page shown to the user.
The fix is one line — strip to an allowlist, [^a-z0-9]+ to dashes — and the lesson is the shape, not the slash: anything a person types that becomes a path, key or URL needs normalising at the boundary, and the failure will be silence rather than an exception. Go the other way and build a blocklist and you will remember / and forget ...
3. The verifier that called 25 correct quotes fabricated
A research agent has to quote its sources; a gate checks each quote actually appears in the page it cites. On one run the gate rejected 25 out of 25.
I nearly shipped that as a headline about model honesty. Every single rejection was our own bug: the HTML-to-text step turned tags into spaces, so Doppler, who became Doppler , who and a correct quote stopped matching the page it came from.
The model was right. The checker was wrong, and it was exactly as confident as a hallucinating model would have been.
What changed: before trusting a rejection, the checker is fed deliberately corrupted claims — one digit changed, a word dropped, a noun swapped, a sentence invented outright — and each has to still fail. A verifier that has only ever seen correct input will approve anything.
4. The 69 bytes added after the gate approved the message
This one is my favourite, because everything worked.
Eighteen lines of plain code decide whether an AI-drafted reply may be sent: every fact in it has to exist in a file a human wrote. On the filmed run, 6 drafts, 6 cleared honestly, every fact cited.
Then I read the message back out of Sent Mail and diffed it against the body the gate approved:
Approved body bytes 182
Landed body bytes 251
Added after body 69
This email was sent automatically with n8n. — the send node's attribution option, on by default, applied after the node's parameters resolve, which is after every check in the workflow. The same feature on the HTML path of an earlier build added 312 bytes, including a tracked link with a campaign parameter.
Nothing malicious. But the sentence I was about to say out loud — "nothing goes out that isn't supported by the source file" — was false, and it was false in a way no test in the workflow could ever have caught, because the workflow is upstream of the thing that modified the message.
The shape
In all four, a check passed and reported on the wrong object:
| the check said | what it was actually about |
|---|---|
| the run succeeded | the query, not the mailbox |
| the directory was created | the string, not the location |
| the quote is absent | our parser, not the source |
| the body is approved | memory, not the delivered message |
Three habits that catch this class, none of them clever:
- Assert on the artifact, not the step. "Did it run" is nearly free information. "Does the thing I wanted exist, where I wanted it, containing what I expected" is the assertion worth writing.
- Record counts, not just outcomes. "Ran, returned 0" and "ran, returned 40" showing the same green tick is what hides a broken pipeline for days.
- Read it back from the far end. Whatever your system produces — a file, a row, an email — fetch it from where the recipient gets it and compare. It is the only check that survives a dependency quietly changing behaviour next quarter.
Every number above came off a recorded run, and the workflows are public: https://github.com/Ships-Itself/builds
Top comments (1)
"The check reported on something adjacent to the thing that mattered" is going straight into my incident vocabulary. We just lived a version of this: a scheduled content pipeline that ran green for over a week while producing nothing, because an upstream process leak was starving the scheduler's evening slots. Every individual component was healthy. The schedule existed, the runner worked, exit codes were fine. The only thing missing was the output — and nothing asserted on the output.
The fix that actually stuck for us was embarrassingly low-tech: a canary that checks the artifact, not the process. "Did a file matching today's date appear in this directory by 10am, yes or no." It would have caught all four of your costumes too, I think — the 1-of-18 trigger fails a count assertion, the misplaced folder fails a path assertion, and the 69-byte post-gate mutation fails a checksum taken at the gate.
Your IMAP case is the nastiest of the four because the empty result is indistinguishable from a legitimately quiet mailbox. Did you end up seeding a known-good sentinel message so "zero" becomes distinguishable from "broken"? That's the only defense I've found against triggers whose failure mode is silence.