We had a checkout test at my last job that everyone called "the coin flip." Green for a week, red twice on a Tuesday, green again. Someone eventually wrapped it in a retry and it sat like that for eight months before anyone looked at it again. Turned out the real bug was a webhook that occasionally fired before the order record finished writing to the DB - a two-hundred-millisecond gap that only showed up under load. The test wasn't broken. It was the only thing in the entire pipeline that noticed.
That's usually the story. Someone blames the test - bad selector, missing wait, a sleep(2) some intern left in there three years ago, and half the time they're right. But when a test flakes repeatedly and nobody can explain why, the test is rarely the actual problem. It's just the part of the system rude enough to say something.
A few places I keep finding the real cause hiding.
Tests that quietly depend on each other
Test A writes a row, Test B reads it and never knew it needed to. Run B by itself, it passes. Run the suite in a different order, or in parallel, and B fails for no reason anyone can point to. I've lost a full afternoon to this exact thing more than once - a cache value from Test 12 leaking into Test 47.
The actual fix is annoying and unglamorous: every test gets its own fixtures, its own scoped data, no assumptions about what ran before it. If your suite only goes green in one specific order, you don't have a flaky test. You have an undocumented dependency graph, and it's going to bite someone eventually.
The app is racing, not the test
Click a button, immediately assert on the result - that's a bet that the UI update lands the instant the click handler returns. It usually does, on your machine, on a good day. Add a debounce, a background job, or just enough network latency and that bet stops paying off.
This one's frustrating because the test isn't being paranoid. The app genuinely has a race condition. The test just runs the interaction often enough, across enough machines, that it eventually catches the app mid-race - something a human clicking through the same flow once or twice would probably never notice.
Clocks, timezones, and CI boxes that aren't your laptop
Date.now() and anything DST-adjacent will happily pass for months and then fail on one specific day of the year, or the moment your CI runner's timezone doesn't match what the original author assumed (usually their own laptop, usually not documented anywhere). Same idea with tests that were written assuming reasonably fast hardware - throw them on a throttled CI box and timing assumptions that never mattered locally suddenly do.
Mocks that stopped matching reality
This is the sneaky one, because it doesn't look like flakiness at all - it looks like a test that's passing. Someone mocks a third-party API, gets it right on day one, and then the real API changes: a field gets renamed, an error shape changes, rate limiting gets added. The mock has no idea. It keeps returning exactly what it always returned, the test stays green, and production quietly starts failing in a way nothing in CI can see.
If anything this is worse than a red test, because it erodes trust in the wrong direction - teams end up more confident in a mock-backed test than they should be, right up until it fails somewhere that actually matters.
The twenty-minute question
It's easy to treat a flaky test as noise to manage - quarantine it, slap a retry on it, mute it and move on. But almost every one of them is pointing at something real: state nobody's tracking, a race condition in the app itself, an environment assumption that was never written down, or a mock that fell out of sync with the thing it's supposed to represent.
Muting it doesn't make any of that go away. It just makes the pipeline quieter while whatever's actually wrong keeps happening, unwatched, in production.
Next time a test earns a reputation, it's worth the twenty minutes to ask why before reaching for a retry decorator. Most of the time it wasn't lying to you.
Top comments (0)