A failing test tells you something is broken. You fix it, the bar goes green, everyone moves on. That test did its job.
A flaky test is different. It passes, then it fails, then it passes again, and nothing in your code changed. It's the test that cost my team more hours last year than any real bug we shipped.
Here's what makes it worse than a normal bug. A normal bug is visible. It fails, you see red, you fix it. A flaky test hides in plain sight, because half the time it agrees with you. It's the coworker who's right often enough that you stop questioning them, and wrong exactly when it matters.
What flaky tests train people to do
The real damage from a flaky test isn't the failed run. It's what it teaches your engineers to do.
The first time a test fails for no reason, someone re-runs the pipeline. It goes green. Lesson learned: red doesn't always mean broken. The second time, they re-run it without even reading what failed. By the tenth time, the whole team has quietly agreed that a red build is a suggestion, not a fact.
That's the expensive part. You didn't just lose ten minutes per re-run. You lost the thing the suite was supposed to give you: the ability to trust green. Once nobody trusts the suite, a real failure slides through, because the muscle memory is to hit retry and move on.
Where flakiness usually hides
Most flaky tests I've dug into come from a small set of causes. A test that depends on timing and assumes an async operation finishes in 100ms. A test that shares state with another test and only fails when they run in a certain order. A test that hits a real network or a real clock instead of a controlled one.
None of these are exotic. Every one is the test author borrowing reliability from something they don't control. The fix is almost always to stop borrowing: freeze the clock, isolate the state, mock the boundary, wait on a condition instead of a fixed duration.
The worst one I chased took a week. A test passed alone and failed in CI, only on Tuesdays. A different test was writing a date-based cache key and never cleaning it up, and the collision only happened on certain weekdays. The bug was never in the code under test. It lived in the test next door.
At Shanti Infosoft we treat a flaky test as a real bug with its own ticket, not a nuisance to retry past. If it can lie to us, it's broken, even when it's green.
Quarantine, then fix or delete
When we find one we can't fix in the moment, we don't leave it in the main suite poisoning trust. We move it to a quarantine group that runs separately and doesn't block the build. Then it gets a deadline.
Here's the part people resist. If a quarantined test sits unfixed past its deadline, we delete it. A test that never runs and never gets fixed isn't protecting anything. It's guilt in the repo with a green checkmark. A smaller suite you trust completely beats a large suite you argue with.
The goal was never a big number of tests. The goal was to look at a green checkmark and believe it.
It's the same instinct behind the QA layer we add when AI writes most of the code: a green check you can't trust is worse than no check at all.
When your build goes red, does your team read the failure, or reach for retry?
Top comments (0)