Most teams treat test coverage as a proxy for reliability. The logic feels airtight: more tests mean more things are being verified, which means fewer things can go wrong. So they scale the suite. More scripts, more flows, more cases. And for a while, it works.
Then it stops working - not suddenly, but gradually. Tests start failing for reasons no one can immediately explain. Some get re-run until they pass. Some get quarantined. Some just get ignored. The suite still runs. The numbers still look respectable. But nobody actually trusts the results anymore.
That is not a tooling problem. It is a strategy problem that tooling is being blamed for.
Automation does not fail at scale because teams do it wrong. It fails because the conditions under which it succeeded no longer exist.
The early phase of automation looks reliable because the system is simple. Fewer dependencies, more predictable behavior, stable assumptions. When those assumptions hold, even poorly designed tests produce useful signal. Teams mistake that phase for a repeatable blueprint - and apply it to a system that has fundamentally changed.
Mobile makes this worse than anywhere else. Device fragmentation, OS variability, unstable network conditions, asynchronous behavior - these are not edge cases. They are the default environment. Every new layer of coverage introduces new surface area for failure, and the failure modes are not always obvious or reproducible. Flaky tests do not mean the tests are bad. They mean the system has hidden instability that the tests are now exposing in the worst possible way: inconsistently.
The deeper issue is that maintenance never appears on the roadmap but always competes with it. A UI change cascades into a dozen broken scripts. A flow update requires hunting down every test that touched that path. What started as a velocity investment becomes a velocity tax. Engineers stop building with tests as support and start maintaining tests as a second product.
More coverage is not the answer when the problem is that you are automating the wrong things.
The teams that get automation right at scale are not the ones with the largest suites. They are the ones that have drawn a clear line between what is worth automating and what is not. High-impact, stable flows deserve automation. Exploratory areas, rapidly changing UI, and edge cases that only surface in production do not. Shrinking the surface area is not cutting corners - it is what makes the remaining coverage trustworthy.
Automation should give you confidence to move faster. When it starts doing the opposite - when failures are ignored more than investigated, when releases feel slower despite high coverage, when debugging a test takes longer than fixing the bug - the suite has crossed from asset to liability.
That is not a sign to add more tests. That is a sign to rethink what the tests are for.
👇 Read the full breakdown: Test Automation for Mobile Apps: Why Most Setups Fail at Scale
Top comments (0)