DEV Community

Cover image for Most “Flaky Tests” Are Really Architecture Tests
Antoine Dubois
Antoine Dubois

Posted on

Most “Flaky Tests” Are Really Architecture Tests

There’s a point in almost every browser automation project where someone says:

“The tests are getting flaky.”

And then the team starts fixing the tests.

Longer waits. More retries. New selectors. Another helper function. Maybe a wrapper around the wrapper around click().

Sometimes that’s the right answer.

But increasingly, I think “flaky test” is becoming a catch-all phrase for something much broader:

Modern frontends have become surprisingly difficult environments to observe deterministically.

The test isn’t always the problem.

The DOM Isn’t Really the DOM Anymore

A decade ago, browser automation had a fairly straightforward mental model.

Load page. Find element. Click element.

Today you might be dealing with:

  • Shadow DOM
  • nested iframes
  • third-party widgets
  • microfrontends
  • virtualized lists
  • components that render differently based on their container size
  • UI updates arriving over WebSockets
  • client-side navigation without full page loads

That’s a different problem.

For example, testing Shadow DOM, nested iframes, and embedded widgets is less about writing the perfect selector and more about whether your automation system understands the boundaries between those different contexts.

A perfectly valid selector can still be useless if your test is searching in the wrong DOM context.

Responsive Testing Got More Complicated Too

We used to think about responsiveness in terms of viewport width.

Desktop. Tablet. Mobile.

CSS container queries changed that assumption.

A component can now change dramatically while the browser window stays exactly the same size.

That means testing only a handful of viewport resolutions can miss real regressions. There’s a useful breakdown of testing container queries, resize behavior, and breakpoint edge cases that illustrates how these failures tend to appear.

This is one of those changes that looks small from the CSS side and surprisingly large from the QA side.

Then There’s Time

A lot of UI state isn’t created by page load anymore.

It arrives later.

Consider an application consuming WebSockets or Server-Sent Events.

The test opens the page.

The page says “Connected.”

The test waits for the transaction.

Nothing happens.

Failure.

Run it again.

Pass.

This is where arbitrary sleeps become dangerous. You’re trying to fix a state synchronization problem with a stopwatch.

A better approach is understanding how WebSocket, SSE, and live-data browser tests fail and synchronizing against meaningful application state instead of elapsed time.

The same principle applies to SPAs.

Browser history, back-button behavior, restored scroll positions, and cached component state can produce bugs that don’t appear when every automated test starts from a clean URL.

That makes client-side routing and scroll restoration worth treating as first-class behavior rather than navigation trivia.

Selectors Still Matter. Just Differently.

I don’t think selectors are going away.

But I do think teams often spend too much time trying to make one clever selector survive every imaginable frontend change.

That becomes especially painful with microfrontends.

One team renames a component.

Another team introduces a wrapper.

A shared UI package changes its DOM structure.

Suddenly 87 tests fail.

A sane selector strategy for microfrontends and shared component libraries is mostly about establishing contracts between developers and tests.

That’s much more scalable than XPath archaeology.

Virtualized Interfaces Are Another Trap

Infinite scroll and virtualized tables are great examples.

An item can conceptually exist in the application while not existing in the DOM.

That distinction matters.

It’s why comparing approaches such as Endtest vs Playwright for heavily virtualized interfaces gets interesting. The question isn’t merely whether the tool can click an element.

The question is how much machinery you have to build around that click.

The Useful Question

When a browser test fails, I’d stop asking:

“How do we make this test pass?”

At least initially.

Ask:

“What assumption did this test make about the application?”

Maybe it assumed the element was already rendered.

Maybe it assumed the route caused a page load.

Maybe it assumed the item existed in the DOM.

Maybe it assumed the component responded to viewport width.

Maybe it assumed there was only one document context.

Once you find that assumption, the failure usually becomes much less mysterious.

And strangely enough, your tests tend to become much less flaky too.

Top comments (0)