DEV Community

Cover image for Why Your Playwright Suite Gets Flaky After 50 Tests (And How to Fix It)
Shaban Umar
Shaban Umar

Posted on Fully Autonomous

Why Your Playwright Suite Gets Flaky After 50 Tests (And How to Fix It)

#webdev #playwright #typescript #testing

Every Playwright suite starts clean. Twenty tests, all green, runs in ninety seconds. Then it grows.

By test fifty, someone has added a waitForTimeout to "fix" a race. By test a hundred, the suite fails maybe one run in four — never the same test twice. People start re-running the pipeline until it goes green. At that point the suite has stopped being a safety net and become a tax.

I've watched this happen on enough projects to notice it's almost always the same five causes. None of them are exotic. Here they are, with the fix for each.

1. waitForTimeout is not a wait, it's a bet

This is the big one. It shows up the first time a test fails intermittently and someone reaches for the fastest fix:

await page.click('#submit');
await page.waitForTimeout(2000);        // ← the bet
await expect(page.locator('.result')).toBeVisible();
Enter fullscreen mode Exit fullscreen mode

The bet is that two seconds is always enough. On your laptop it is. On a loaded CI runner at 3am it sometimes isn't. And on the days it is enough, you're still paying two seconds — multiply that across a hundred tests and you've added three minutes to every run for nothing.

The fix: wait for the thing you actually need.

await page.click('#submit');
await expect(page.locator('.result')).toBeVisible();   // auto-waits, up to the timeout
Enter fullscreen mode Exit fullscreen mode

Playwright's assertions already retry until they pass or time out. You almost never need an explicit wait. When you genuinely do — a click that fires an XHR whose result you need before continuing — wait for that specific thing:

const response = page.waitForResponse(res => res.url().includes('/api/order'));
await page.click('#submit');
await response;
Enter fullscreen mode Exit fullscreen mode

Enforce it with lint, not discipline. Discipline fails under deadline pressure. Add eslint-plugin-playwright and turn on no-wait-for-timeout as an error. Now the pipeline rejects it and nobody has to police it in review.

2. Tests that share data collide in parallel

This one is sneaky because it only appears once you turn on parallelism.

test('registers a new user', async ({ page }) => {
  await registerUser(page, 'test@example.com', 'Password123!');
  await expect(page.locator('.welcome')).toBeVisible();
});
Enter fullscreen mode Exit fullscreen mode

Works perfectly alone. Run it with two workers, or run it twice without resetting the database, and the second one hits "email already registered". The failure looks like a registration bug. It isn't. It's your test data.

The fix: generate unique data per test, never hardcode it.

const user = {
  email: `qa.${randomUUID().split('-')[0]}@example.com`,
  password: 'Str0ng!Passw0rd',
};
Enter fullscreen mode Exit fullscreen mode

A small factory module is worth writing. And — this sounds excessive until it saves you a day — write unit tests for the factory itself. A factory that silently produces duplicates causes failures that look exactly like application bugs, and you will spend hours in the wrong codebase before you suspect it. Asserting "2000 generated emails are all unique" takes thirty milliseconds and removes that entire class of confusion.

3. Every test logs in again

Not flake exactly, but it compounds everything else. If each test starts by filling a login form, you're paying three to five seconds per test and giving yourself a hundred extra chances to hit a timing failure on the one flow that runs most often.

The fix: log in once, save the session, reuse it.

Make a setup project:

// tests/auth.setup.ts
setup('authenticate', async ({ page }) => {
  await page.goto('/login');
  await page.getByLabel('Username').fill(process.env.TEST_USERNAME!);
  await page.getByLabel('Password').fill(process.env.TEST_PASSWORD!);
  await page.getByRole('button', { name: 'Login' }).click();

  await expect(page.getByRole('link', { name: 'Logout' })).toBeVisible();
  await page.context().storageState({ path: 'playwright/.auth/user.json' });
});
Enter fullscreen mode Exit fullscreen mode

Wire it up as a dependency:

projects: [
  { name: 'setup', testMatch: /.*\.setup\.ts/ },
  {
    name: 'chromium',
    use: { ...devices['Desktop Chrome'], storageState: 'playwright/.auth/user.json' },
    dependencies: ['setup'],
  },
]
Enter fullscreen mode Exit fullscreen mode

On a 200-test suite this routinely cuts several minutes off every run. It's usually the single biggest speed win available.

One detail that matters: make the setup assert loudly that login actually succeeded. If it fails silently, every downstream test fails in a confusing way and you'll debug the wrong thing.

4. Locators tied to markup instead of meaning 

await page.click('.btn-primary.submit-form > span:nth-child(2)');
Enter fullscreen mode Exit fullscreen mode

This passes today and breaks the next time a developer wraps something in a div. Worse, it breaks silently in the sense that the failure tells you nothing about what actually changed.

The fix: locate by what the user perceives.

await page.getByRole('button', { name: 'Submit order' }).click();
Enter fullscreen mode Exit fullscreen mode

Priority order that holds up well:

  1. getByRole — survives redesigns, matches what users and screen readers see
  2. getByLabel — ties a field to its accessible label
  3. getByTestId — when the markup gives you nothing semantic
  4. CSS/XPath — last resort

There's a side benefit worth naming: if getByRole and getByLabel can't find your elements, that's usually a real accessibility gap in the app. Your test suite starts doubling as an a11y smoke check for free.

5. Visual tests that fail on everything

Teams add visual regression, get failures on every unrelated change, and switch it off within a month. Three causes:

Animations. A screenshot taken mid-transition never matches. Disable them globally:

expect: {
  toHaveScreenshot: { animations: 'disabled', maxDiffPixelRatio: 0.02 },
}
Enter fullscreen mode Exit fullscreen mode

Font rendering across machines. macOS and Linux anti-alias differently, so baselines generated locally will never match CI. Either generate baselines inside Docker so everyone compares against identical rendering, or keep a small pixel tolerance.

Genuinely dynamic content. Timestamps, avatars, ad slots. Mask them:

await expect(page).toHaveScreenshot('dashboard.png', {
  mask: [page.locator('time'), page.locator('.ad-slot')],
});
Enter fullscreen mode Exit fullscreen mode

And prefer component-level snapshots over full-page ones. A full-page baseline fails when anything anywhere changes. A snapshot of just the checkout form fails when the checkout form changes — which is the signal you actually wanted.

The pattern underneath all five

Every one of these is the same mistake in a different costume: encoding an assumption about timing or state instead of waiting for the real thing.

A fixed delay assumes timing. A hardcoded email assumes no one else is running. A CSS-path locator assumes the DOM shape. A full-page snapshot assumes nothing else on the page ever changes.

Replace each assumption with the actual condition and the flake goes away. Not "mostly goes away" — goes away. Flaky suites aren't an inherent property of browser testing, they're a property of suites built on assumptions.

A practical order to fix this in

If you've inherited a flaky suite, don't rewrite it. Go in this order — cheapest and highest-impact first:

  1. Add the lint rule and let it find every waitForTimeout for you
  2. Add auth caching — biggest speed win, touches one file
  3. Fix the top 3 flakiest tests — your CI history knows which ones they are
  4. Replace brittle locators as you touch each file, not in one big sweep
  5. Add data generation when you next hit a collision

You'll typically get most of the benefit from the first two.

I packaged these patterns — plus API testing, CI sharding and Docker setup — into a Playwright starter kit, since I kept rebuilding the same thing on every project. But everything in this article works on its own; none of it requires buying anything.

Top comments (0)