DEV Community

Cover image for What Actually Breaks When Your Playwright Suite Hits 10,000 Tests
Abd-ur-Raffae Masood
Abd-ur-Raffae Masood

Posted on

What Actually Breaks When Your Playwright Suite Hits 10,000 Tests

Scaling Playwright past a few thousand tests isn't a "throw more CI runners at it" problem. It's an isolation problem. This post covers the architecture decisions — fixtures, storage state, sharding, worker-scoped setup — that determine whether 10,000 tests run in an hour or in a day.


Somewhere between 500 tests and 10,000, every Playwright suite hits the same wall — and it's rarely the thing people expect going in. It's not that Playwright can't handle the volume. It's that the shortcuts that felt harmless at 500 tests — a shared test user, a flat tests/ folder, a waitForTimeout(5000) copy-pasted into fifty files — start compounding into hours of wasted CI time and failures that only reproduce "sometimes."

Here's what actually determines whether a suite scales gracefully or collapses under its own size.

Test count isn't the multiplier — everything test count touches is

More tests means more concurrent browsers, more API calls hitting your backend, more database writes, and — the one people underestimate — more surface area for tests to quietly interfere with each other. A 500-test suite might run in 15 minutes. Scale the same design to 10,000 naively, and it doesn't take 20x longer — it can take days, because interference between tests grows faster than test count does.

More CI runners doesn't fix that. A framework built to isolate from day one does.

The four things that actually determine whether you scale

  • Isolation — can a test run alone, first, last, or in parallel and get the same result every time?
  • Parallelism — is your suite actually structured to use multiple workers, or just configured to?
  • Architecture — can a new teammate find and safely extend a test without reading the whole repo first?
  • Feedback speed — do you find out a test broke in 3 minutes, or 3 hours?

Weakness in any one caps what the other three can do for you. Add all the parallel workers you want — if tests share state, parallelism just makes failures show up more often, not less.

Isolation is the real bottleneck, not hardware

This is the failure mode that kills most scaling efforts:

// Fragile: depends on execution order and shared state
test('creates a discount code', async ({ page }) => {
  await page.goto('/admin/discounts/new');
  await page.getByLabel('Code').fill('SUMMER25');
  await page.getByRole('button', { name: 'Create' }).click();
});

test('applies the discount at checkout', async ({ page }) => {
  // Silently depends on the previous test having run first
  await page.goto('/checkout');
  await page.getByLabel('Promo code').fill('SUMMER25');
});
Enter fullscreen mode Exit fullscreen mode

Works fine sequentially. Add parallel workers and it breaks — sometimes — which is the worst kind of failure to debug. Give every test its own data lifecycle instead:

// Resilient: each test owns its full data lifecycle
test('applies a discount code at checkout', async ({ page, request }) => {
  const code = `TEST-${crypto.randomUUID().slice(0, 8)}`;

  await request.post('/api/discounts', {
    data: { code, percentOff: 25 },
  });

  await page.goto('/checkout');
  await page.getByLabel('Promo code').fill(code);
  await expect(page.getByText('25% off applied')).toBeVisible();

  await request.delete(`/api/discounts/${code}`);
});
Enter fullscreen mode Exit fullscreen mode

A few lines longer, but it can run alone, first, last, or across ten parallel workers and never once care what else is happening.

Ditch global variables, use fixtures

// Dangerous under parallel execution
let currentUser: User;

test('logs in', async ({ page }) => {
  currentUser = await createTestUser();
});
Enter fullscreen mode Exit fullscreen mode

Two workers hitting this simultaneously stomp on each other's state. Fixtures give each test its own isolated instance:

export const test = base.extend<{ testUser: User }>({
  testUser: async ({}, use) => {
    const user = await createTestUser();
    await use(user);
    await deleteTestUser(user.id);
  },
});

test('shows the dashboard after login', async ({ page, testUser }) => {
  await loginAs(page, testUser);
  await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
});
Enter fullscreen mode Exit fullscreen mode

Stop logging in through the UI for every single test

If every test does a full UI login before it starts, you're paying that cost thousands of times over. Authenticate once, save the session, reuse it:

// global-setup.ts
async function globalSetup() {
  const browser = await chromium.launch();
  const page = await browser.newPage();
  await page.goto('/login');
  await page.getByLabel('Email').fill('qa-runner@example.com');
  await page.getByLabel('Password').fill(process.env.TEST_PASSWORD!);
  await page.getByRole('button', { name: 'Sign in' }).click();
  await page.context().storageState({ path: 'auth/state.json' });
  await browser.close();
}
Enter fullscreen mode Exit fullscreen mode
// playwright.config.ts
use: {
  storageState: 'auth/state.json',
}
Enter fullscreen mode Exit fullscreen mode

A 3-4 second UI login, repeated thousands of times, is hours of CI time nobody needed to spend. This one change alone tends to be the single biggest win teams find.

Amortize expensive setup per worker, not per test

export const test = base.extend<{}, { apiClient: ApiClient }>({
  apiClient: [
    async ({}, use) => {
      const client = await ApiClient.connect(process.env.API_URL!);
      await use(client);
      await client.close();
    },
    { scope: 'worker' },
  ],
});
Enter fullscreen mode Exit fullscreen mode

Spinning up a connection 6 times (once per worker) instead of 10,000 times (once per test) adds up fast.

Shard across machines once one machine caps out

npx playwright test --shard=1/4
npx playwright test --shard=2/4
npx playwright test --shard=3/4
npx playwright test --shard=4/4
Enter fullscreen mode Exit fullscreen mode

More workers on one machine eventually hits a CPU/memory ceiling. Sharding splits the whole suite across multiple machines — a 12,000-test, 4-hour suite can often drop to around an hour split across four shards, assuming your tests are isolated enough that shard assignment doesn't matter (which, again, loops back to isolation being the actual foundation here).

Tag tests so PRs don't run everything

test.describe('@smoke', () => {
  test('user can complete checkout', async ({ page }) => { /* ... */ });
});
Enter fullscreen mode Exit fullscreen mode
npx playwright test --grep @smoke       # every PR
npx playwright test --grep @regression  # nightly / pre-release
Enter fullscreen mode Exit fullscreen mode

Running all 10,000 tests on every PR turns a 5-minute check into a 90-minute one. A tight @smoke tag on every push, full suite reserved for nightly or release gates, tends to be the better tradeoff.

Never scale a suite with hardcoded waits

// Harmless in one test. A silent tax across 10,000.
await page.waitForTimeout(5000);

// Wait for the actual condition instead
await expect(page.getByRole('status')).toHaveText('Order confirmed');
Enter fullscreen mode Exit fullscreen mode

Flaky tests need a process, not just a retry count

retries: process.env.CI ? 2 : 0,
Enter fullscreen mode Exit fullscreen mode

Retries should smooth over real infra hiccups, not quietly launder a race condition into green. Rough workflow that's worked well:

reproduce with --repeat-each → classify (app bug / test bug / infra) → fix root cause → drop out of quarantine

Keep a quarantine tag for tests under investigation, but track how long things sit there — indefinite quarantine is just slow-motion ignoring.

Metrics worth watching weekly

  • Total execution time
  • Pass rate
  • Flaky test count
  • Slowest test files (where to optimize first)
  • Retry rate (how often "passing" needed a second try)

A suite that gets 20% slower every month for six months is a suite nobody notices until it's a crisis.

The actual takeaway

None of this is about buying more CI capacity. It's about cutting the unnecessary work your suite does — redundant logins, shared state forcing serialization, sleeps waiting for nothing, full regression runs on every PR when a smoke suite would do. Scaling Playwright isn't a hardware problem wearing an engineering costume. The fixes are the same ones that make a 500-test suite good — they just stop being optional once you cross into five figures.


What's the thing that actually broke first for you at scale — shared test data, CI time, or flaky tests eating trust in the suite? Curious how others' bottlenecks compared.

Top comments (0)