DEV Community: Ankit Patel

Your CI Runs Tests in Parallel. Your Test Data Doesn't Know That.

Ankit Patel — Fri, 18 Sep 2026 16:04:13 +0000

There's a specific kind of test failure that survives every fix you throw at it.

It passes locally. It passes when you re-run the single spec in CI. It passes when you
retry the job. Then it fails again two days later in a different spec entirely, one
nobody touched. Somebody adds retries: 2, the dashboard goes green, and the team
moves on.

That test isn't flaky. It's colliding.

The difference that matters

Flakiness and collision look identical on a dashboard and have nothing in common
underneath.

A flaky test is non-deterministic in isolation. Race condition, animation timing,
a network call without a proper wait. Run it alone a hundred times and it will
eventually fail on its own.

A colliding test is perfectly deterministic. Run it alone a thousand times, it passes a
thousand times. It only fails when another test is touching the same data at the same
moment.

The tell is in which test fails. Real flakiness fails the same spec repeatedly.
Collision moves. Today it's checkout, next week it's profile settings, and the only
thing they share is a database row.

If your failures wander, stop tuning waits. You have a state problem.

Where it comes from

Almost always the same place: a fixture written when the suite ran serially, that
nobody revisited after fullyParallel: true got switched on.

The classic is one seeded account.

// seeded once, used everywhere
const TEST_USER = { email: 'qa@example.com', password: 'Test123!' };

Serially, that's fine. In parallel it's four workers logging into the same account,
invalidating each other's sessions, mutating the same cart, racing the same
notification list. Nothing is non-deterministic. Everything is contended.

The same shape shows up as a global-setup record every spec mutates, a hardcoded
order_id two suites both update, or a cleanup step that truncates a table while
another worker is mid-assertion.

Why the usual fixes plateau

Retries convert a real signal into a slower green. The collision still happens.
You've just agreed to stop hearing about it. Worse, retries hide the failure that would
have told you which shared record is the problem.

test.describe.serial works, and it's the fix most teams reach for. It also
surrenders the parallelism you turned on for a reason. Do it in enough places and your
20-minute suite is a 50-minute suite again, which is usually the moment somebody starts
arguing for deleting tests.

A bigger CI machine does nothing. Contention is logical, not about cores.

Each of these treats the symptom because the symptom is what the dashboard shows you.

Give every worker its own data

The fix is that no two workers should ever be able to see each other's records.
Playwright gives you the seam for this directly: worker-scoped fixtures.

// fixtures.ts
import { test as base } from '@playwright/test';

type WorkerFixtures = {
  account: { id: string; email: string; password: string };
};

export const test = base.extend<{}, WorkerFixtures>({
  account: [
    async ({}, use, workerInfo) => {
      // unique per worker AND per run
      const email = `qa+w${workerInfo.workerIndex}-${Date.now()}@example.com`;
      const account = await createAccount({ email, password: 'Test123!' });

      await use(account);

      await deleteAccount(account.id);
    },
    { scope: 'worker' },
  ],
});

Two things are doing the work. scope: 'worker' means the account is created once per
worker process, not once per test, so you get isolation without paying signup cost on
every spec. And workerInfo.workerIndex plus a timestamp means worker 0 and worker 3
cannot collide, and neither can this run and the one still finishing from the last
merge.

Specs then say nothing about which user they're using:

import { test } from './fixtures';

test('cart survives a page reload', async ({ page, account }) => {
  await login(page, account);
  await addToCart(page, 'SKU-1041');
  await page.reload();
  await expect(page.getByTestId('cart-count')).toHaveText('1');
});

For data that genuinely must be per-test rather than per-worker (an order that gets
mutated, say), use a test-scoped fixture and namespace it the same way, with
testInfo.parallelIndex or a run id in the key.

Then clean up by prefix, never globally:

// delete only what this worker created
await db.deleteWhere({ emailLike: `qa+w${workerInfo.workerIndex}-%` });

A TRUNCATE in an afterAll is the single most reliable way to break a suite that was
otherwise fine.

The objection worth taking seriously

Creating real accounts per worker costs setup time, and on a large suite that is a
genuine tradeoff. Somebody will point out that one seeded user is faster.

They're right about the arithmetic and wrong about the comparison. Worker-scoped means
the cost is paid once per worker, not per test. On a 400-test suite across 4 workers,
that's 4 signups, not 400. Weigh those 4 against the retries you're currently running
and the hours spent re-investigating a wandering failure every sprint.

The other fair objection: some systems can't create accounts on demand: a legacy
provisioning flow, a paid third-party seat, a partner sandbox with fixed credentials.
That's real. There you want a pool: a fixed set of N accounts, one checked out per
worker, released at the end. Slower to set up, same isolation property. What doesn't
work is one account and hope.

What to do on Monday

Pull your last 30 CI failures. Group them by spec. If the failures are spread across many specs rather than concentrated in a few, you have collisions, not flakiness.
Grep the suite for hardcoded credentials and ids: qa@, test@, any literal id in a fixture. Each one is a shared mutable resource.
Take the single most contended one and move it to a worker-scoped fixture with a worker-indexed key.
Find every describe.serial and note why it was added. Some are protecting real sequences. Most are scar tissue from a collision nobody diagnosed.
Check your cleanup for a global delete or truncate. Scope it by prefix.

Suites that stay fast under parallelism aren't the ones with clever waits. They're the
ones where no two tests can reach the same row.

I'm Ankit Patel, Director of Test Automation at QAble.
I spend most of my time inside other teams' test suites. Currently building Testbo-X, a
Playwright framework focused on intelligent test selection and parallel CI execution.

Why Your Playwright Suite Gets Slower Every Sprint (And the Fix Nobody Talks About)

Ankit Patel — Fri, 04 Sep 2026 15:51:11 +0000

Every Playwright suite starts fast. Forty tests, ninety seconds, green on every push. Then eighteen months pass and the same suite takes twenty-six minutes, half the team has stopped reading the CI output, and somebody suggests "maybe we just run E2E nightly."

I have watched this happen across enough client codebases now that I no longer think it's a discipline problem. Teams that are careful about test quality still end up here. The slowdown is structural, and most of the advice you'll find treats the symptom.

Here's the part that actually matters: your suite doesn't get slower because you added tests. It gets slower because you added setup.

The thing that actually grows

Pull up your slowest spec and time the assertions. On most suites I've profiled, the assertions are a rounding error. The time goes to getting the browser into the state where the assertion makes sense.

A concrete example. This is a checkout test, lightly anonymised, that took 41 seconds:

test('applies a discount code at checkout', async ({ page }) => {
  await page.goto('/signup');
  await page.fill('[name=email]', `user-${Date.now()}@example.com`);
  await page.fill('[name=password]', 'hunter2');
  await page.click('button[type=submit]');
  await page.waitForURL('/onboarding');

  await page.click('text=Skip for now');
  await page.waitForURL('/dashboard');

  await page.goto('/products/sku-1024');
  await page.click('text=Add to cart');
  await page.goto('/cart');
  await page.click('text=Checkout');

  await page.fill('[name=discount]', 'SAVE20');
  await page.click('text=Apply');
  await expect(page.locator('.total')).toHaveText('$80.00');
});

One assertion. Thirteen lines of getting there. And every one of those thirteen lines is a full round trip through your app — real HTTP, real render, real navigation.

Now multiply. You have sixty tests that need a logged-in user. Sixty signups. If signup takes six seconds, that's six minutes of your suite doing nothing but proving that signup works, which you already have a dedicated test for.

Why the usual fixes plateau

Sharding. Genuinely useful, and you should do it. But sharding divides wall-clock time, not work. Four shards turn 26 minutes into 7, and your CI bill goes up roughly fourfold. You've bought time with money, and in another eighteen months you'll buy it again.

Trimming "flaky" tests. Usually this means deleting coverage in exchange for a green dashboard. The test was often telling you something true about a race condition in your app.

fullyParallel: true. Good default. Doesn't help when the bottleneck is that sixty workers are each independently signing up a user against the same backend.

These aren't wrong. They're just operating downstream of the actual problem.

Move setup out of the browser

The fix is to stop performing setup through the UI and start injecting it.

Playwright's storageState handles authentication. Sign in once, globally, save the cookies and localStorage, reuse everywhere:

// global-setup.ts
import { chromium, FullConfig } from '@playwright/test';

async function globalSetup(config: FullConfig) {
  const browser = await chromium.launch();
  const page = await browser.newPage();

  await page.goto('http://localhost:3000/login');
  await page.fill('[name=email]', 'seeded-user@example.com');
  await page.fill('[name=password]', process.env.TEST_PASSWORD!);
  await page.click('button[type=submit]');
  await page.waitForURL('**/dashboard');

  await page.context().storageState({ path: 'storage/user.json' });
  await browser.close();
}

export default globalSetup;

// playwright.config.ts
export default defineConfig({
  globalSetup: require.resolve('./global-setup'),
  use: { storageState: 'storage/user.json' },
});

Sixty signups collapse into one. That alone usually takes a third off a mature suite.

Then push further. Application state — a cart with items, an account mid-onboarding, an order awaiting refund — belongs in a fixture that talks to your API, not your UI:

// fixtures.ts
import { test as base } from '@playwright/test';

type Fixtures = { cartWithItems: { cartId: string } };

export const test = base.extend<Fixtures>({
  cartWithItems: async ({ request }, use) => {
    const res = await request.post('/api/test/carts', {
      data: { items: [{ sku: 'sku-1024', qty: 1 }] },
    });
    const cart = await res.json();
    await use(cart);
    await request.delete(`/api/test/carts/${cart.cartId}`);
  },
});

The checkout test becomes:

test('applies a discount code at checkout', async ({ page, cartWithItems }) => {
  await page.goto(`/cart/${cartWithItems.cartId}/checkout`);
  await page.fill('[name=discount]', 'SAVE20');
  await page.click('text=Apply');
  await expect(page.locator('.total')).toHaveText('$80.00');
});

41 seconds to about 4. Same coverage of the thing the test is named after.

The objection worth taking seriously

"If you don't sign up through the UI, you're not testing signup."

Correct — and you shouldn't be, in a test called applies a discount code at checkout. You should be testing signup in a test called signup. One of them. Running the signup flow sixty times as a side effect doesn't give you sixty times the confidence; it gives you one test's worth of confidence and fifty-nine copies of the runtime.

Keep a small set of full-journey tests that go through the UI end to end, no shortcuts. Five or six, covering your critical paths. Those are your integration canaries. Everything else injects state and tests one thing.

The real cost of injected setup is a different one: you now have test-only API endpoints, and they need to be gated so they can't run in production. That's a genuine tradeoff. In my experience it's worth it, but go in knowing you've taken on that responsibility.

What to do on Monday

Don't refactor the suite. Do this instead:

Run with --reporter=json and sort specs by duration.
Take the slowest five. For each, count the lines before the first meaningful assertion.
Pick the one with the worst ratio and convert just that one to a fixture.
Measure again.

You'll have a real number for your codebase within an afternoon, and a much better basis for arguing about where the remaining time goes.

The suites that stay fast aren't the ones with fewer tests. They're the ones where each test does the minimum work required to reach the thing it's actually asserting.

I'm Ankit Patel, Director of Test Automation at QAble. I spend most of my time inside other teams' test suites. Currently building Testbo-X, a Playwright framework focused on intelligent test selection and parallel CI execution.