Playwright: Isolate Email Tests in Parallel CI

Email verification tests often pass on a laptop and fail as soon as CI runs four workers. The usual explanation is “timing,” but timing is only the visible symptom. The deeper problem is shared state: two tests are looking at the same inbox, or one test reads a message created by another.

When you generate a disposable email address for an end-to-end test, treat that address as a test resource with an owner. A good fixture should create it, use it once, collect enough evidence to debug a failure, and clean it up when the test is done. This makes a stable names for Playwright test inboxes useful as part of a larger isolation contract, not just a naming trick.

Why parallel email tests become flaky

Imagine two tests running at the same time:

test A -> signs up [e-mail] -> waits for verification
test B -> signs up [e-mail]   -> waits for verification

If both tests use a shared mailbox, the first unread message is not necessarily the message your test expects. A test may click the wrong link, consume a message before its owner sees it, or pass only because the workers happened to finish in a lucky order.

There is a second failure mode. A test can create an address with a disposable email address generator, but reuse it on retry. The old verification email is still visible, so the retry clicks an expired token and reports a confusing product failure. Parallelism exposes these races more often, but serial execution does not make the design safe.

The isolation contract

Before writing the fixture, define five rules:

1. Every test gets a unique address or inbox label.
2. The test records the address and a correlation value in its report.
3. The reader waits for a message belonging to that address and test run.
4. A retry gets a new address, unless the test is explicitly checking recovery.
5. Cleanup happens after evidence is captured, even when an assertion fails.

The address is not the only identifier. Include a short run ID in the signup name or a supported metadata field. If the service cannot expose metadata, make the local test record precise enough to compare the recipient, subject, and creation time. “I saw an email” is weak evidence; “I saw the verification email for this test owner” is much better.

A Playwright fixture for one inbox per test

The exact API differs between providers, but the fixture shape can stay stable. Keep provider calls behind a small client so the test itself remains readable.

import { test as base } from '@playwright/test';
import { MailClient } from './mail-client';

type Fixtures = {
  testMail: { address: string; waitForCode: () => Promise<string> };
};

export const test = base.extend<Fixtures>({
  testMail: async ({}, use, testInfo) => {
    const mail = new MailClient();
    const owner = `${testInfo.project.name}-${testInfo.workerIndex}-${testInfo.testId}`;
    const inbox = await mail.createInbox(owner);

    try {
      await use({
        address: inbox.address,
        waitForCode: () => mail.waitForVerification(inbox.id, owner),
      });
    } finally {
      await mail.disposeInbox(inbox.id);
    }
  },
});

The finally block matters. Without it, failed CI runs leave test mail behind and make later diagnosis harder. The fixture also avoids hiding provider errors inside a generic timeout, which is a small but important improvement for QA work.

Polling, ownership, and cleanup

Email delivery is asynchronous, so polling is reasonable. Poll for a bounded period with a short interval, and stop when the message matches the owner. Do not select “the newest message” unless the provider guarantees a mailbox dedicated to this test.

Useful matching fields are the recipient, a subject prefix containing the run ID, and a message timestamp after the signup request. Log those fields in the test attachment, but redact message bodies and tokens when they are not needed. For teams that use a replayable run receipts pattern, the mail event belongs in the same receipt as the browser trace and request logs.

If a test needs a link, verify that the link belongs to the current message before clicking it. A stale message should produce a clear “wrong owner” failure, not a generic “verification failed” result. This difference saves time when the product is healthy and the fixture is not.

Some teams search for a temp mail so provider during development. That can be fine for a manual check, but CI still needs an explicit retention and privacy policy. A test address is disposable; the logs should not be.

A CI debugging checklist

When an email test fails in parallel CI, check these in order:

• Was the address unique to the test attempt, not merely to the worker?
• Did the signup request and the inbox message share a correlation value?
• Did the poller filter by recipient and ownership before reading the message?
• Was the test retried with a fresh inbox?
• Did cleanup preserve the message metadata and trace before deleting it?
• Could an old temp org mail phrase or fixture value be confusing a text assertion?
• Does the failure report show the provider response status and elapsed time?

Avoid increasing the timeout first. A longer timeout can hide a race while making the pipeline slower. First prove that each test owns exactly one inbox, then tune polling for the actual delivery behavior.

Final takeaway

Reliable email automation is mostly about boundaries. Give every Playwright test a unique owner, require the reader to prove ownership, and record a small failure receipt before cleanup. Once those rules are in place, parallel CI stops being a source of mysterious inbox races and becomes a useful way to find real product problems.