Playwright Email Tests Need a Clock Contract

A practical Playwright pattern for making email verification tests deterministic across local runs and CI.

Email verification tests often look flaky for mysterious reasons. A user is created, the application sends a message, Playwright opens an inbox, and the test waits for a link. Sometimes it passes in 12 seconds. Sometimes it times out after two minutes. The failure looks like an email problem, but the real bug is often a missing clock contract.

This is the contract between the test and the mailbox: when a message becomes observable, how long it remains valid, which run owns it, and what evidence proves that it is the right message. Without those rules, a burner email or throwaway email is just another shared asynchronous dependency.

The naming is messy in real bug reports: one team may write “tempail mail” and another “temp org mail.” The label matters less than making the mailbox behavior explicit.

The flaky test is usually a clock problem

Consider a common flow:

1. Create a unique account.
2. Submit the signup form.
3. Poll a test mailbox.
4. Click the newest verification link.
5. Assert that the account is active.

The test can fail at every boundary. The application may queue the email after the HTTP response. The inbox may expose the message before its body is fully available. A previous retry may leave an older message in the same mailbox. CI may start another worker with the same fixture name. Then a short timeout fails even though the system is healthy, or a loose selector clicks the wrong link.

I used to treat these as separate waiting problems. They are easier to diagnose when treated as one lifecycle: reserve, send, observe, consume, and expire. The clock starts when the fixture is reserved, not when the assertion happens.

Define the email clock contract

Before writing a Playwright helper, write down four timestamps or states:

• Reserved: the test owns the mailbox and records a run ID.
• Requested: the application accepted the signup or verification request.
• Observed: the expected message is available with a stable identifier.
• Consumed: the link was used and the account state changed.

The fixture should also define an expiry time. If an email arrives after the lease expires, the test should report an expired fixture, not silently reuse it. This distinction gives useful informations during triage: a slow system and an incorrectly isolated test are different failures. The test don't need to guess which one happened.

A small event schema helps. This small schema for reliable email fixtures is a useful reference for recording message ownership, timestamps, and delivery state. You do not need a large database; a JSON record per test run can be enough.

Build a lease-aware Playwright fixture

The fixture needs one owner, one deadline, and one polling function. Keep the mailbox details behind a small interface so a local fake inbox and a CI service expose the same behavior.

import { test as base, expect } from '@playwright/test';

type MailFixture = {
  address: string;
  waitForVerification: () => Promise<string>;
  release: () => Promise<void>;
};

export const test = base.extend<{ mail: MailFixture }>({
  mail: async ({}, use, testInfo) => {
    const lease = await reserveMailbox(`${testInfo.project.name}-${testInfo.testId}`);
    try {
      await use({
        address: lease.address,
        waitForVerification: () => waitForMessage(lease, {
          subject: 'Verify your account',
          runId: lease.runId,
          deadline: lease.expiresAt,
        }),
        release: () => releaseMailbox(lease),
      });
    } finally {
      await releaseMailbox(lease);
    }
  },
});

test('verifies a new account', async ({ page, mail }) => {
  await page.goto('/signup');
  await page.getByLabel('Email').fill(mail.address);
  await page.getByRole('button', { name: 'Create account' }).click();

  const link = await mail.waitForVerification();
  await page.goto(link);
  await expect(page.getByText('Account verified')).toBeVisible();
});

The example keeps cleanup in finally, which matters when an assertion fails. The message query should filter by the run ID or recipient, then by a known event such as the verification subject. “Newest message” alone is not ownership. It can pass on a laptop and fail in CI when a retry is running beside it. The mailbox are shared infrastructure, not a private variable.

Make assertions about evidence, not hope

A useful polling error should answer three questions: what was searched, how long it waited, and what messages were found. Include the mailbox lease ID, message IDs, and the last observed state in the test attachment. Avoid logging full email bodies or tokens; a redacted subject and message hash are usually enough.

Polling should have a deadline and a deliberate interval. A short interval makes a suite noisy and can overload the inbox API. A long interval hides the actual delivery delay. Start with a bounded interval, then tune it from observed CI timings instead of guessing. Also check that the link belongs to the expected account before navigating to it.

If a test is re-runed, it must receive a new lease or an explicit new attempt ID. Reusing the old address saves a few seconds but makes the result hard to trust. The same rule applies to parallel workers: every worker gets a unique ownership key.

A CI checklist for stable email tests

Before calling the test reliable, check these items:

• The mailbox address includes a unique run or worker identifier.
• The lease expiry is shorter than the overall test timeout.
• The application request and email event have separate timestamps.
• Polling filters by recipient, run ID, and message type.
• Cleanup runs after both passing and failing tests.
• The report contains redacted evidence, not verification tokens.
• A late message is reported as late, not accepted by a later test.
• Retries create a new attempt record.

For larger suites, keep the test-run metadata beside the deployment metadata. A small deploy ledger shows the same useful idea from a rollback perspective: a compact record makes a distributed process easier to explain after the fact. Email tests benefit from that boring paper trail too.

Closing thought

The goal is not to make email instantaneous. The goal is to make every wait explainable. Give the fixture an owner, a deadline, and a message identity; then make Playwright assert on evidence from that contract.

Once the clock is explicit, a timeout stops being a random failure. It tells you whether delivery was late, the wrong message was selected, the lease was reused, or the verification state never changed. That is the kind of failure a QA engineer can fix, and it makes the whole suite more stable then another blind retry ever will.