A Playwright Mailbox Lease for Reliable Tests

Email verification tests often fail for a reason that has nothing to do with the verification code. Two workers read the same inbox, a previous message wins the race, or cleanup happens before the test has captured its evidence. The browser step looks flaky, but the real problem is ownership.

A useful mental model is to treat a test mailbox as a short-lived lease. One test owns it for one scenario, knows when the lease started, and can prove which message belongs to the run. This small change makes Playwright email testing much easier to reason about.

Why email tests become flaky

Imagine a sign-up test that creates an account and waits for a verification email. A simple implementation searches for the newest message. That sounds reasonable until CI runs four copies of the test at once. The newest message might belong to another worker, or a delayed message from the last run.

The same issue appears with a throwaway email or a test-only mailbox. If the address is shared, the test has no reliable identity boundary. It can pass locally and fail only when the suite is busy. The first signal are usually misleading: a timeout in expect(page).toHaveURL() hides the fact that the wrong link was opened.

For a safer design, give each test a unique mailbox address or unique inbox label. Record an ownership token, the creation time, and the test run ID. A mailbox trust boundary is useful here; this mailbox trust boundary explains the same idea for OAuth links.

Define the mailbox lease

A lease only needs a few explicit fields:

type MailboxLease = {
  address: string;
  runId: string;
  createdAt: number;
  expiresAt: number;
};

The runId should be present in the mailbox address when the provider allows it. Otherwise, keep it in a test-side registry. createdAt gives you a lower bound for valid messages, while expiresAt prevents a failed test from holding resources forever.

Before the browser starts, create the lease. After the test finishes, release it in a finally block. This is boring infrastructure, but it save hours of guessing when a pipeline turns red.

Do not use a keyword such as temp mailid as if it were an ownership mechanism. It may describe a mailbox, but it does not prove that a message belongs to this run. A typo or search phrase like tem email should not end up as an assertion rule either.

Implement the Playwright workflow

The important part is not the provider API. It is the sequence of checks around the API:

test('verifies a new account', async ({ page }, testInfo) => {
  const lease = await createMailboxLease(testInfo.workerIndex);

  try {
    await page.goto('/signup');
    await page.getByLabel('Email').fill(lease.address);
    await page.getByRole('button', { name: 'Create account' }).click();

    const message = await waitForMessage({
      address: lease.address,
      after: lease.createdAt,
      subject: /verify your account/i,
      timeoutMs: 30_000,
    });

    expect(message.recipient).toBe(lease.address);
    expect(message.receivedAt).toBeGreaterThanOrEqual(lease.createdAt);

    const verificationUrl = extractVerificationUrl(message.text);
    await page.goto(verificationUrl);
    await expect(page.getByText('Account verified')).toBeVisible();
  } finally {
    await releaseMailboxLease(lease);
  }
});

There are three protections in this example. The recipient must match, the message must be newer than the lease, and the subject must identify the expected workflow. If your email service exposes a message ID, store it in the test report too. It makes a re-run much more useful.

Polling should have a deadline and a visible last observation. Instead of logging every response, record a compact timeline: lease created, message count seen, matching message ID, link extracted, and cleanup result. This is enough to explain most failures without saving the full contents of private emails.

If you already test account lockout or security notifications, the same ownership checks apply to those email alert tests. The assertion changes, but the mailbox contract does not.

Diagnose failures with evidence

When a test fails, classify the failure before changing the timeout:

1. No message: inspect provider delivery time, lease creation, and the application event log.
2. Wrong message: inspect recipient, run ID, and parallel worker allocation.
3. Malformed link: inspect URL extraction and whether the message was plain text or HTML.
4. Expired link: compare the application token lifetime with the polling delay.
5. Cleanup failure: record it separately so it does not hide the original assertion.

Increasing a timeout can cover a slow provider, but it cannot fix cross-test contamination. A test can loose its evidence when cleanup deletes the message too early, so capture the message ID and timestamps before releasing the lease.

A practical CI checklist

Before merging an email-driven Playwright suite, check that:

• every worker gets an isolated address or inbox namespace;
• the test records a run ID and lease start time;
• polling filters by recipient, subject, and creation time;
• token extraction rejects unexpected hosts;
• the report keeps IDs and timestamps, not unnecessary email bodies;
• cleanup runs even after a browser assertion fails;
• expired leases are reclaimed by a scheduled job;
• a dry run can exercise the workflow without sending a real invite.

For teams that need a disposable inbox during manual diagnosis, a service such as tempmailso can be one input to the workflow. Keep it behind the same lease and privacy rules as any other provider; a temporary address is not automatically isolated.

Common questions

Should every test create a new mailbox?

For sign-up, password reset, and ownership tests, usually yes. Reusing a mailbox is acceptable for a read-only parsing test, but it needs a fixed fixture and strict message filtering.

How long should the lease last?

Set it slightly above the test timeout and cleanup window. The exact value depends on the provider. The key is to make expiration explicit instead of leaving abandoned inboxes around.

Is Playwright waiting enough?

No. Playwright can wait for the page, but it cannot know whether an external message belongs to the current scenario. The mailbox lease supplies that missing identity and timing contract.

Reliable email tests are less about clever selectors and more about ownership. Once each message has a test, worker, and time boundary, failures become diagnosable instead of random. The workflow is a little more setup, but the CI signal is much cleaner.