Playwright Email Tests Need an Inbox Lease

Email verification tests often fail in a way that looks random. The browser clicks the right button, the application sends a message, and then the test waits. Sometimes the link arrives. Sometimes an older link is read. Sometimes the inbox is empty even though delivery was only a little slow.

The test are often blamed on Playwright timing, but the deeper problem is usually ownership. Which test owns this inbox? Which message belongs to this run? When may the inbox be reused?

An inbox lease is a small agreement that answers those questions. It gives one test run an address for a limited period, records the expected message, and expires the address after the run. This makes a use and throw email address behave like a test fixture, instead of an untracked shared dependency.

The hidden problem with a passing email test

A shared test address can make a suite appear healthy while hiding several defects:

• A previous run leaves a verification message behind.
• Two workers poll the same inbox and consume each other's evidence.
• A delayed message arrives after the test has already timed out.
• A retry receives a link created for a different user or session.
• A mailbox provider returns messages in an order the test did not expect.

These failures are hard to reproduce because the test log usually records only “email not found.” It does not record the inbox identity, message age, delivery request, or last polling attempt. The failure has no story.

The lease gives it one. In practise, a lease can be represented by five values:

type InboxLease = {
  address: string;
  runId: string;
  createdAt: string;
  expiresAt: string;
  expectedSubject: string;
};

The runId is more important than it first looks. It lets the application and the test report connect a signup, a delivery request, and a message to one run.

What an inbox lease records

Before opening the browser, create a lease with a unique address and a short expiry. The exact address format depends on the email test service; the important part is that the address is not silently shared.

For local experiments, teams sometimes use a disposable address service such as tempmail.so as a contextual test input. Treat that address as test data, not as a place for real accounts or sensitive information. A dummy e mail can be useful for a happy-path check, but it should not become a permanent fixture in a production-like environment.

Store the lease in the test report before the first navigation. If the test fails, you then know whether the message was sent to the expected address and whether the lease was already expired.

It is also worth doing a privacy review for staging email tests. The test may be synthetic, but the inbox can still expose tokens, names, or environment details.

A Playwright implementation

The browser test should wait for a message that belongs to the current lease, not merely for any message to appear.

import { test, expect } from '@playwright/test';

test('verifies a new account with its leased inbox', async ({ page }, testInfo) => {
  const lease = await inboxes.create({
    runId: testInfo.testId,
    ttlSeconds: 180,
  });

  await page.goto('/signup');
  await page.getByLabel('Email').fill(lease.address);
  await page.getByRole('button', { name: 'Create account' }).click();

  const message = await inboxes.waitForMessage({
    address: lease.address,
    subject: lease.expectedSubject,
    after: lease.createdAt,
    timeoutMs: 45_000,
    pollEveryMs: 2_000,
  });

  expect(message.receivedAt).toBeGreaterThan(lease.createdAt);
  await page.goto(message.verificationUrl);
  await expect(page.getByText('Your email is verified')).toBeVisible();

  await inboxes.expire(lease);
});

There are three useful guards here. The address narrows ownership, the subject narrows intent, and after prevents an old message from satisfying the assertion. The check are simple, but together they remove a surprising amount of flakiness.

Always expire the lease in a finally block in the real helper. If the browser assertion fails, cleanup should still happen. Keep the raw message metadata in CI for a limited retention period, while redacting the token and full message body.

Diagnosing the common failure modes

When the wait times out, inspect the lease evidence in this order:

1. Was the send request accepted? Record its request ID and response status.
2. Was the message sent to the leased address? A UI may show success while the backend used a stale profile value.
3. Did the message arrive after the lease was created? Compare timestamps from the provider and the test runner.
4. Did parsing fail? Log the subject and message ID, but never the verification token.
5. Did the browser use the same session? A verification link should be tied to the intended user flow.

For OAuth-style flows, it helps to define trust boundaries around email links before writing the assertion. A test that only proves “a link opened” may miss that the link could be replayed in the wrong session.

A lease does not fix slow delivery. It makes slow delivery observable. If the provider takes longer than the timeout, the report should say “delivery exceeded budget,” not “Playwright is flaky.” This is a bit more clear for the person fixing the pipeline.

A small CI checklist

Before merging an email test, check that it:

• creates a fresh address or an isolated mailbox per worker;
• records a run ID, creation time, expiry, and expected message;
• filters messages by address, subject, and creation time;
• has one explicit delivery timeout and a visible polling interval;
• captures safe metadata when it fails;
• expires the lease even after an assertion failure;
• avoids putting real user data in the inbox;
• distinguishes delivery failure from link or session failure.

Do not increase the timeout first. That can hide a reliablity problem and make the suite slower for everyone. First determine which part of the contract failed, then change the contract or the dependency with evidence.

Final takeaway

An email test is not only a browser interaction. It is a small distributed workflow involving a test runner, an application, a delivery system, and a mailbox. Giving the mailbox a short-lived lease makes ownership explicit and failures easier to explain.

Once each run has its own address, timestamp, and cleanup path, the test becomes less mysterious. It can still fail, but it will fail with useful evidence. That is the difference between a flaky check and a QA signal you can trust.