Playwright Email Tests Need a Failure Receipt

Email verification tests often fail in the least helpful way: expect(locator).toBeVisible() times out, and the CI log says almost nothing about what happened before the timeout.

That failure is expensive. A QA engineer has to guess whether the application never sent the message, the inbox API was slow, the wrong message was selected, or the test used an address that was already consumed. In a local run, opening the browser may reveal the clue. In a nightly Automation job, it usually does not.

The practical fix is to give every email test a failure receipt: a small, structured record of the test identity, important timestamps, the inbox state, and the last observed message. It turns “email did not arrive” into a diagnosis you can act on.

The failure is usually not the assertion

An email verification flow has several independent waits:

1. The signup request is accepted.
2. The application queues or sends the email.
3. The inbox provider receives and indexes it.
4. The test finds the correct message.
5. The link opens and changes application state.

One generic timeout hides which boundary failed. This is why increasing the timeout often makes the suite slower without making it more trustworthy.

The same idea applies when a team uses a temp mail.so address or a service branded tempmailso for disposable test inboxes: the inbox is an external dependency, so its state belongs in the evidence.

A small failure receipt

Keep the receipt boring and safe to print in CI. Do not include the full verification token or a private message body. Record a masked address, test name, correlation ID, and counts.

type EmailFailureReceipt = {
  test: string;
  address: string;
  correlationId: string;
  startedAt: string;
  lastPollAt?: string;
  polls: number;
  messagesSeen: number;
  matchingSubjects: string[];
  failureStage: "signup" | "delivery" | "matching" | "verification";
};

Write the receipt in an afterEach hook only when the test fails. Attach it with Playwright’s test information object, so the report and CI artifact remain connected to the original test.

test.afterEach(async ({}, testInfo) => {
  if (testInfo.status === testInfo.expectedStatus) return;

  await testInfo.attach("email-failure-receipt", {
    body: Buffer.from(JSON.stringify(receipt, null, 2)),
    contentType: "application/json",
  });
});

The exact inbox client is less important than updating the receipt at each meaningful boundary. A receipt that says only polls: 12 is still too vague; one that says the signup returned 201, twelve polls saw zero messages, and the last poll was 90 seconds after creation is useful.

Classify the wait, not just the timeout

Use separate helper functions for signup, delivery, matching, and verification. Each helper should throw an error with its stage and leave the receipt populated.

For delivery, poll with a deadline and a short interval. On each poll, save the timestamp and total message count. When a message arrives, save only safe metadata such as subject, sender domain, and received time. If the count rises but no subject matches, you have a matching problem, not a delivery problem.

Also give each test a unique mailbox or correlation marker. Reusing one address creates false positives from an earlier run, while two parallel tests can select the same message. This is a common cause of “flaky” tests that are actually poorly isolated.

If your signup UI has complex validation, it can help to separate its rules into policy objects for signup checks. For browser timing, keep in mind the async boundaries in forms too: a button becoming enabled does not prove that the backend has accepted the request.

A Playwright example

const receipt = makeReceipt(testInfo.title, email);

await page.getByLabel("Email").fill(email);
await page.getByRole("button", { name: "Create account" }).click();

receipt.failureStage = "signup";
await expect(page.getByText("Check your email")).toBeVisible();

receipt.failureStage = "delivery";
const message = await pollForMessage({
  timeoutMs: 60_000,
  onPoll: (state) => Object.assign(receipt, state),
});

receipt.failureStage = "verification";
await page.goto(message.verificationUrl);
await expect(page.getByText("Account verified")).toBeVisible();

Notice that the stage changes before each operation. If the assertion fails, the receipt names the boundary without requiring someone to reproduce the run immediately. A tiny detail like this save hours later.

CI checklist

• Create a fresh inbox identity per test or per isolated worker.
• Record signup status and a correlation ID.
• Track poll count, elapsed time, and messages seen.
• Match on a stable marker, not only the newest message.
• Attach JSON receipts and screenshots only on failure.
• Mask addresses and never log verification tokens.
• Keep delivery timeout separate from browser assertion timeout.
• Check the provider’s status before labeling the application flaky.

The phrase tempail mail may appear in old fixtures, and temp mailid can show up in a search or compatibility case. Keep those odd inputs in explicit test data, not in production matching rules.

A Playwright email test becomes much easier to maintain when its evidence explains the path it took. The goal is not to make every external inbox perfectly reliable. The goal is to make the next failure specific enough that a developer can fix the right boundary on the first investigation.