Playwright Email Tests: Diagnose the Real Failure

#qa #testing #automation #playwright

Email verification tests often fail with the least useful message in the suite: “verification email not found.” That sentence describes an outcome, not a cause. The inbox may be slow, the test may be watching the wrong mailbox, the application may never have queued the message, or the selector may be too broad.

When a QA team treats all of these as one failure, retries hide the signal. A better approach is to make the test explain which boundary broke. This is especially important when a temporary inbox is used for an isolated signup. A temp gamil com typo in a fixture or a stale temp mailid can look like a provider outage, but the evidence usually tells a different story.

The symptom is not the diagnosis

An email flow crosses several systems:

The browser submits the signup form.
The application accepts the request and queues a message.
The mail service delivers the message to the test inbox.
The test reads the message and follows its link or code.

Each boundary needs a separate observation. Record the user ID or request ID created by the signup, the mailbox identity, the time polling started, and the message identifier once it arrives. This small evidence trail makes a flaky test much easier to reproduce.

For a useful companion on OTP evidence, see better evidence for OTP tests. For CI-friendly reporting, GitHub Actions email summaries are a good next step.

A failure taxonomy for email checks

I use four categories when triaging these failures:

Submission failure: the form did not submit, returned a validation error, or received a 5xx response.
Queue failure: the application accepted the signup, but no email job was created.
Delivery failure: the job exists, but the message has not reached the inbox before the deadline.
Read or assertion failure: the message arrived, but parsing, filtering, the link, or the expected state is wrong.

The distinction matters. Increasing the polling timeout may help delivery latency. It cannot repair a queue failure. Likewise, a more specific subject filter may fix a read failure, but it will not help if the test accidentally created two accounts with the same email.

Capture evidence at every boundary

Playwright can attach a compact diagnostic record when a step fails. Keep sensitive message content out of normal logs, but record safe metadata such as status codes, message IDs, and timestamps.

const startedAt = Date.now();
const response = await page.waitForResponse(
  r => r.url().endsWith('/api/signup') && r.request().method() === 'POST'
);

await test.info().attach('signup-meta', {
  body: JSON.stringify({
    status: response.status(),
    elapsedMs: Date.now() - startedAt,
    mailbox: testMailbox.address,
  }),
  contentType: 'application/json',
});

Avoid attaching the full email unless the test environment has an approved retention policy. A screenshot of the final browser state, the API response status, and inbox metadata are often enough. It is a bit boring, but boring evidence saves hours later.

Use bounded polling in Playwright

Fixed sleeps are tempting because they make a local test pass. They also make every run wait the same amount, whether the email arrived immediately or never arrives. Use bounded polling instead: poll at a modest interval, stop at a clear deadline, and include the last observed state in the error.

async function waitForVerificationEmail(
  readInbox: () => Promise<{ id: string; subject: string } | null>,
  timeoutMs = 30_000,
) {
  const deadline = Date.now() + timeoutMs;
  let lastSubject = 'none';

  while (Date.now() < deadline) {
    const message = await readInbox();
    if (message) return message;
    await new Promise(resolve => setTimeout(resolve, 750));
    lastSubject = message?.subject ?? lastSubject;
  }

  throw new Error(`Verification email timed out; last subject: ${lastSubject}`);
}

The example keeps the timeout explicit and the error actionable. In production, also filter by a run-scoped recipient or token. A broad “latest email” query can pass while reading a message from another test, which is a quiet and dangerous false positive.

A practical QA checklist

Before retrying a failed test, check:

Did the signup request return the expected status?
Did the response contain a request or user ID you can correlate?
Was a unique mailbox created for this run?
Did the email job exist, and when was it queued?
Did polling use a deadline instead of an unbounded loop?
Was the message filtered by recipient, subject, and run token?
Does the failure report say submission, queue, delivery, or read?
Can another engineer replay the check from the attached metadata?

If these answers are visible, Playwright email tests become diagnostic tools instead of random alarms. The test may still fail—networks and providers are imperfect—but the next fix is clear, and the retry decision is based on evidence. That is the difference between a flaky check and a maintainable QA workflow.

DEV Community