Playwright Retries Need Email Receipts

When a Playwright test retries after an email step fails, the second attempt often looks cleaner than the first one. That is exactly why these bugs waste time. The suite passes on retry, everybody moves on, and the real issue stays in the build for another week or two.

The pattern that helped me most was treating every email assertion like a receipt, not just a mailbox search. Before polling the inbox, I want a tiny record of what message should exist, why it should exist, and which run owns it. That sounds a bit strict, maybe even boring, but it makes flaky tests way less mysterious.

Why Playwright retries can hide the wrong email

Retrying a browser step is not wrong by itself. The problem starts when the inbox is shared across attempts and the test only asks, "did an email arrive?" A green result can still be misleading if:

• the email came from the previous attempt
• the right recipient got the wrong template
• a delayed message arrived after the app state had already changed
• the subject matched, but the business event did not

I have seen runs where the retry passed because the first attempt had already queued the email. The suite looked healthy, but the original failure was actualy still there. Once you spot that pattern, generic retries feel less comforting.

This confusion gets worse when test notes contain loose phrases like temp mailid or dummy e mail and nobody can tell whether they are describing the tool, the inbox, or the bug. QA needs cleaner evidence than that.

Define a receipt before polling the inbox

My rule is simple: create an email receipt object as soon as the user action happens. The receipt is not the message body. It is the minimum contract that lets you prove whether the inbox hit belongs to the current attempt.

For a signup flow, the receipt usually contains:

• the normalized recipient address
• the template or event type
• a subject pattern
• the current test attempt
• a trigger timestamp or correlation id

That last part matters more than teams expect. If you can tie the inbox result to an attempt number or server-side event id, a retry becomes much easier to reason about. This is close to the same discipline I like in other inbox contract design workflows: define the boundary first, then let the executor check it.

Store retry evidence beside the trace

Playwright already gives us good browser traces, but the trace alone does not explain whether the email system behaved correctly. I like storing a small JSON artifact next to the trace for each attempt:

type EmailReceipt = {
  recipient: string;
  template: "verify-email" | "password-reset";
  subjectPattern: RegExp;
  attempt: number;
  triggeredAt: number;
  correlationId?: string;
};

After polling, I attach another record with message metadata:

type EmailReceiptResult = {
  receipt: EmailReceipt;
  inboxCount: number;
  matchedMessageId: string | null;
  matchedReceivedAt: string | null;
  matchedSubject: string | null;
};

This artifact does two useful things. First, it tells me whether the retry saw a brand new message or one that was already hanging around. Second, it keeps failure analysis out of vague chat threads. The evidence sits in the run output, nice and plain.

If your team also validates operational notifications, these drift email verification patterns show the same idea from another angle: alerts are easier to trust when each email can be tied back to one concrete event window.

A small helper for retry-safe assertions

The helper does not need to be clever. In fact, clever email helpers usually age badly. I prefer something short:

async function waitForReceiptMatch(receipt: EmailReceipt) {
  const messages = await inbox.poll({
    recipient: receipt.recipient,
    timeoutMs: 20_000
  });

  const match = messages.find((message) => {
    return receipt.subjectPattern.test(message.subject)
      && Date.parse(message.receivedAt) >= receipt.triggeredAt;
  });

  return {
    receipt,
    inboxCount: messages.length,
    matchedMessageId: match?.id ?? null,
    matchedReceivedAt: match?.receivedAt ?? null,
    matchedSubject: match?.subject ?? null
  };
}

Three details matter here:

1. Filter by recipient first, always.
2. Compare against the action timestamp so stale emails cannot quietly win.
3. Return evidence, not just true or false.

That last bit is where lots of flake hides. A boolean pass can be easy to read during a happy path, but it is pretty useless at 2 AM when CI starts wobbling and everyone is guessing.

QA checklist before you increase the timeout

When a retried email check still fails, I go through this list before touching the timeout:

• Did the app record the expected send event for this attempt?
• Does the inbox match include a timestamp after the action fired?
• Did the retry use a fresh receipt, or did it inherit stale state?
• Is the subject pattern too broad for similar templates?
• Are we proving ownership of the message, or just finding any message?

Only after that do I ask whether the timeout is too short. Too many teams jump straight from "flaky" to "wait longer," and that can hide product bugs for ages. A longer timeout is sometimes correct, sure, but it should be supported by delivery behavior you can explain.

Q&A

Should every retry use a new inbox?

Not always. A fresh inbox is nice when the setup cost is low. But even with a shared inbox, you can get reliable checks if each attempt writes a clear receipt and filters aggressively.

What if the provider is a create temporary mail service or a disposable email address generator?

The same receipt pattern still applies. Provider choice changes polling behavior and cleanup, but it should not replace correlation, timestamps, and per-attempt evidence.

Is this overkill for a small QA team?

I do not think so. The receipt object is tiny, and the debugging win is outsized. Once you add it, a lot of "weirdly flaky" email tests become pretty normal, which is what I want from automation tbh.