Playwright Email Tests Need Receipt Logs

When an email assertion fails in Playwright, the first bad habit is to rerun the suite and hope the problem goes away. I used to see this a lot in signup and password reset checks. The UI step looked fine, the backend event looked fine, and still the inbox assertion was fuzzy. In most cases, the missing piece was not a bigger timeout. It was a receipt log that proved what the test asked for and what the inbox actually returned.

This matters even more when your test setup uses a disposable email account for fast isolation. The mailbox is only one part of the system. If the test does not log the trigger time, the expected subject, and the messages it inspected, you end up with a flaky result and almost no trail. That is were teams start guessing.

Why email assertions fail without receipt evidence

Playwright is good at showing browser evidence. You can capture a trace, a screenshot, and console output pretty easliy. Email checks often lag behind. Teams poll an inbox, find nothing, and throw a generic timeout error. That error says the test failed, but it does not explain why.

The failure usually sits in one of four buckets:

• the app never emitted the email event
• the event was emitted, but the wrong inbox was queried
• the message arrived after the test budget expired
• the message arrived, but the assertion matched too loosely or too late

Without a receipt log, all four can look the same. I like to connect email checks with versioned email events in backend flows, because a versioned event plus a receipt log gives QA a cleaner chain of evidence from UI action to inbox result.

What I put in a receipt log

A receipt log should be tiny, boring, and consistent. If it becomes a giant debug blob, nobody reads it. Mine usualy includes:

• scenario id
• worker id
• normalized recipient
• expected subject fragment
• trigger timestamp
• polling deadline
• message ids inspected
• final match id or rejection reason

That structure gives me enough to answer the annoying follow-up questions quickly. Did the suite look in the right inbox? Did the message land after the click? Did a stale email get scanned first? If someone mentions temp org mail in a bug note, I still want the receipt log to say which scenario owned that inbox and what the poller saw.

I also keep receipt logs close to the Playwright artifacts. If the trace and the inbox data live together, triage is much faster and a bit less chaotic.

A Playwright helper that records the right facts

The helper does not need to know everything about your mail provider. It just needs to log the contract clearly and return a structured result. That is the part many teams skip, then regret later.

Here is a compact pattern:

type ReceiptLog = {
  scenarioId: string;
  workerId: number;
  recipient: string;
  subjectIncludes: string;
  triggeredAt: number;
  deadlineAt: number;
  inspectedIds: string[];
  matchedId: string | null;
  outcome: "matched" | "timeout" | "rejected";
};

export async function waitForEmailReceipt(inbox, claim): Promise<ReceiptLog> {
  const receipt: ReceiptLog = {
    scenarioId: claim.scenarioId,
    workerId: claim.workerId,
    recipient: claim.recipient.toLowerCase(),
    subjectIncludes: claim.subjectIncludes,
    triggeredAt: Date.now(),
    deadlineAt: Date.now() + 15000,
    inspectedIds: [],
    matchedId: null,
    outcome: "timeout"
  };

  while (Date.now() < receipt.deadlineAt) {
    const messages = await inbox.list({ recipient: receipt.recipient });

    for (const message of messages) {
      receipt.inspectedIds.push(message.id);
      if (
        message.subject.includes(receipt.subjectIncludes) &&
        Date.parse(message.receivedAt) >= receipt.triggeredAt
      ) {
        receipt.matchedId = message.id;
        receipt.outcome = "matched";
        return receipt;
      }
    }

    await new Promise((resolve) => setTimeout(resolve, 1000));
  }

  return receipt;
}

Two practical notes:

1. Set triggeredAt right before or right after the action that causes the email. Do not set it ten lines earlier.
2. Write the receipt log even on success. Teams often keep only failure evidence, which sounds fine untill the one success that should have failed slips through.

This pattern also pairs well with expiry ownership rules for disposable inboxes. Once inbox ownership is defined, the receipt log becomes the proof that the ownership rule was followed.

How receipt logs shorten failure triage

The big win is not prettier logs. The win is faster elimination. If a failed run shows zero inspected messages, I look at the trigger path first. If it shows ten inspected messages and all were older than triggeredAt, I look at cleanup and inbox reuse. If it shows one close match with the wrong subject, I look at template drift or test data.

That kind of triage makes reruns less tempting. It also makes bug reports better. Instead of saying "email step failed again," QA can say "scenario signup-admin queried the right inbox, inspected three messages, and all were older than the trigger." That is much more usefull for the engineer who has to fix it.

There is also a trust benefit. When your Automation suite records how it made a decision, other teams stop treating email checks like magic. They can inspect the evidence and see the test was strict enough, or not strict enough yet.

Checklist before you rerun the suite

Before I rerun a flaky email test, I check these:

• Does the receipt log store both triggeredAt and deadlineAt?
• Does the test keep message ids that were inspected?
• Can I tell whether the inbox was empty or just mismatched?
• Is the recipient normalized before polling?
• Is the receipt artifact attached near the Playwright trace?

If two or more answers are no, I fix the evidence first. A rerun may pass, but it wont teach you much.

Q&A

Should every test save a receipt log?

For email-dependent scenarios, yes. The payload can stay small, so the overhead is minor and the debugging payoff is real.

Is this only for Playwright?

No. The idea works anywhere, but Playwright teams benefit fast because they already think in steps, traces, and artifacts.

Do I still need separate inboxes?

Sometimes yes, sometimes no. A solid receipt log does not replace isolation, but it makes shared or reused inboxes way less mysterious when things go sideways.