Trace Email Failures in Playwright

Email tests are annoying in a very specific way: the build fails, everybody stares at the timeout, and nobody can tell whether the app failed to send, the inbox API lagged, or the test simply looked at the wrong message. In QA, that is the moment where a flaky check starts costing more than the bug it was meant to catch.

What helped me most was treating email verification as an evidence problem. I still use Playwright waits and assertions, of course, but I no longer trust a red test unless it leaves behind enough proof to explain itself. Once the trace, inbox data, and app logs line up, the failure gets much easier to fix. It sounds fussy, maybe, but it saves hours later.

Why email failures stay hard to explain

Most teams already capture one artifact, not three. They may keep the Playwright trace, or they may dump inbox JSON, or they may check backend logs. Looking at only one source is where the story gets muddy.

An email test can fail while the UI is correct.
It can also pass while the email flow is wrong.
That second case is the nasty one, because it teaches the team to trust a weak signal.

I usually see the same pattern:

1. The test creates an account.
2. It waits for any email that looks close enough.
3. The timeout fires.
4. The retry or rerun hides what actualy happened.

If you have ever seen a staging note mention temp org mail and nobody can tell which inbox belonged to which run, you have already felt this problem. The issue is not just waiting longer. The issue is that the run did not produce evidence with a stable identity.

The three artifacts I want from one failed run

When a signup or reset flow fails, I want these three things from the same attempt:

• a Playwright trace showing what the user did and when
• an inbox snapshot showing which messages existed for that exact mailbox
• an app-side delivery id or job id that ties the send attempt to logs

Any one of those helps. All three together are what makes the review quick instead of hand-wavey.

The Playwright trace viewer is especially useful here because it timestamps each action and network event. That lets me compare "user clicked Create account at 10:03:12" with "provider accepted email at 10:03:13" and "mailbox exposed message at 10:03:19". When those timestamps drift, the cause is usualy pretty visible.

A trace-first Playwright pattern

I like to create one mailbox identity per attempt, save it in the trace context, and attach inbox state on failure. The exact helper names do not matter much. The contract does.

import { test, expect } from "@playwright/test";

test("signup sends one verification email", async ({ page }, testInfo) => {
  const runId = `${testInfo.parallelIndex}-${testInfo.retry}-${Date.now()}`;
  const inbox = `signup-${runId}@example.test`;
  const startedAt = new Date().toISOString();

  await testInfo.attach("email-context", {
    body: JSON.stringify({ runId, inbox, startedAt }, null, 2),
    contentType: "application/json",
  });

  await page.goto("/signup");
  await page.getByLabel("Email").fill(inbox);
  await page.getByLabel("Password").fill("Str0ngPass!123");
  await page.getByRole("button", { name: "Create account" }).click();

  await expect(page.getByText(inbox)).toBeVisible();

  const message = await waitForVerificationEmail({
    recipient: inbox,
    subjectIncludes: "Verify your account",
    afterIso: startedAt,
  });

  expect(message.recipient).toBe(inbox);
  expect(message.html).toContain("/verify?token=");
});

This is basic on purpose. A lot of flaky suites get more complicated before they get more precise. I would rather start with one clear run id, one inbox, and one time window. From there, the trace gives you the UI sequence and the mailbox gives you the external result.

That is also why I like ideas similar to inbox evidence without guesswork. Even though that post is about a different stack, the review principle is the same: make the run observable enough that another engineer can reconstruct what happened without asking the original author.

Where a throwaway inbox fits

Teams ask me fairly often what the best throwaway email setup is for E2E tests. My answer is a bit boring: the best one is the setup that keeps inboxes isolated, queryable, and easy to correlate with logs. Fancy features matter less than evidence discipline.

A throwaway email address is useful because it gives each run a clean external boundary. You can validate subject lines, delivery ordering, and final links without mixing in real inbox history. If you use a hosted service like tempmailso, treat it as a verification edge, not the only debugging source. I still want provider events or queue logs from the app side. Otherwise the inbox tool ends up carrying too much responsibility, and the failure review gets mushy real fast.

This separation also keeps your test design closer to audit-friendly auth email checks. Validation belongs at the boundary. Diagnosis needs internal evidence too. When those jobs stay seperate, false positives drop a lot.

A QA checklist for failure reviews

Before I call an email workflow reliable, I want this checklist to be true:

• each retry gets a fresh mailbox or alias
• the mailbox name appears in the trace or attached metadata
• inbox polling filters by recipient and creation time
• the app records a delivery id, queue id, or message id
• failures attach the latest inbox snapshot instead of only a timeout
• reviewers can tell in a minute whether the problem was send, delay, or matching

None of this is glamorous. It is just the kind of boring structure that makes automation easier to trust. And honestly, trust is the whole game in QA.

Q&A

Do I need both traces and inbox dumps?

For small projects, maybe not every single time. But once tests run in parallel or CI infrastructure gets noisy, having both is worth it. One shows user intent, the other shows delivery reality.

Should I retry the whole test?

Only after the mailbox identity is isolated. Retrying a shared inbox test can produce a green build for the wrong reason, which is probly worse than a loud failure.

What is the first thing to fix if failures feel random?

Usually mailbox reuse. If two runs can see the same inbox history, your debugging story is already compromised before Playwright even opens the page.