Playwright Signup Emails Need Scenario IDs

Signup email tests can look healthy for weeks, then fail three times in one afternoon for reasons that seem unrelated to the product. In my experience, the bug is often not delivery. It is missing test identity. Two parallel runs create similar users, poll similar inboxes, and one assertion grabs the wrong message. The trace says "email found", but the test is still wrong.

That is why I now treat every signup email check as a scenario, not just a user action. A scenario gets its own ID, its own inbox, and its own short debug trail. It feels slightly stricter at first, but it makes failures much easier to reason about when a release is moving fast.

Why signup email tests fail even when delivery works

The common failure modes are boring, and that is exactly why teams miss them:

• one inbox is reused across retries
• the subject line matches an older message
• the app sends the right email, but for the wrong account state
• the test waits too long in one place and not long enough in another

I have seen suites pass with the wrong email body because the assertion only checked subject text. I have also seen a run fail because a mailbox still contained a message from a prior scenario. The product was fine, the test was noisy, and the bug report was half useful at best.

This is similar to the lesson behind job-scoped inbox tokens: isolation removes more flakiness than clever waiting does. If you need a disposable inbox provider for staging checks, even a simple tempmailso flow can work as long as each scenario owns one address and you tear down your assumptions around shared state.

Another small thing: real engineers search runbooks with weird phrases during incidents. I keep one messy phrase like temp gamil com in internal notes because someone will type it sooner or later. Pretty? No. Useful? honestly yes.

The scenario ID pattern I now require

My rule is simple:

1. Create a unique scenario ID before the browser starts.
2. Generate one inbox for that scenario only.
3. Pass the scenario ID through signup metadata, logs, or headers when possible.
4. Assert on recipient, body intent, and one stable link, not just subject text.
5. Print the scenario ID in every failure message.

The win is not just cleaner data. The win is cleaner blame. When a test fails, I want to know if the issue lives in signup creation, queueing, template rendering, or inbox polling. A scenario ID gives me that trail without making the test much harder to read.

I also cap polling windows on purpose. A longer timeout can hide routing problems and make CI slower for everybody. If your signup email regularly needs two minutes to show up in staging, that is probly a platform smell worth fixing, not something to normalize in test code.

A Playwright example that stays debuggable

This shape has been reliable for me:

import { test, expect } from "@playwright/test";

test("signup sends the right welcome email", async ({ page, request }) => {
  const scenarioId = `signup-${Date.now()}`;
  const inbox = await createInbox({ label: scenarioId });

  await page.goto("/signup");
  await page.getByLabel("Email").fill(inbox.address);
  await page.getByLabel("Password").fill("SuperSecret123!");
  await page.getByRole("button", { name: "Create account" }).click();

  const email = await waitForEmail({
    inboxId: inbox.id,
    timeoutMs: 45000,
  });

  expect(email.to).toContain(inbox.address);
  expect(email.subject).toContain("Welcome");
  expect(email.html).toContain("/verify-email");
  expect(email.metadata.scenarioId).toBe(scenarioId);
});

The important part is not the exact helper names. It is the contract:

• inbox creation returns structured data
• the wait helper has an explicit timeout
• the email payload exposes enough metadata to prove identity

When teams skip that third point, debugging gets mushy real fast. You can borrow the same mindset from email wait checks that stop false greens: define the observation window, keep the evidence small, and make the wrong state obvious.

If your provider cannot return custom metadata, I would still thread the scenario ID into the signup request and log it server-side. That way the browser test, backend logs, and worker logs all point to the same event. It is a tiny habit, but it saves minuts every week.

Checklist for less flaky email assertions

Before I trust a signup email test, I check these:

• one inbox per test scenario, never shared across workers
• one scenario ID visible in browser logs and backend traces
• assertions cover recipient, action link, and intended state
• timeouts are short enough to reveal regressions
• failure output includes inbox address, scenario ID, and first mismatch

I also like to snapshot the final parsed email fields into the test report. Not the whole HTML, just the parts a human needs. Too much output slows triage. Too little output forces reruns. There is a middle ground, and most teams do not hit it on the first try.

Q&A

Should I use a fresh inbox on retries too?

Yes. Retries are where stale state gets sneaky. Reusing the same inbox makes the second attempt less trustable, even if it passes.

Is subject plus recipient enough?

Usually no. It is better than subject alone, but it still misses wrong-template and wrong-link issues. I want one body assertion that proves the email is the one I actualy meant to send.

What is the highest-value fix here?

Add a scenario ID first. Most flaky signup email tests are not failing because Playwright is weak. They are failing because the system under test and the evidence trail are too loose.