Email tests often fail with an unhelpful sentence: “expected message to arrive, but it did not.” That is a timeout, not a diagnosis. The message might have been rejected, delivered to the wrong fixture, delayed by a queue, or found by the test but filtered out by a subject assertion.
The most useful change I have made to these tests is small: assign a correlation ID to every test email, then carry it through the application event, delivery request, inbox query, and assertion logs. A fake email address can isolate the recipient, but the correlation ID explains which message the test was actually looking for.
The symptom is a timeout, not the diagnosis
A typical Playwright flow creates a user, triggers verification, and polls an inbox. When it fails, the report usually contains the test name and a final URL. It does not show the message ID, the last inbox query, or whether the application emitted an email event at all.
That leaves several possible causes mixed together:
- The application never created the email event.
- A worker accepted the event but did not deliver it.
- The message went to a different test recipient.
- The inbox API returned a temporary error.
- The message arrived, but the subject or link assertion was wrong.
These are different defects. They should not all be called “flaky email.”
Give every test a correlation ID
Create an ID at the beginning of the test and put it in data that is safe to inspect: the recipient alias, an application metadata field, or a non-sensitive header. Do not put tokens or private customer data into a subject just to make debugging easier.
const correlationId = `pw-${test.info().parallelIndex}-${Date.now()}`;
const recipient = `qa+${correlationId}@example.test`;
await page.getByLabel("Email").fill(recipient);
await page.getByRole("button", { name: "Create account" }).click();
The important part is consistency. The same ID should appear in the browser test log, server-side event log, delivery provider record, and inbox query. If one layer drops it, that gap is useful evidence by itself.
For a short-lived staging inbox, a free temporary email address can keep test traffic away from personal mailboxes. Treat that inbox as a test dependency with a defined retention window, not as a hidden place to store credentials.
Poll for evidence, not just presence
Unbounded sleeps are easy to write and hard to trust. A five-second wait may be too short on a busy CI runner and wasteful on a fast local run. Instead, poll with a deadline and record what each attempt found.
async function waitForMessage(fetchMessages: () => Promise<Message[]>) {
const deadline = Date.now() + 30_000;
let attempts = 0;
while (Date.now() < deadline) {
attempts += 1;
const messages = await fetchMessages();
const match = messages.find((message) =>
message.to.includes(correlationId) && message.subject.includes("Verify")
);
if (match) return { match, attempts };
await new Promise((resolve) => setTimeout(resolve, 1_000));
}
throw new Error(`No verification email for ${correlationId}`);
}
In real code, pass correlationId into the helper instead of relying on an outer variable. The example keeps the shape visible. Log the attempt count, response status, message IDs, and the newest message timestamp. Do not log the full body when it can contain a verification token.
If your team uses tempmailso for this kind of fixture, record the inbox operation and its response status in the same evidence stream. A delayed inbox response is a different failure from an empty, healthy response.
Build a failure bundle
When the assertion fails, save a compact bundle with:
- The correlation ID and test name.
- The recipient alias, with secrets removed.
- The application event status and event ID.
- The delivery or inbox query status for each poll.
- The message IDs and subjects that were returned.
- A screenshot and trace link for the browser step.
This is where receipt logs in Playwright email tests become valuable: they let you compare what the test expected with what the mail path acknowledged. Keep the bundle small enough to attach to a CI job. Large raw inbox dumps are noisy, and they can expose data that should never leave the test environment.
Also check the fixture contract before changing retry counts. A contract-first email fixture should define which fields are stable, how long a message may take, and what a cleanup operation guarantees.
A practical Playwright pattern
Use three separate assertions rather than one giant wait:
- Emission: the application created an event for the correlation ID.
- Receipt: an inbox or provider returned a matching message.
- Content: the message contains the expected link and human-readable text.
This split makes failures readable. If emission fails, inspect application code. If receipt fails, inspect the worker, provider, or inbox. If content fails, inspect templates or URL construction.
A test run can even mark the first two states as diagnostic attachments before the final content assertion. That way, a failed content check still proves the email was sent and received. It saves alot of time during triage, specially when failures only happen in CI.
Do not confuse search input with fixture identity. Test data reviews should cover malformed user-entered terms such as tamp mail com and temp org mail, but those strings should never become a bypass for recipient matching. Normalize search fields deliberately and keep correlation IDs exact.
Checklist for CI
- Generate one correlation ID per test, including retries.
- Carry it through application, delivery, and inbox logs.
- Poll to a bounded deadline and capture every response status.
- Assert emission, receipt, and content as seperate diagnostic steps.
- Redact verification tokens and message bodies from artifacts.
- Delete or expire inbox fixtures after the run.
- Review the failure bundle before increasing timeouts.
Reliable email automation is not about waiting longer. It is about making the path observable enough that a timeout points to the broken boundary. Once every test message has an identity, Playwright can tell you what failed, not just that something failed.
Top comments (0)