Debug Playwright Email Delays With Timelines

When a Playwright email test fails, the first guess is usually "the inbox was slow again". Sometimes that is true. Just as often, the app queued the message late, the worker retried, or the test started polling before the request that matters had even finished. If you change waits before you understand the sequence, you can make the suite pass for a day and still learn almost nothing.

The workflow that has helped me most is boring in a good way: capture a tiny delivery timeline for one run, then decide which boundary is unstable. It sounds almost too simple, but it turns vague flakiness into something you can act on. For QA teams using Playwright and Automation heavily, this keeps triage grounded in evidence instead of vibes.

Why email delays look random in Playwright

Email checks often hide three separate clocks:

1. the browser action that triggers delivery
2. the app job or provider handoff
3. the inbox polling loop in the test

If those clocks are not visible, one red test can lead to three wrong fixes. I still see suites where the test bumps expect.poll() from 10 seconds to 30 seconds, the failure rate drops a bit, and everyone moves on. Then the same bug comes back next week in CI, just with more waiting and less signal. That is why I now prefer a small timeline log before I touch any timeout, even if it feels slighly slower at first.

This also pairs well with parallel inbox isolation. Isolation stops worker collisions, but it does not explain whether a single worker saw a slow app event, a slow mailbox query, or a broken assertion. You need both pieces.

Capture a delivery timeline before changing waits

My rule is simple: for one failing scenario, write down five moments.

1. user action submitted
2. network response returned
3. app acknowledged the email job
4. inbox polling started
5. matching email found

That list looks obvious, but teams skip it because they want a fix fast. The timeline is what tells you if the test is early, the app is late, or the mailbox API is noisy. Without it, a "slow inbox" story can hide a regression in queue publishing or a missing idempotency key.

I also like putting a run id in the email subject or metadata so the inbox query is narrow. If the test is searching a shared mailbox with only a broad subject filter, you will get messy evidence realy fast. A dedicated scenario id makes the failure easier to read and much easier to replay.

Here is the small helper shape I reuse:

import { expect, Page } from "@playwright/test";

type InboxProbe = {
  startedAt: number;
  foundAt?: number;
  attempts: number;
};

export async function waitForVerificationEmail(
  page: Page,
  fetchMessage: () => Promise<{ subject: string } | null>,
) {
  const marks: Record<string, number> = {
    submitClickedAt: Date.now(),
  };

  await page.getByRole("button", { name: "Create account" }).click();
  marks.responseSettledAt = Date.now();

  const probe: InboxProbe = { startedAt: Date.now(), attempts: 0 };

  const message = await expect
    .poll(async () => {
      probe.attempts += 1;
      const result = await fetchMessage();
      if (result) {
        probe.foundAt = Date.now();
      }
      return result;
    }, { timeout: 20_000, intervals: [500, 1_000, 2_000] })
    .toBeTruthy();

  return { message, marks, probe };
}

The code is not fancy, and that is the point. I want just enough structure to answer two questions after a failure:

• Did the browser finish the triggering step when we thought it did?
• Did the email arrive late, or did our polling begin too soon?

If your app can emit a job-created event or response header, log that too. One extra timestamp can save an hour of guessing.

A small Playwright helper for timeline logging

Once the timing marks exist, I write them to a tiny artifact. A JSON file is enough. The useful part is not the format, it is that the file survives the failed run and can be compared across green and red builds. In CI, I upload it the same way I upload traces and screenshots.

import { writeFile } from "node:fs/promises";

await writeFile(
  "artifacts/email-timeline.json",
  JSON.stringify(
    {
      scenario: "signup-verification",
      marks,
      probe,
      totalMs: (probe.foundAt ?? Date.now()) - marks.submitClickedAt,
    },
    null,
    2,
  ),
);

This is where preflight evidence checks gave me a useful mindset. Before you trust the outcome, make sure the evidence files you need were actually produced. For tests, that means asking "do I have the timeline, the trace, and the request id?" before I accept a flaky failure story.

One practical note: if your staging flow uses a shared inbox provider for throwaway accounts, label the account by scenario and keep the query narrow. I have seen teams debug the wrong message because a dummy e mail account from a previous retry was still visible in the same mailbox window. That kind of mistake feels small, but it can waste a whole mornng.

Checklist for deciding whether the app or the test is wrong

After a couple of runs, I use this checklist:

1. If submit-to-response is unstable, inspect the UI trigger first.
2. If response-to-job-create is unstable, inspect the backend handoff.
3. If job-create-to-found is unstable, inspect the mail provider or inbox filter.
4. If the email is found but the assertion fails, inspect parsing and matching logic.

That sequence keeps the triage calm. It also stops the common "just increase the timeout" reflex, which is usefull in emergencies but rarely a good root-cause fix. It also makes bug reports more consisent, because each failure gets discussed with the same few timestamps.

When I review these failures with teams, the most common surprise is how often the mailbox is innocent. The app retried late, the worker batched jobs, or the test started polling before the save action finished. The timeline makes those cases much less arguable, which is nice when everybody is a bit tired and somtimes a little impatient and just wants the red build gone.

Q&A

Should I always log a timeline for email tests?

Not for every green run. I usually keep the helper available and persist the artifact for CI failures, nightly runs, or newly unstable flows. That gives enough evidence without bloating every job.

What timeout should I start with?

Start from a timeout that matches the system you actually own, then narrow the polling intervals and evidence around it. A random bigger timeout can hide the regression for a bit, but it does not make the suite more reliable.