Playwright Email Tests Need a Quarantine Queue

Email verification tests often look like browser tests, but the hardest part is usually the boundary between the application and the inbox. The browser clicks Send verification email, while a separate service delivers a message, indexes it, and exposes a link. If the test treats that whole path as one instant operation, a failure can look random even when the cause is repeatable.

In QA automation, I prefer to treat every email test as a small investigation. The test should have an isolated inbox, a deadline, and enough evidence to answer one question: did the message never arrive, arrive late, or arrive with the wrong state? A quarantine queue is a simple way to preserve that distinction without letting one suspicious run poison the rest of the suite.

The failure is usually a race, not a random browser problem

Consider this sequence:

1. Playwright submits the registration form.
2. The API accepts the request and queues an email.
3. A worker renders and sends the message.
4. The inbox service receives and indexes it.
5. The browser opens the verification link.

The test can only observe the edges of this process. A fixed waitForTimeout(2000) hides the uncertainty, but it does not remove it. On a fast laptop it wastes time; in a busy CI environment it can be too short. That is why the same test may pass locally and fail in an enviroment with slower workers.

The first useful change is to give each run a unique address or mailbox namespace. A burner email can be appropriate for a controlled test system, as long as the address is isolated, disposable, and never used for real customer data. Do not share one inbox across parallel workers: an old verification link can make a new test appear healthy.

What belongs in the quarantine queue

Quarantine is not a second pass that reruns every red test forever. It is a bounded holding area for failures where the application result is not yet known. I normally record:

• the Playwright test title and commit SHA;
• the worker and browser project;
• the run-scoped inbox identifier;
• the request time and the first time a matching message was seen;
• the message subject, provider status, and extracted link host;
• a screenshot, trace, and relevant server correlation ID.

The evidence should not contain the full email body by default. Store a hash, selected headers, and the minimum redacted excerpt needed to diagnose the problem. If a test searches for a phrase typed by a user, remember mistakes such as tepm mail com or tem email; keep those values as test inputs, not failure labels.

A Playwright fixture with an evidence trail

The fixture below shows the important shape. The inbox client is deliberately abstract.

import { test as base, expect } from '@playwright/test';

type Mail = { id: string; receivedAt: number; text: string };

type Fixtures = {
  runId: string;
  waitForVerification: () => Promise<Mail>;
};

export const test = base.extend<Fixtures>({
  runId: async ({}, use, testInfo) => {
    await use(`${testInfo.workerIndex}-${testInfo.testId}-${Date.now()}`);
  },

  waitForVerification: async ({ runId }, use, testInfo) => {
    const startedAt = Date.now();
    const waitForMessage = async (): Promise<Mail> => {
      // Replace with a bounded poll against the test inbox service.
      throw new Error(`No matching message for ${runId}`);
    };

    await use(async () => {
      try {
        const mail = await waitForMessage();
        expect(mail.receivedAt).toBeGreaterThanOrEqual(startedAt);
        return mail;
      } catch (error) {
        await testInfo.attach('mail-context', {
          body: JSON.stringify({ runId, startedAt }),
          contentType: 'application/json',
        });
        throw error;
      }
    });
  },
});

The real poller should use a deadline and a backoff, not an unbounded loop. It should also filter by runId before checking the subject. That ordering matters becuase two messages can have the same subject while only one belongs to this test.

For a more formal boundary, the ideas in typed contracts for email tests are useful: make invalid states difficult to pass into the next step. The fixture can return a VerifiedMessage only after the timestamp, recipient, and link checks succeed.

How to classify and replay a failure

When a test fails, ask what the last confirmed state was:

• No send request: investigate the product flow or the browser action.
• Send accepted, no message: inspect the worker, provider, and inbox boundary.
• Message found, wrong run ID: investigate data isolation and cleanup.
• Correct message, bad link: treat it as an application or template failure.
• Correct link, browser failure: inspect navigation, authentication, or the target page.

Only the second and third categories are good candidates for a bounded quarantine retry. Replay with the same evidence and a new run ID. Never replay by reusing the original mailbox, because that can turn a stale message into a false pass. If the retry succeeds, keep both attempts visible; a green retry is a signal about reliability, not proof that the first failure was harmless.

When the test uses an external address for a documented scenario, use a natural label such as temp mail for facebook only in the test-data notes, not as a substitute for a production identity. The test should verify the product's behavior and cleanup policy, not encourage a real account workflow.

Practical checklist

Before merging an email test, check:

• Is the inbox unique per worker and run?
• Does the message query require a run identifier and a fresh timestamp?
• Is polling bounded and observable?
• Are traces and screenshots attached only when useful?
• Can the failure be classified from the saved evidence?
• Does quarantine expire and avoid automatic retry loops?
• Is cleanup attempted even when the browser step fails?

A test that fails with a causal timeline is much cheaper to maintain than one that merely says “verification email not found.” The goal is not to make Playwright tolerate every delay. It is to make the next failure tell the team where the delay or state mismatch began.

Questions worth asking

Should I retry every missing email?

No. Retry only when the evidence says the request was accepted and the missing message may be a transient delivery problem. A missing send request is a product failure and should remain red.

Is a disposable mailbox always safe for testing?

No. Keep it restricted to non-sensitive data, understand who can access the inbox, and delete messages after the run. The isolation contract matters more than the label “disposable.”