DEV Community

Cover image for Running 100 Playwright Tests in Parallel Without Inbox Collisions
zerodrop
zerodrop

Posted on

Running 100 Playwright Tests in Parallel Without Inbox Collisions

#playwright #testing #cicd #githubactions

If you've ever tried to run a large Playwright test suite in parallel — the kind that tests email verification flows, magic links, or password resets — you've probably hit this problem:

Two tests run at the same time. Both sign up with the same test email address. Test A waits for a verification email. Test B's email arrives first. Test A reads it. Test B times out. The whole suite goes red, and you spend an hour debugging a race condition that only happens in CI.

This is the inbox collision problem. It's subtle, it's intermittent, and it's completely avoidable.

Why shared inboxes fail in parallel CI

Most email testing tools give you one of two options:

Option 1: A single shared inbox (Mailpit, MailHog, local SMTP)
All tests funnel into the same inbox. The first test that polls gets whatever email arrived most recently — which might belong to a completely different test. In parallel builds, this is a guaranteed race condition.

Option 2: A limited pool of inboxes
Better, but still breaks when your parallel worker count exceeds the inbox pool. A 20-worker matrix build against a 10-inbox limit means half your tests are fighting over shared state.

The real fix is simpler: one isolated inbox per test run, always.

Zero cross-test contamination by default

ZeroDrop generates a new inbox on every call — no shared state, no pools, no configuration:

const mail = new ZeroDrop();
const inbox = mail.generateInbox(); // void-a3k9x@zerodrop-sandbox.online
Enter fullscreen mode Exit fullscreen mode

Each inbox name is a random adjective + 7-character alphanumeric string. The address space is large enough that collision probability across thousands of parallel runs is effectively zero. Every test run gets a cryptographically isolated inbox that no other test can see or contaminate.

This isn't a setting you enable. It's how the system works by default.

Parallel matrix builds in GitHub Actions

Here's a real example — 4 parallel workers, each running a separate auth flow test, each with its own isolated inbox:

name: E2E Auth Tests (Parallel)

on: [push]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        shard: [1, 2, 3, 4]  # Run 4 workers in parallel

    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      # Each worker gets its own isolated inbox
      - name: Generate isolated test inbox
        id: inbox
        uses: zerodrop-dev/create-inbox@v1

      - name: Run Playwright shard
        run: npx playwright test --shard=${{ matrix.shard }}/4
        env:
          TEST_INBOX: ${{ steps.inbox.outputs.inbox }}
Enter fullscreen mode Exit fullscreen mode

Four workers. Four inboxes. Zero collisions. The zerodrop-dev/create-inbox Action runs on each worker independently — no shared state, no coordination required.

What this looks like at scale

The pattern scales linearly. 10 workers, 10 inboxes. 100 workers, 100 inboxes. There's no pool to exhaust, no lock to acquire, no cleanup step between runs.

// In your Playwright test — works identically across all parallel workers
import { test, expect } from '@playwright/test';
import { ZeroDrop } from 'zerodrop-client';

const mail = new ZeroDrop();

// process.env.TEST_INBOX is set per-worker by the GitHub Action
// Falls back to a fresh inbox when running locally
const inbox = process.env.TEST_INBOX ?? mail.generateInbox();

test('email verification flow', async ({ page }) => {
  await page.goto('/signup');
  await page.fill('[name="email"]', inbox);
  await page.click('[type="submit"]');

  // This worker's inbox — no other worker can see this email
  const email = await mail.waitForLatest(inbox, { timeout: 15000 });
  const link = email.body.match(/https?:\/\/\S+verify\S+/)?.[0];
  await page.goto(link);

  await expect(page).toHaveURL('/dashboard');
});
Enter fullscreen mode Exit fullscreen mode

Each worker is completely self-contained. The test doesn't know or care how many other workers are running.

The inbox pool problem at scale

Some email testing tools cap concurrent inboxes on lower-tier plans — commonly 10 per account. If your CI matrix runs more than 10 parallel workers, which is common in enterprise pipelines, you either hit the limit and tests fail, or you upgrade to a higher tier for unlimited inboxes.

ZeroDrop's free tier has no inbox limit. Every test run generates a fresh inbox instantly, at the edge, with no network request during generation. The inbox name is computed locally on the runner — there's nothing to hit a rate limit on.

Inbox isolation properties

Each ZeroDrop inbox is isolated by design:

  • Unique per run — random name generated at test start, never reused
  • Ephemeral — auto-deleted after 30 minutes via Redis TTL
  • Private — only accessible via the exact inbox name; no enumeration API
  • Edge-routed — emails are caught at Cloudflare's global edge, not a central server

The 30-minute TTL means stale test data never accumulates. A test suite that ran 6 hours ago has left zero traces.

Working example

A complete parallel Playwright setup with ZeroDrop, including the GitHub Actions matrix configuration and full auth flow tests:

→ github.com/zerodrop-dev/zerodrop-playwright-example

The example uses a single worker for simplicity, but the pattern scales directly to matrix builds — just add the strategy.matrix block shown above.

Free tier

ZeroDrop's free tier includes unlimited inboxes, the full SDK, and the GitHub Action — no signup required, no credit card, no paywall on the API.

For teams who need custom domains, 7-day retention, and shared API keys:
→ zerodrop.dev

Top comments (0)