Run-Scoped Mailboxes for GitHub Actions API Tests

Email-dependent API tests often look random when the real problem is shared state. Two GitHub Actions jobs create the same user, one job reads an old verification message, and a retry consumes the only useful link. The test reports a timeout, but the root cause is an inbox that belonged to another run.

I use a run-scoped mailbox contract to make these failures easier to reason about. Each workflow run gets its own disposable address, a short lease, and a small evidence record. The approach is simple enough for a few API tests, but it also scales better than asking every test helper to invent its own cleanup rules.

Why shared inboxes make API tests nondeterministic

A shared test inbox creates three kinds of collisions:

• Identity collisions: two jobs register the same email address and one receives a conflict response.
• Message collisions: a test matches a verification email from a previous run.
• Lifecycle collisions: one cleanup step deletes data while another test is still polling it.

The usual reaction is to increase the polling timeout. That makes the pipeline slower and doesnt fix a stale message. A better boundary is to give every run an address derived from a run ID, then refuse to read messages outside that boundary.

The address does not need to expose the branch name or a user email. A short random suffix, a workflow run identifier, and a provider-managed domain are enough. Store the full address in the test process, but put only a fixture label in CI logs.

On a busy morning, this is usualy the first signal that saves a developer from chasing the wrong job.

Define a mailbox lease for each workflow run

Think of the mailbox as a leased test resource. The lease has four values:

1. run_id: the workflow execution that owns the mailbox.
2. address: the disposable address used by the API test.
3. expires_at: the time after which the fixture is no longer valid.
4. cleanup_state: whether the test released it, timed out, or failed.

The test should always query by run_id or a unique message marker. Subject text alone is too broad. A useful marker can look like ci-verification-${run_id} and should be passed as an API field or a controlled email subject, not hidden in an undocumented helper.

This is also where teams should decide what a disposable address means. A temp mail so service can be useful for manual checks or isolated test fixtures, but it is not a substitute for ownership and retention rules. The API test still needs to prove that the message belongs to the current run.

Implement the fixture contract

Keep the fixture interface small. For example:

type MailboxLease = {
  runId: string;
  address: string;
  marker: string;
  expiresAt: string;
};

async function waitForVerification(
  lease: MailboxLease,
  timeoutMs = 30_000,
): Promise<string> {
  const deadline = Date.now() + timeoutMs;

  while (Date.now() < deadline) {
    const message = await inbox.find({
      to: lease.address,
      subjectContains: lease.marker,
    });

    if (message) return extractVerificationUrl(message);
    await delay(1_000);
  }

  throw new Error(`No message for run ${lease.runId} before timeout`);
}

The important detail is not the polling function. It is the contract around it: the mailbox address and marker are created together, and the failure names the run. If a test uses a fake value such as temp gamil com or temp org mail during a manual experiment, keep that value out of production fixtures and logs so it doesnt become a confusing search term later.

For an overview of keeping auth email state aligned, the same principle applies on the client side: the address, request, and verification state should describe one attempt. The backend and UI should not each guess which email is current.

Wire it into GitHub Actions

Create the lease before the test command and pass only the minimum values through the environment:

- name: Create mailbox lease
  id: mailbox
  run: node scripts/create-mailbox-lease.mjs > mailbox.json

- name: Run API tests
  env:
    MAILBOX_LEASE_FILE: mailbox.json
  run: npm test -- --runInBand

- name: Upload test evidence
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: api-mailbox-evidence-${{ github.run_id }}
    path: artifacts/api-mailbox
    retention-days: 7

- name: Release mailbox lease
  if: always()
  run: node scripts/release-mailbox-lease.mjs mailbox.json

The lease step should fail closed if it cannot create a unique fixture. Do not silently fall back to a team inbox. That fallback is convenient for one green run and painful when a parallel job starts.

Keep the ownership rule seperate from the provider client. The test should know who owns the fixture, even if the mail adapter changes later.

For teams comparing the best throwaway email option for a test address, the practical decision is less about a catchy provider name and more about whether the fixture can be isolated, queried, and expired predictably.

Clean up without hiding failures

Cleanup should run even when the test fails, but its result must not erase the original error. Save a small JSON receipt before release:

{
  "run_id": "1842",
  "fixture": "signup-verification",
  "messages_seen": 1,
  "test_status": "failed",
  "cleanup_status": "released"
}

Keep message bodies out of the artifact unless a specific review requires them. A message ID, subject hash, timestamp, and redacted error usually gives enough context. This makes the cleanup is easier to audit and reduces the chance that a real address or token survives in CI storage.

If the provider cannot release a mailbox immediately, mark the lease expired and let a scheduled janitor remove it later. Never reassign an unexpired lease to another run. Reuse is the shortcut that turns one stale message into a hard-to-reproduce incident.

The result are easier to review when expiry and release appear as explicit states instead of hidden cleanup behavior.

Common questions

Should every API test create a new mailbox?

Not always. A test suite can share one lease when its cases are deliberately sequential and use distinct markers. Parallel tests should get separate leases, especially when they create users or consume one-time verification URLs.

Is a disposable address enough for privacy?

No. It limits exposure of personal mail, but logs, artifacts, provider retention, and application databases still need rules. Record fixture IDs instead of full addresses wherever possible.

How long should a lease live?

Long enough for the slowest supported CI path plus a small buffer. The exact duration depends on the provider and test environment. The key is to make expiration explicit and to report it as a fixture failure, not as an unexplained API timeout.

A small rollout checklist

Start with one flaky email API test:

1. Generate a unique lease per workflow run.
2. Add a run marker to every message lookup.
3. Record status, message ID, and redacted timing evidence.
4. Upload evidence with if: always().
5. Release or expire the lease in a final step.
6. Verify two concurrent jobs cannot read each other's messages.

The before-and-after improvement is straightforward: a failed test now says which run owned the mailbox, which marker was expected, and whether cleanup completed. That is a much better starting point than “email timed out again.” It also makes isolated inbox checks in CI a reusable engineering pattern instead of a one-off workaround.