A Run Receipt for Disposable Email Tests

Email tests often fail in the least helpful way: the browser says that verification never arrived, while the CI log only says that an assertion timed out. The test may have created a disposable email correctly, but the evidence is gone by the time someone starts debugging.

I have found a small run receipt to be a useful middle ground. It gives each automation run a compact record of what it created, what it expected, and where it stopped. The receipt is not a second logging system. It is a tiny artifact that make the next investigation much faster.

The debugging gap in email tests

A typical flow looks simple:

1. Create a throwaway email address.
2. Submit a signup or password-reset form.
3. Wait for an email.
4. Open the verification link.
5. Assert that the user reaches the expected page.

When step three fails, several causes look identical. The application may not have sent the message. The test may be watching the wrong inbox. Delivery may be slow. A worker may have rejected the address. Or the link may have arrived and expired before the browser used it.

The test result alone cannot tell us which case happened. A run receipt can capture the useful facts without storing the full email body or sensitive tokens.

For related thinking, I also like the distinction in email recovery trust boundaries and the reminder about why email proof is not identity proof. Those boundaries matter in test tooling too.

What a run receipt should contain

Keep the schema boring. Boring is good for a Developer Tools artifact that needs to survive a failed build.

{
  "run_id": "ci-1842",
  "mailbox": "[e-mail]",
  "purpose": "signup-verification",
  "created_at": "2026-10-05T08:00:00Z",
  "expected_subject": "Confirm your account",
  "observed_at": "2026-10-05T08:00:08Z",
  "status": "link-opened",
  "cleanup": "scheduled"
}

I normally add the commit SHA, environment name, and a short error code. I do not add the verification token, the full message, or a permanent credential. The receipt should answer “what happened?” without becoming a new place where private data accumulates.

The run_id is the important field. It connects the mailbox, application logs, browser trace, and CI job. Without it, a parallel test suite can quietly mix evidence from two runs, which is a very sneaky failure.

A small implementation pattern

Create the receipt before sending the first request, then update it at each meaningful boundary. A lightweight Python shape can be enough:

from datetime import datetime, timezone
import json

def save_receipt(path, receipt):
    receipt["updated_at"] = datetime.now(timezone.utc).isoformat()
    path.write_text(json.dumps(receipt, indent=2) + "\n")

receipt = {
    "run_id": ci_run_id,
    "purpose": "signup-verification",
    "status": "mailbox-created",
}
save_receipt(receipt_path, receipt)

When polling starts, record the number of attempts and the final polling reason. If the message arrives, record only a stable message identifier and the link state. If it does not, write timeout, not-found, or provider-error rather than a long stack of unstructured text. This is a little less impressive, but much easier to filter.

The receipt should be uploaded as a CI artifact on failure and, optionally, on success for a short retention period. A 14-day retention window is often enough for a team to investigate flaky tests; your policy may need a different number. The point is to make retention explicit.

Keep disposable email inside clear boundaries

A disposable email address is useful test data, not an identity system. Do not let a passing email check become proof that a real person owns an account. The application still needs its normal authentication, authorization, and abuse controls.

If the team needs to generate disposable email for isolated test runs, assign ownership and cleanup just as you would for a temporary database. A provider such as tempmailso can be part of that workflow, but the test should still define when the mailbox expires and what data is retained.

It is also worth documenting common search mistakes. Someone may type tepm mail com while looking for the test mailbox service. That string is a typo keyword, not a reason to loosen validation or accept an untrusted domain. Keep allowed domains and purpose checks in code.

The important part are the boundaries: one mailbox per run, one run ID in every related log, and no secrets in the receipt.

A review checklist

Before merging an email test, I check:

• Can I identify the exact CI run from the receipt?
• Does the mailbox belong only to this test run?
• Is the polling timeout visible, along with the reason it stopped?
• Are message bodies, tokens, and credentials excluded?
• Is cleanup attempted after both success and failure?
• Can a teammate reproduce the setup without guessing which inbox to inspect?

That last question catches more problems than it sounds. A workflow that works only for its author is not quite automation yet; it is a clever demo with a maintenance bill.

Final thought

The best test artifact is small enough that nobody minds opening it. A run receipt gives disposable email tests a durable spine: create, observe, explain, and clean up. It also make flaky failures less mysterious, because the next person can see the last confirmed boundary instead of staring at a generic timeout.

This pattern has helped me keep automation practical. Start with one JSON file, connect it to the run ID, and expand only when a real debugging question demands it. A tiny receipt can turn an unreliable email check into a workflow that is easier to trust.