DEV Community

jasonmills94
jasonmills94

Posted on

Isolate Email Tests in Docker CI on AWS

Email verification tests become unreliable when every CI run shares the same inbox. A retry can consume the token from the first attempt, a parallel branch can read another branch's message, and an old message can make a broken flow look healthy.

I have had better results treating the test mailbox like a short-lived infrastructure resource. Each run gets an identity, a lease, and a cleanup rule. The application still sends a normal verification email, but the test knows exactly which inbox belongs to it.

Why email tests leak across CI runs

The usual shortcut is to configure one test address in a repository secret and poll it until a message arrives. It works on a quiet day. Under parallel CI, it creates a race:

build A -> shared inbox -> token A
build B -> shared inbox -> token B
build A reads whichever message arrives first
Enter fullscreen mode Exit fullscreen mode

The problem is not only the inbox. Test state also lives in the application database, the Docker network, and the CI retry history. A reused fake email address can leave an account, token, or delivery event behind. Then the next run starts with facts it did not create.

The first rule is simple: a test run must be able to prove ownership of the message it reads. A run ID in the address, a correlation header, or both is enough for most staging systems. The test is not done when the container starts; its done when the expected message is matched and its evidence is saved.

Give every run an isolated mailbox lease

Create a mailbox lease before starting the application stack. The lease record should contain at least:

{
  "run_id": "github-1842-7f31",
  "address": "verify-github-1842-7f31@example.test",
  "expires_at": "2026-10-30T12:30:00Z",
  "owner": "payments-api",
  "status": "active"
}
Enter fullscreen mode Exit fullscreen mode

The exact provider is less important than the contract. If a test needs a generated throwaway email, the generator should return a unique address and a cleanup deadline together. For short staging checks, a service such as tempmail.so can be one option for inspecting a disposable inbox, but it should not become a hidden dependency in production code.

Keep the lease in the CI job's environment, not in a global configuration file:

export TEST_EMAIL="verify-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}@example.test"
export TEST_RUN_ID="${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
docker compose -f compose.ci.yml up -d --wait
Enter fullscreen mode Exit fullscreen mode

The GITHUB_RUN_ATTEMPT part matters. A retry of the same workflow should not quietly reuse the state created by the failed attempt. It is a small detail, but it saves a lot of confusing debugging later.

A Docker and AWS workflow

In Docker CI, put the application, worker, and email-capture dependency on the same isolated network. Pass only the lease values into the app container. Do not bake a shared mailbox into the image or commit it to a test fixture.

On AWS, the same pattern works with ephemeral task or namespace metadata. For example, an ECS task can receive TEST_RUN_ID and TEST_EMAIL from the job that launched it. A Kubernetes job can use the run ID as part of its namespace and label. The infrastructure name then becomes useful evidence when a test fails.

Add a short receipt after the message is found. This follows the idea behind API smoke tests with receipts: record what was observed, not just that an assertion passed.

{
  "run_id": "github-1842-7f31",
  "message_id": "mail-8841",
  "subject": "Verify your account",
  "received_at": "2026-10-30T12:04:18Z",
  "token_asserted": true
}
Enter fullscreen mode Exit fullscreen mode

Store this receipt with the CI artifacts. When the test fails, you can tell whether the application never sent the message, delivery was delayed, or the poller selected the wrong message. That distinction is more useful than a screenshot of a red assertion.

Failure handling and cleanup

Polling needs a deadline and a clear terminal state. Use a bounded loop, filter by run ID and subject, and mark the result as received, timed_out, or invalid. Never let an unbounded poll keep an AWS task alive forever.

Cleanup should run in a finally step even when the test fails. Delete the mailbox lease, remove the Docker stack, and expire any test account created by the scenario. If the cleanup job fail, the lease expiration is the second safety net. It should be short enough to control cost but long enough for a developer to inspect artifacts.

One warning: avoid putting the full email body into normal application logs. Tokens and reset links are credentials for the duration of the test. Keep a redacted receipt, and restrict access to the CI artifact. This is also why a typo such as tamp mail com should stay in test content only, never in a real customer workflow.

Practical checklist

  • Create a new mailbox lease for every run and retry attempt.
  • Include the run ID in the address, metadata, or both.
  • Run Docker services on an isolated network.
  • Filter messages by ownership, subject, and a bounded time window.
  • Save a redacted delivery receipt as a CI artifact.
  • Clean up in a finally step with lease expiry as a backup.
  • Keep tokens and links out of ordinary logs.
  • Connect infrastructure alerts to the same run ID, like rollout alerts tied to one deploy.

Common questions

Should every test create a new fake email address?

Every isolated run should. Individual assertions inside one run can share a lease when they deliberately test the same account lifecycle. Separate parallel scenarios should not share it.

Is a local email catcher enough?

It is often enough for unit and integration tests. Use a controlled remote inbox when you need to verify provider delivery, DNS, or the same path used by staging. Keep that boundary explicit, so a provider outage does not look like a Docker regression.

What is the most useful first change?

Add a run ID to the mailbox and save one delivery receipt. That small change usually exposes whether the flakiness comes from shared state, slow delivery, or a missing cleanup path.

Reliable email tests are mostly an ownership problem. Give each Docker CI run a mailbox lease, carry its identity into AWS resources, and leave behind enough evidence to replay the failure. The test suite becomes a little more structured, but the failures become much less mysterious.

Top comments (0)