Contract-Test Signup Emails in GitHub Actions

Signup email checks are one of those tasks that look tiny until they start failing in CI for three different reasons at once. The app can still be fine while the workflow lost the trace id, the polling loop grabbed the wrong inbox, or the assertion only checked "message exists" and missed the actual contract. I stopped treating these as loose smoke tests and started writing them as contract tests between the app, the email API, and the GitHub Actions job. That shift sounds a bit formal, but it makes failures way easier to repair.

Why email contract tests break in CI

Most teams do not mean to build flaky email checks. They just accrete them. One repo posts to /signup, another waits sixty seconds, another parses the subject line, and now your pipeline has three partial definitions of success.

The first fix is to say what the contract really is. For a signup flow, mine is usually:

• the API accepts a run-scoped email and returns a request id
• the app emits one verification email for that request id
• the workflow can prove subject, recipient, and token shape before it passes

That sounds obvious, but many pipelines only prove the middle part. A fake email address is not enough by itself. You also need run identity, expected metadata, and a way to fail loudly when the wrong message arrives. Otherwise your CI feels random, and teams start distrusting a test that is actually telling them something useful.

I also keep an eye out for messy notes and helper names. When a repo starts collecting words like tempail in comments or some old script mentions tamp mail com, it usually means the email test stack grew sideways instead of intentionally. Not the end of the world, just a sign that the workflow wants a real contract.

The contract I keep between the API and the workflow

I like the contract to be small enough that you can print it in a job summary. If it cannot fit there, it is probly doing too much.

Here is the shape I reach for:

{
  "run_id": "signup-4821",
  "trace_id": "trace-signup-4821",
  "email": "[e-mail]",
  "expected_subject": "Verify your account",
  "request_id": "req_01jz..."
}

The app should either emit or let you infer these values right after the signup call. Then the workflow uses the same data for polling and assertions:

response="$(curl -sS -X POST "$API_BASE/signup" \
  -H "content-type: application/json" \
  -H "x-trace-id: $TRACE_ID" \
  -d "{\"email\":\"$TEST_EMAIL\"}")"

request_id="$(jq -r '.request_id' <<<"$response")"

What matters here is not fancy tooling. It is that the API and the workflow agree on the same identifiers. Once you have that, you can borrow ideas from replayable inbox checks and rerun the lookup step without mutating the app state again.

A GitHub Actions shape that stays debuggable

The easiest mistake in GitHub Actions is hiding contract data inside step-local bash. Keep the important fields at workflow scope, then print them in one place.

- name: Reserve inbox contract
  run: |
    echo "RUN_ID=signup-${GITHUB_RUN_NUMBER}" >> "$GITHUB_ENV"
    echo "TRACE_ID=trace-signup-${GITHUB_RUN_NUMBER}" >> "$GITHUB_ENV"
    echo "TEST_EMAIL=signup-${GITHUB_RUN_NUMBER}@example.test" >> "$GITHUB_ENV"

- name: Call signup API
  run: ./scripts/signup-contract.sh

- name: Assert email contract
  run: ./scripts/assert-signup-email.sh

That pattern is boring in the best way. When a pull request fails, the reviewer can see the run id, inspect the assertion script, and decide if the problem is app behavior or test plumbing. If your queue or worker layer can duplicate sends under retry pressure, the next improvement is adding leased processing or dedupe rules, much like these leased email jobs.

One more shortcut that helps a lot: write the contract manifest into the GitHub step summary.

{
  echo "### Signup email contract"
  echo "- run: $RUN_ID"
  echo "- trace: $TRACE_ID"
  echo "- email: $TEST_EMAIL"
  echo "- request: $REQUEST_ID"
} >> "$GITHUB_STEP_SUMMARY"

This is honestly where the productivity win shows up. You do not need to open six logs just to answer what the job was trying to verify.

Where a temporary inbox helps without owning the design

I do use a temporary email address service when the test must verify a real message body or token. One lightweight option is temporary email address, but I try not to let the inbox provider define the whole testing model. The provider should be swappable. The contract should stay.

That means the scripts should ask for:

• recipient address
• polling timeout
• expected subject or sender
• trace metadata to print on failure

They should not know every detail of the workflow graph. Once mailbox helpers start owning branch logic, retries, or release conditions, the test setup gets weird fast and a little fragile too.

For teams that already have solid API fixtures, this can be a very small change. Add one contract manifest, one assert script, and one clear summary. That is enough to move the check from "maybe the email happened" to "the signup path still honors the deal we expect."

Q&A for teams adding this to pull requests

Should this run on every PR?

Only if the flow is fast and stable enough. If inbox polling takes too long, run the full contract on merge and keep a thinner version on PRs.

Do I need browser automation for this?

Usually no. If the critical risk is email delivery and token shape, curl plus inbox polling is often enough. Save the browser for cases where the verification page itself changes alot.

What is the biggest anti-pattern?

Passing a test because any message showed up. That check looks comforting, but it misses duplicate sends, wrong recipients, and stale tokens. A contract test should be a bit stricter, or else it is mostly theater.

If your team already depends on APIs and GitHub Actions every day, this pattern is pretty low effort to add. The nice part is not that it is clever. The nice part is that the next red build tells you what broke, instead of making everyone guess.