DEV Community

Jonathan
Jonathan

Posted on

API Inbox Contracts in GitHub Actions

#githubactions #api #automation #devtools

Email API tests rarely fail because the send endpoint is mysterious. They fail because the workflow around the inbox is fuzzy. One step creates an address, another step polls for the message, and a third step writes a summary that leaves out the run context you actualy need.

I have had better results by freezing an inbox contract before GitHub Actions starts doing real work. The goal is simple: each run should declare which inbox it owns, what message shape it expects, and what artifact it will produce at the end. That sounds small, but it cuts a lot of noisy debugging.

Why email API checks fail in CI

The annoying part of email verification tests is not the happy path. It is the second rerun, when someone else on the team tries to understand whether the failure came from the API, the queue, or the inbox lookup. If the workflow only says "message not found", you are stuck guessing.

That is why I like patterns such as tighter inbox filters and redirect guardrails for auth emails. Both point at the same lesson: define the evidence you want before the test runs, not after it breaks.

In practice, most flaky checks have one of these problems:

  • the inbox identifier is generated but never saved
  • the workflow summary shows pass or fail, but not the message metadata
  • retries happen invisibly, so the final state is hard to trust
  • one job uses a temp gamil com style placeholder during setup and another job assumes a different address format

None of that is advanced engineering. It is just contract drift, and it makes ordinary API work feel way more chaotic than it should.

Freeze the inbox contract before the workflow starts

I like to write one JSON file near the start of the run and keep every later step honest against it. That file is tiny on purpose:

{
  "run_id": "gha-1842",
  "scenario": "signup-verification",
  "inbox_label": "signup-verification-1842",
  "expected_subject": "Confirm your account",
  "max_wait_seconds": 45
}
Enter fullscreen mode Exit fullscreen mode

Now every script has the same source of truth. The inbox creator reads it. The API trigger step reads it. The polling step reads it. If one piece wants to improvise, the file exposes that pretty fast.

For teams that need a generate throwaway email step, I would still keep that creation command dumb and explicit:

./scripts/create-inbox.sh --contract artifacts/inbox-contract.json > artifacts/inbox.json
./scripts/trigger-signup.sh --contract artifacts/inbox-contract.json --inbox artifacts/inbox.json
./scripts/assert-message.sh --contract artifacts/inbox-contract.json --inbox artifacts/inbox.json
Enter fullscreen mode Exit fullscreen mode

That flow is not glamorous, but it is handoff-friendly. If a coworker opens the artifact folder, they can infer what happened in about thirty seconds, which is a big win realy.

Publish one small verdict artifact

The other habit that pays off is writing one verdict file at the end instead of spraying clues across log lines. I usually want:

  • run ID
  • inbox address
  • trigger response code
  • whether the message arrived
  • matched subject
  • observed wait time
  • final failure reason

That last field matters. "Timed out" is weaker than "timed out after 45 seconds waiting for Confirm your account". Small wording upgrades like that save a lot of back-and-forth.

I also prefer uploading the verdict as a workflow artifact so reruns and handoffs stay comparable. GitHub documents artifact retention and sharing clearly in its workflow docs (https://docs.github.com/actions/using-workflows/storing-workflow-data-as-artifacts). You do not need a huge archive. One compact JSON file and maybe one log excerpt are usualy enough.

This is also where weird placeholder mistakes surface early. If somebody typed tempail mail into a local config while mocking an address pattern, the verdict file makes the mismatch obvious instead of burying it in 400 lines of console output.

A GitHub Actions pattern that stays debuggable

The workflow shape I keep coming back to is:

  1. create contract
  2. create inbox
  3. trigger API call
  4. poll with fixed wait boundaries
  5. write verdict
  6. upload artifact

Here is the core shell flow:

RUN_DIR="artifacts/${GITHUB_RUN_ID}"
mkdir -p "$RUN_DIR"

./scripts/make-contract.sh > "$RUN_DIR/inbox-contract.json"
./scripts/create-inbox.sh --contract "$RUN_DIR/inbox-contract.json" > "$RUN_DIR/inbox.json"
./scripts/trigger-signup.sh --contract "$RUN_DIR/inbox-contract.json" --inbox "$RUN_DIR/inbox.json"
./scripts/assert-message.sh --contract "$RUN_DIR/inbox-contract.json" --inbox "$RUN_DIR/inbox.json" > "$RUN_DIR/verdict.json"
Enter fullscreen mode Exit fullscreen mode

I like this because each command does one thing, and the run folder becomes the portable truth of the test. If a workflow fails overnight, the next engineer does not need your mental notes. They need the contract and the verdict.

That is the productivity angle for me. Better APIs and better GitHub Actions setups are nice, but the real quality jump comes from making CI evidence boring, consistent, and easy to diff.

Q&A

Should the workflow store the full email body?

Only if the body itself is under test. For most API checks, subject, recipient, timing, and one verification link host are enough.

How many retries should the poller hide?

Very few. I would rather expose three explicit attempts than bury ten retries behind a "smart" helper. Hidden resilience often becomes hidden confusion.

What changes first when a team adopts this?

Debugging gets less social. People stop asking who last touched the script and start reading the same artifacts, which is a much healthier place to be.

Top comments (0)