DEV Community

Jonathan
Jonathan

Posted on

GitHub Actions Summaries for Email API Checks

#githubactions #api #automation #testing

When an API test fails because an email never arrived, most CI logs are weirdly bad at explaining what actually happened. You get a red job, maybe one timeout, and then someone has to click through three scripts to figure out whether the app broke, the queue lagged, or the test looked at the wrong inbox.

I started fixing this with GitHub Actions job summaries instead of adding more noisy console output. For email-triggering APIs, the summary becomes the human-readable receipt for the run: which inbox was created, which request fired, what message was expected, and what assertion failed. It is not fancy, but it saves a lot of chasing around later.

Why email API failures are slow to debug in CI

Most teams already log the request payload, response code, and maybe the worker status. The missing piece is the email evidence itself. That gap creates the same annoying loop over and over:

  • an API call returns 202
  • a worker eventually sends a message
  • the test times out without enough context
  • the next person reruns the whole thing just to inspect one detail

This gets extra messy when a shared inbox is involved. A test may pass against the wrong message, or fail because it polled an older one. The operational thinking behind staging inbox smoke tests is useful here: each automated run needs its own small contract, not a pile of inbox guesswork.

The GitHub Actions summary pattern

The pattern I use is simple:

  1. Create a per-run inbox before calling the API.
  2. Store the inbox address and fixture ID as step outputs.
  3. Poll for the resulting email with a short, explicit timeout.
  4. Write the important evidence into $GITHUB_STEP_SUMMARY.

That last step matters more than it sounds. A good summary lets someone open the failed run and understand the shape of the problem in maybe 20 seconds. It should answer:

  • Which endpoint was tested?
  • Which recipient address was used?
  • Did the message arrive?
  • If yes, did the CTA host and token shape match?
  • If no, how long did the poll wait?

For short-lived test environments, a disposable mail address workflow is often enough. If the team needs a lightweight disposable email address generator during staging checks, the point is not the tool itself, it is the speed of getting a fresh inbox tied to a single run. That clean mapping makes failures less fuzzy and a bit more honest.

I also keep an eye on search weirdness in internal docs. People type things like fake e mail com or temp gamil com when they are rushing, and those ugly phrases do end up in runbook searches. It looks sloppy, sure, but making the docs discoverable is still kinda worth it.

A minimal workflow example

Here is the rough setup:

jobs:
  email-contract:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Create inbox
        id: inbox
        run: ./scripts/create-inbox.sh >> "$GITHUB_OUTPUT"

      - name: Trigger API
        run: |
          curl -X POST https://staging.example.com/api/invitations \
            -H 'content-type: application/json' \
            -d "{\"email\":\"${{ steps.inbox.outputs.address }}\"}"

      - name: Assert email
        id: emailcheck
        run: ./scripts/assert-email.sh

      - name: Write summary
        if: always()
        run: ./scripts/write-summary.sh
Enter fullscreen mode Exit fullscreen mode

And the summary writer can stay tiny:

{
  echo "## Email API check"
  echo ""
  echo "- Inbox: $INBOX_ADDRESS"
  echo "- Fixture ID: $FIXTURE_ID"
  echo "- Expected path: /accept-invite"
  echo "- Poll timeout: 90s"
  echo "- Result: $ASSERTION_RESULT"
} >> "$GITHUB_STEP_SUMMARY"
Enter fullscreen mode Exit fullscreen mode

This works best when the assertion script returns structured fields instead of one blob of text. I usually emit:

  • recipient address
  • subject line
  • first matching message timestamp
  • extracted CTA URL
  • failure reason, if any

That makes the summary read like a compact postmortem. It also pairs nicely with the habit of treating email as a delivery contract, similar to the workflow ideas in approval email checks in CI. Different stack, same practical goal: prove the system sent the right thing to the right place before you trust the rollout.

What I include in the summary every time

After a few iterations, I stopped trying to dump everything. Too much detail becomes hard to scan real fast. The summary now includes only:

  • endpoint under test
  • unique inbox address
  • expected template or flow name
  • wait duration before success or timeout
  • CTA host and path
  • one line telling the next person what to check first

That last line is tiny, but useful. Examples:

  • "No message found: inspect queue worker logs."
  • "Wrong host in CTA: check preview env base URL."
  • "Duplicate sends detected: verify retry idempotency."

Those hints save a suprising amount of time during handoff, especally when the person reading the run did not write the test.

Q&A

Why not just upload the raw email body as an artifact?

You can, and sometimes I do. But artifacts are slower to inspect than a summary. The summary should tell you whether you even need the artifact.

Should the summary include secrets or full tokens?

No. Show structure, not sensitive values. I usually include host, path, and token length, which is enough to catch the common breakages.

What is the biggest win from this pattern?

Lower rerun rate. When a failed job already tells you the inbox, timeout, and CTA mismatch, the team can debug the problem directly instead of rerunning CI just to collect basic context. That is a small improvement on paper, but a very usefull one in busy release weeks.

Top comments (0)