Replayable API Smoke Tests in GitHub Actions

An API smoke test should answer one quick question: does the most important path still work? In practice, teams often get a green check without knowing which environment ran, which test data it used, or what to do when an email arrives late. The check passes once, then becomes hard to replay.

I have found a small run contract makes these workflows much more useful. Give each run an explicit input, a bounded wait, and an evidence bundle. That turns GitHub Actions from a button that says “probably okay” into a developer tool you can inspect and re-run.

Why smoke tests become hard to replay

The first version is usualy simple: install dependencies, call an endpoint, and exit non-zero on failure. The trouble starts when the test depends on a temporary account, an email link, or a shared staging record. A retry may use stale data. Two jobs may read the same inbox. A timeout gets reported as a generic assertion failure.

Before changing the test code, write down four values:

• Run ID: unique for the workflow attempt.
• Environment: the API base URL and deployment revision.
• Test subject: a disposable user or isolated fixture.
• Deadline: the maximum time allowed for asynchronous work.

These values are small, but they make a log searchable. They also stop a developer from guessing which account a failed check touched. For OAuth-heavy systems, safer boundaries for OAuth email flows are a useful companion to this approach.

Give every check a run contract

Pass the contract through environment variables or a checked-in config file. Keep secrets in GitHub Actions secrets, while ordinary run metadata can be visible in the job summary.

env:
  API_BASE_URL: https://staging.example.com
  SMOKE_RUN_ID: ${{ github.run_id }}-${{ github.run_attempt }}
  POLL_DEADLINE_SECONDS: "45"

The test should create unique data with SMOKE_RUN_ID, then clean it up when the API allows that. If cleanup is unsafe, tag the fixture for a scheduled janitor job. Avoid using the current timestamp alone; parallel jobs can still collide, and timestamps are not very nice to search.

For a verification flow, a temp mail inbox can be appropriate for an isolated smoke test. Treat it as test infrastructure, though, not as a shortcut around production trust controls. Also document the exact service and retention behavior. A note like “tamp mail com” in an old runbook is not enough to identify a dependency, and “tepm mail com” can send an investigator searching in the wrong place.

Build a useful GitHub Actions workflow

Make the workflow runnable on both pushes and manual dispatch. Manual dispatch is the fastest way to reproduce a staging failure without editing code.

name: API smoke

on:
  push:
    branches: [main]
  workflow_dispatch:

jobs:
  smoke:
    runs-on: ubuntu-latest
    timeout-minutes: 5
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - run: npm run smoke -- --run-id "$SMOKE_RUN_ID"
      - if: always()
        uses: actions/upload-artifact@v4
        with:
          name: smoke-evidence-${{ github.run_id }}
          path: artifacts/smoke/

The always() condition matters. Without it, the most valuable files disappear exactly when the check fails. Keep the artifact small: request and response metadata, step timings, correlation IDs, and a sanitized error. Never upload access tokens or full email contents by accident.

Keep email checks isolated

Asynchronous email checks need a poll loop with a deadline, not an unbounded sleep. Poll at a modest interval, record each attempt, and distinguish “message not received yet” from “provider rejected the request.” The test can then report whether the failure is in the API, delivery path, or test dependency.

If the UI also exercises this path, async boundaries in signup forms explains why the client should expose a pending state rather than pretending the request completed. The same idea applies to CI: pending is a state with a deadline, not a reason to hide progress.

Store evidence for the next failure

Write one machine-readable receipt per run:

{
  "run_id": "1842-1",
  "environment": "staging",
  "status": "failed",
  "stage": "verification_email",
  "elapsed_ms": 47120,
  "retryable": true
}

A stable receipt lets a later script summarize failures across runs. It also supports a practical retry policy: retry delivery timeouts, but do not retry a 401, schema mismatch, or deterministic assertion. This saves minutes during incident triage and avoids creating duplicate test accounts.

A compact review checklist

• Can a developer start the test with workflow_dispatch?
• Is every fixture unique to the run?
• Is the asynchronous wait bounded and observable?
• Does failure upload sanitized evidence?
• Are retryable and permanent errors distinct?
• Can the job identify its API revision and environment?
• Are external test dependencies documented and isolated?

Final thoughts

The biggest productivity win is not a faster smoke test. It is a smoke test that explains itself when it fails. A run ID, a deadline, and a small evidence receipt are enough to make many API checks replayable. Add those pieces first, then tune parallelism or polling only after the workflow tells you where the time goes.