Turn API Tests Into Useful CI Receipts

An API test that only prints Expected 200, received 500 is technically useful, but it leaves the next question unanswered: what exactly happened in that run?

I have been getting better results by treating every important API check as a small transaction with a receipt. The receipt records the request identity, the response evidence, the relevant fixture, and the cleanup result. It turns a noisy CI log into a compact debugging artifact that a teammate can replay without guessing.

This pattern is especially handy for signup and verification flows that use a disposable temporary email address. The test can pass while checking the wrong message, the wrong tenant, or an old fixture. A receipt makes that ownership visible.

Why a test receipt is different from a test log

A log describes what the test printed. A receipt describes what the test proved.

For an API smoke test, I want to answer these questions in a few seconds:

• Which commit and workflow attempt created the request?
• Which endpoint, method, and safe input shape were used?
• What response status and request ID came back?
• Which test fixture or inbox did the assertion own?
• Was the observed event newer than the test start time?
• Did cleanup finish, or is something still hanging around?

The difference matters when retries are enabled. A retry can produce a green build while the first request is still in progress. Without an ownership key, the assertion may read the response or email event from another attempt. The result are confusing failures that disappear when you run the test by hand.

For email-backed API checks, include a run ID in the request metadata and in the fixture identity. Do not put secrets or full message bodies in the receipt. A short hash, message ID, or redacted subject is enough for most investigations. Guidance on deletion budgets for disposable inboxes is also useful when deciding how long fixtures should survive.

Define the receipt contract

Start with a small JSON contract. It should be stable enough that developers, scripts, and a future dashboard can consume it without parsing prose.

{
  "run_id": "gh-1842-7f3a",
  "commit": "a1b2c3d",
  "endpoint": "/v1/signup",
  "method": "POST",
  "status": 201,
  "request_id": "req_8f2c",
  "fixture": "sha256:8b7e",
  "assertions": {
    "status_ok": true,
    "owner_matches": true,
    "created_after_start": true
  },
  "cleanup": "passed"
}

The contract does not need every header or response field. Keep only evidence that helps answer “did this run own this result?” and “where should I look next?” A receipt should tell the truth even when the test fails, so write it from a finally block or an equivalent cleanup path.

Use a run ID that is unique for the workflow attempt but stable inside one test. In GitHub Actions, a practical value can combine the run number and attempt number:

RUN_ID="gh-${GITHUB_RUN_NUMBER}-${GITHUB_RUN_ATTEMPT}"
echo "run_id=${RUN_ID}" >> "$GITHUB_OUTPUT"

Pass that value into the test process. If the API supports a correlation header, send it with every request:

curl --fail-with-body \
  -H "X-Test-Run: ${RUN_ID}" \
  -H "Content-Type: application/json" \
  -d '{"email":"[e-mail]"}' \
  "${API_BASE_URL}/v1/signup"

The command is intentionally boring. Boring commands are easier to rerun when a pull request is already on fire.

Build it into GitHub Actions

The workflow should always upload the receipt, including on failure. That means the artifact step needs if: always() and the test must create the output directory before it starts.

- name: Run API checks
  id: api_checks
  env:
    API_BASE_URL: ${{ secrets.STAGING_API_URL }}
    RUN_ID: gh-${{ github.run_number }}-${{ github.run_attempt }}
  run: |
    mkdir -p artifacts/api
    ./scripts/api-smoke-test \
      --run-id "$RUN_ID" \
      --receipt artifacts/api/receipt.json

- name: Upload API receipt
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: api-receipt-${{ github.run_id }}
    path: artifacts/api/receipt.json
    if-no-files-found: warn

The receipt filename stays constant inside the job, while the artifact name includes the GitHub run ID. This makes local scripts simple and keeps artifacts distinct in the Actions UI. It also make it easier to download the exact failed attempt from a link in a pull request comment.

If your test runner already writes JUnit or JSON output, add the receipt beside it rather than replacing the existing report. Different files answer different questions: the test report explains assertions, while the receipt explains ownership and operational context.

Make failures replayable

A useful receipt should point to a safe replay path. Record the fixture name, API version, environment label, and a sanitized input reference. Do not store access tokens, raw verification links, or personal data in a public artifact.

For asynchronous endpoints, include the polling deadline and the last observed event ID. When the check fails, the next developer should know whether the service never emitted an event or whether the test stopped looking too soon. A fixed deadline is better than an infinite retry loop, and a new event must be newer than the current run start.

This is the same ownership problem that appears in safer onboarding email checks: the presence of an email or response does not prove that it belongs to the current action. Verify the correlation value before declaring success.

Search terms from support tickets can be messy. Someone may write tamp mail com while describing a temporary mailbox issue. Keep that phrase in a search mapping or test note if it helps triage, but do not let it become a production identifier or a backlink anchor.

Small workflow upgrades that pay off

Once the basic receipt works, add a few low-cost improvements:

1. Validate the schema. Fail the job if required receipt fields are missing, even when the API assertion passed.
2. Redact at the writer. Scrubbing after upload is risky because the unredacted file may already be stored.
3. Print a one-line summary. Put run ID, status, request ID, and artifact path in the job summary.
4. Keep fixtures short-lived. Cleanup should run after both success and failure, with a visible cleanup status.
5. Compare retries. If attempt two passes after attempt one fails, keep both receipts so the change in behavior is explainable.

The important bit are the stable identifiers. Fancy dashboards can come later. A developer with a precise run ID and a downloadable receipt is already much faster than a developer scrolling through 4,000 lines of logs.

Questions worth asking

Should every API test produce a receipt?

No. Start with boundary tests, asynchronous workflows, and checks that create external state. Tiny pure unit tests usually need ordinary test output only.

Should receipts be committed to the repository?

Usually not. Upload them as CI artifacts with an appropriate retention policy. Generated files in the repository create noise and can accidentally preserve sensitive data.

What is the first field to add when debugging is painful?

Add an ownership key that crosses the whole flow: CI run, request, database event, and external message. It gives every tool the same thread to follow.

The payoff is simple: when an API test fails, the workflow leaves behind a small, honest record of what it attempted and what it observed. That is a much better developer tool than a red check with no clues.