API Contract Tests Need a GitHub Actions Receipt

An API test that ends with a green check is useful, but it is not always reviewable. When a contract test fails only in CI, the next question is usually not “did it fail?” It is “which request, fixture, and response caused the failure?”

I have started treating every API check in GitHub Actions as a small delivery with a receipt. The receipt is a compact artifact containing the endpoint, contract version, response class, and links to the relevant logs. It turns a transient workflow result into evidence that another developer can inspect later.

The green check that explains nothing

Many workflows report only a test command and its exit code:

- name: Run API tests
  run: npm test -- api

This is fine while the suite is small. As the number of APIs grows, it becomes hard to tell whether the failure came from an expired fixture, an unexpected status code, a changed response field, or a dependency that was not ready yet. The check is technically correct, but the feedback loop is slow.

The fix is not to print every response body into the job log. That can leak tokens, customer-shaped data, and alot of irrelevant noise. Instead, define what each contract test promises and save only the information needed to verify that promise.

Define a small contract for every API check

Before adding more assertions, write down the observable contract. A useful contract might look like this:

name: create_invoice
request: POST /v1/invoices
fixture: invoice-minimal.json
expected_status: 201
required_fields: id, status, created_at
max_duration_ms: 800

This gives the test and the receipt the same vocabulary. It also makes failures easier to classify. A 401 means authentication setup, a 422 means request validation, and a missing created_at field means the response shape changed. Those are different fixes, even when they all produce exit code 1.

For email-related endpoints, fixture ownership matters just as much. This guide on a better fixture contract for CI is a good companion because it treats test data as an explicit interface instead of an invisible helper.

I keep the contract close to the test, usually as JSON or TypeScript data. The file does not need to be clever. It needs to be easy to review when an endpoint changes.

Build a reviewable GitHub Actions receipt

The workflow can collect a small JSON receipt after the test command completes, including when the command fails. That last part is important: a failure without its evidence is the moment when you need the artifact most.

- name: Run API contract tests
  id: contract_tests
  continue-on-error: true
  run: npm run test:contract -- --reporter=json --outputFile=artifacts/api-results.json

- name: Build test receipt
  if: always()
  run: node scripts/build-api-receipt.mjs

- name: Upload API receipt
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: api-receipt-${{ github.run_id }}
    path: artifacts/api-receipt.json

- name: Fail workflow when contract tests fail
  if: steps.contract_tests.outcome == 'failure'
  run: exit 1

The receipt builder can summarize each check without copying sensitive payloads:

{
  "run_id": "1842",
  "commit": "abc1234",
  "checks": [
    {"name": "create_invoice", "status": "passed", "http_status": 201, "duration_ms": 214}
  ]
}

The continue-on-error step is deliberate. It lets the workflow create evidence before the final step restores the failing status. Without it, the job may stop before the receipt exists. A tiny bit of extra YAML saves alot of clicking through reruns.

Keep fixtures and secrets out of the log

A receipt should help someone debug, not create a security incident. Redact authorization headers, cookies, full email addresses, and response fields that can contain personal data. Store a fixture name and a hash when you need to prove which input was used.

Avoid putting search terms such as tempmail so, tp mail so, or tempail mail into production-like fixture values unless the test is specifically checking search or text handling. They can make an artifact look like a real user record and confuse later analysis.

For signup APIs, the failure path deserves the same attention as the happy path. When an automated check sees a blocked or delayed email, designing an appeal path for signup email blocks helps separate product recovery from test retries. A retry is not a user-facing explanation.

Before and after workflow

The old workflow answers: “Did the command exit successfully?” The receipt-based workflow answers several more useful questions:

• Which contract failed?
• What status and response shape did it observe?
• Which commit and workflow run produced the result?
• Where is the sanitized artifact?
• Was the failure a request, dependency, fixture, or contract problem?

That extra context changes code review. A pull request can show a stable contract receipt rather than a screenshot of a green job. It also makes flaky failures less mysterious, because duration and dependency state are visible in one place. The setup is a little more work, but the payoff is quick.

Quick questions

Should every response body be uploaded?

No. Save structured summaries by default. Keep full bodies only in a protected debugging run, with redaction and a short retention period.

Does the receipt replace test logs?

No. Logs are detailed investigation material; the receipt is the index. Link the two with the workflow run ID and a stable check name.

What is the smallest useful first step?

Add one artifact with status, HTTP code, duration, commit, and fixture hash. Once that is reliable, add richer classifications or dashboards.

A CI result should be more than a traffic light. With a small contract and a durable GitHub Actions receipt, API failures become much faster to understand and much easier to fix.