Help Me test

Posted on Jul 25 • Originally published at helpmetest.com

Approval Testing Explained: What It Is and How It Works

#javascript #tutorial #testing #programming

What do you do when you can't write an expected value for your assertion? When the output is a complex JSON object, a rendered report, a formatted table, or an entire HTML page—and writing expect(output).toBe(...) would require typing out hundreds of lines?

Approval testing solves this. Instead of specifying expected output upfront, you capture real output, review it, and approve it as correct. Future test runs compare against that approved output. If anything changes, the test fails and shows you exactly what changed.

The Core Concept

Traditional assertion-based testing:

// You write the expected value
expect(calculateTax(100, "CA")).toBe(9.25);

Approval testing:

1. Run the code → get output
2. Review the output → "yes, this looks right"
3. Approve it → save as the "golden master"
4. On future runs: compare output to approved version
5. If different → test fails, shows diff

The key insight: you're reviewing, not pre-specifying. The first approval is a human judgment call. After that, it's automated regression detection.

Why It Exists

Some output is too complex to specify manually. Consider:

A PDF report with 200 fields
A rendered React component tree
A serialized domain object graph
A formatted email template
A generated SQL migration

Writing the full expected value is impractical. But you can look at the output and say "yes, that's correct." Approval testing turns that human judgment into a test.

How It Works in Practice

Step 1: First run (no approved file)

The test runs and produces output. No approved file exists, so the test fails. The framework shows you the output.

FAILED: no approved file found
Received output: 
  { "user": { "id": 1, "name": "Alice", "plan": "pro", "createdAt": "2025-01-15" }, 
    "invoice": { "amount": 100, "currency": "USD", "dueDate": "2025-02-15" } }

Step 2: Review and approve

You look at the output. It's correct. You approve it—this saves the output to a .approved.txt file.

tests/
  __approved__/
    UserInvoiceTest.produces_invoice.approved.txt

The approved file now contains exactly what the test produced.

Step 3: Subsequent runs

The test runs again. Output is compared to the approved file.

Same output: test passes
Different output: test fails with a diff

  { "user": { "id": 1, "name": "Alice", "plan": "pro", 
-   "createdAt": "2025-01-15"
+   "createdAt": "2025-01-15T00:00:00Z"
  }, ...

You see exactly what changed. Now you decide: is this an intentional change (approve the new output) or a regression (fix the code)?

A Complete Example

Using the approval-tests library for JavaScript:

import { verify } from "approvals/lib/Providers/Jest/JestApprovals";

function generateUserSummary(user, orders) {
  return {
    user: { id: user.id, name: user.name, memberSince: user.createdAt },
    orderCount: orders.length,
    totalSpent: orders.reduce((sum, o) => sum + o.amount, 0),
    lastOrder: orders[orders.length - 1]?.createdAt ?? null
  };
}

it("generates user summary", () => {
  const user = { id: 1, name: "Alice", createdAt: "2024-01-01" };
  const orders = [
    { id: 101, amount: 49.99, createdAt: "2025-01-10" },
    { id: 102, amount: 99.99, createdAt: "2025-01-20" }
  ];

  const summary = generateUserSummary(user, orders);

  // First run: fails, shows output, you approve it
  // Subsequent runs: compares to approved output
  verify(JSON.stringify(summary, null, 2));
});

On the first run, the test fails and opens a diff tool showing you the output. You review it, close the diff tool accepting the changes, and the approved file is created. Every subsequent run compares automatically.

The Approval Workflow

          ┌──────────┐
          │ Run Test │
          └────┬─────┘
               │
               ▼
         Output exists?
         /           \
        No            Yes
        │              │
        ▼              ▼
   Show output    Compare to
   → Approve?     approved file
        │              │
       Yes         Match?
        │          /     \
        ▼         Yes     No
   Save to        │        │
   approved       ▼        ▼
   file        PASS     Show diff
                        → Fix code
                          or approve

What Gets Approved

Approval tests can capture any string output:

JSON objects: serialized domain objects, API responses
Plain text: formatted reports, emails, logs
HTML: rendered templates, component output
XML: configuration, API responses
SQL: generated queries, migrations
Binary (via base64 or hex representation): images, PDFs (though this is rare)

The output must be deterministic. Random IDs, timestamps, and volatile data must be normalized before approval.

Handling Non-Determinism

The biggest challenge with approval testing is making output deterministic. Random values and timestamps change on every run.

function generateOrder(userId) {
  return {
    id: Math.random().toString(36), // PROBLEM: changes every run
    userId,
    createdAt: new Date().toISOString(), // PROBLEM: changes every run
    status: "pending"
  };
}

Solutions:

// Option 1: Inject deterministic values
function generateOrder(userId, { id = uuid(), createdAt = new Date() } = {}) {
  return { id, userId, createdAt: createdAt.toISOString(), status: "pending" };
}

// In test: inject fixed values
const order = generateOrder(1, { id: "order-001", createdAt: new Date("2025-01-01") });
verify(JSON.stringify(order, null, 2));

// Option 2: Scrub non-deterministic fields before approving
const order = generateOrder(1);
const scrubbed = { ...order, id: "[id]", createdAt: "[timestamp]" };
verify(JSON.stringify(scrubbed, null, 2));

Benefits

You don't need to know the expected output upfront. This is ideal for legacy code, complex algorithms, or systems that are being reverse-engineered.

Regression detection is automatic. Any change to output triggers a failure with a precise diff.

Tests are readable. The approved file is the expected behavior. New developers can read it to understand what the system produces.

Refactoring is safe. If behavior doesn't change, tests stay green. If it does change, you get an immediate, detailed diff.

Limitations

Approved files must be committed to version control
Output must be deterministic (timestamps, IDs need handling)
Initial approval requires human review time
Large approved files can be hard to read in diffs

How HelpMeTest Complements Approval Testing

Approval tests are excellent for verifying internal outputs—generated text, serialized objects, formatted reports. But they don't test the end-to-end user experience.

HelpMeTest runs plain-English tests against your live application—real browser, real flows. When your approval tests confirm the order summary generates correctly, HelpMeTest confirms that the order summary displays correctly to users in the actual UI.

Summary

Approval testing captures real output and uses it as the expected result
You review and approve output on the first run; automation handles subsequent runs
Any change triggers a diff—you decide if it's intentional (approve) or a regression (fix)
Best suited for complex output that's impractical to specify manually
Requires deterministic output—scrub or inject volatile values
Approved files live in version control alongside tests

DEV Community