Rizwan Saleem

Posted on Jun 4

A Practical Guide to Property-Based Testing for Web APIs

#frontend #typescript #webdev

A Practical Guide to Property-Based Testing for Web APIs

Property-based testing (PBT) is a powerful approach that shifts your mindset from verifying single inputs to asserting general properties across a wide range of inputs. Rather than writing dozens of example-based tests that cover specific cases, you describe invariants your API must uphold and let the test engine generate inputs that stress those invariants. This tutorial walks you through setting up PBT for a typical REST/GraphQL web API, with concrete, runnable examples in JavaScript/TypeScript and Python. You’ll learn when to use PBT, how to design meaningful properties, how to integrate with your existing test suites, and how to interpret failures.

Overview of what you’ll build

A small web API (simulated) that manages a simple resource, such as "widgets" with CRUD operations.
A property-based test suite that checks core invariants, including normalization, idempotence, validation, and error handling.
A minimal mutation-based testing strategy to explore edge cases and unexpected inputs.
Practical tips for narrowing failing cases, shrinking inputs, and maintaining fast feedback loops.

Why property-based testing for APIs

Covers a much larger input space than example-based tests.
Finds edge cases that you didn’t anticipate.
Encourages designing robust input validation and well-defined invariants.
Works well with contract tests and can complement fuzz testing.

Prerequisites

Familiarity with your project’s testing framework (Jest/Jasmine, Pytest, etc.).
Basic understanding of property-based testing concepts.
A running API project or a local mock you can test against.

Part 1. Foundations: properties you want to encode
Define a concise, meaningful set of properties for your API. These should reflect business rules and technical invariants, such as:

Validation: requests with invalid fields are rejected with a 400/422 and meaningful error messages.
Normalization: input normalization (trim, lowercase, canonical IDs) happens consistently.
Idempotence: repeated same operation yields the same result and no unintended side effects.
Consistency: related resources stay in sync (e.g., creation returns a stable ID; fetching by that ID returns identical data).
Error handling: invalid IDs or missing resources return standard error responses.

For a “widget” resource, example properties:

Creating a widget with a valid schema returns 201 and a non-empty, stable ID.
Fetching a widget by its ID returns data identical to what was created.
Updating a widget with no substantive changes should not alter the last-updated timestamp.
Deleting a widget makes it inaccessible; subsequent fetch returns 404.
Input normalization: widget names are stored in title case; IDs are UUIDs in canonical form.
Validation: negative price or zero stock is rejected with a clear error.

Part 2. Tooling choices

JavaScript/TypeScript: fast-check is a popular property-based testing library.
Python: Hypothesis is a leading PBT library.
Ensure you can run tests in isolation (use a test database or an in-memory store, or mock the store).

Part 3. Example: a REST-like widget API (local mock)
We’ll simulate a minimal in-memory API in TypeScript for clarity. You can adapt to your framework (Express, Fastify, NestJS, etc.).

Code: in-memory widget store and API stubs

widgets have: id (string, UUID), name (string), price (number), stock (number), createdAt (string), updatedAt (string)

TypeScript snippet (in a test file or module):

This is a lightweight mock; replace with real API calls in your app.

// Simple in-memory store
type Widget = {
id: string;
name: string;
price: number;
stock: number;
createdAt: string;
updatedAt: string;
};

const store: Map = new Map();

function nowIso(): string {
return new Date().toISOString();
}

function normalizeName(name: string): string {
// Example normalization: trim and title-case
const trimmed = name.trim();
return trimmed.replace(/\w\S*/g, (w) => w.toUpperCase() + w.slice(1).toLowerCase());
}

function getWidget(id: string): Widget | null {
return store.get(id) ?? null;
}

function deleteWidget(id: string): boolean {
return store.delete(id);
}

// Simple API-like wrappers (simulate HTTP responses)
function apiCreateWidget(input: { name: string; price: number; stock: number }): { status: number; body?: any } {
try {
const w = createWidget(input);
return { status: 201, body: w };
} catch (e) {
return { status: 400, body: { error: (e as Error).message } };
}
}

function apiGetWidget(id: string): { status: number; body?: any } {
const w = getWidget(id);
if (!w) return { status: 404, body: { error: "Widget not found" } };
return { status: 200, body: w };
}

function apiDeleteWidget(id: string): { status: number; body?: any } {
const ok = deleteWidget(id);
return ok ? { status: 204 } : { status: 404, body: { error: "Widget not found" } };
}

export {
apiCreateWidget,
apiGetWidget,
apiUpdateWidget,
apiDeleteWidget,
// exports for tests
store,
Widget,
};

Notes:

In real tests, you’d replace the in-memory store with your API endpoints or a test server.

Part 4. Writing property-based tests
Choose a library:

TypeScript: fast-check
Python: Hypothesis

We’ll outline test properties and show example test cases in both ecosystems.

A. TypeScript with fast-check (Jest)
Install: npm install fast-check save-dev

Example test suite (pseudo-API wrappers from above):
import { apiCreateWidget, apiGetWidget, apiUpdateWidget, apiDeleteWidget } from "./widgetApi";
import fc from "fast-check";

describe("Widget API property tests", () => {
test("creating a valid widget yields 201 and stable ID", () => {
fc.assert(
fc.record({
name: fc.stringOf(fc.ascii(), { minLength: 1, maxLength: 50 }),
price: fc.double({ min: 0, max: 10000 }),
stock: fc.integer({ min: 0, max: 1000 }),
}).map(input => {
// normalize to a safe lowercase version
return input;
}),
{ verbose: true }
);
});

// Concrete property-based tests
test("creating then fetching yields identical data", () => {
fc.assert(
fc.record({
name: fc.string({ minLength: 1, maxLength: 60 }),
price: fc.double({ min: 0, max: 1000 }),
stock: fc.integer({ min: 0, max: 500 }),
}).chain(input => {
const res = apiCreateWidget(input);
if (res.status !== 201) return fc.constant(false);
const created = res.body as any;
const fetch = apiGetWidget(created.id);
if (fetch.status !== 200) return fc.constant(false);
const fetched = fetch.body;
// properties to compare
return fc.constant(
fetched.id === created.id &&
fetched.name === created.name &&
fetched.price === created.price &&
fetched.stock === created.stock
);
})
);
});

test("updating with no changes does not alter updatedAt", () => {
// create first
const res = apiCreateWidget({ name: "Sample", price: 10, stock: 5 });
if (res.status !== 201) return;
const id = res.body.id;
const before = apiGetWidget(id).body;
// patch with no changes
const patch = {};
const updated = apiUpdateWidget(id, patch);
const after = apiGetWidget(id).body;
// updatedAt should be equal
expect(after.updatedAt).toBe(before.updatedAt);
});

test("deleting a widget makes it inaccessible", () => {
const res = apiCreateWidget({ name: "Temp", price: 1, stock: 1 });
if (res.status !== 201) return;
const id = res.body.id;
apiDeleteWidget(id);
const fetch = apiGetWidget(id);
expect(fetch.status).toBe(404);
});
});

Notes:

In fast-check, you can compose arbitraries to generate valid/invalid inputs and then assert API behavior.
Use fc.constant to embed fixed expectations or fc.sample for quick checks.

B. Python with Hypothesis (pytest)
Install: pip install hypothesis

Example test file (tests/test_widget_api.py):
from widget_api import api_create_widget, api_get_widget, api_update_widget, api_delete_widget

from hypothesis import given
from hypothesis import strategies as st

@given(
name=st.text(min_size=1, max_size=60),
price=st.floats(min_value=0.0, max_value=10000.0),
stock=st.integers(min_value=0, max_value=1000),
)
def test_create_and_fetch_idempotent(name, price, stock):
res = api_create_widget({"name": name, "price": price, "stock": stock})
assert res["status"] == 201
widget_id = res["body"]["id"]
fetch = api_get_widget(widget_id)
assert fetch["status"] == 200
created = res["body"]
fetched = fetch["body"]
assert fetched["id"] == created["id"]
assert fetched["name"] == created["name"]
assert float(fetched["price"]) == float(created["price"])
assert int(fetched["stock"]) == int(created["stock"])

@given(id=st.text(min_size=1).filter(lambda s: True)) # random probe
def test_delete_then_get(id):
del_res = api_delete_widget(id)
# If it exists and is deleted, subsequent get should be 404
if del_res["status"] == 204:
get_res = api_get_widget(id)
assert get_res["status"] == 404

Notes:

Hypothesis strategies can be tuned to generate realistic business data (e.g., names with certain patterns, prices with currency-like ranges).
Use given decorators to create a suite of randomized inputs and failures.

Part 5. Practical strategies for effective PBT

Start with high-level properties first. Don’t try to encode every possible edge case at once.
Use shrinking: rely on the library’s shrinking to minimize failing inputs to the smallest reproducer.
Tidy your API contract: ensure error responses are consistent (status codes, error shapes). This makes properties easier to express and test.
Separate concerns: test input validation with dedicated tests, then test core invariants with PBT.
Combine PBT with contract tests: use PBT to stress the contract while relying on explicit tests for critical business rules.
Limit test time: use reasonable bounds on generated inputs and a maximum number of test cases to keep feedback fast.

Part 6. Interpreting failures and debugging tips

Read the shrinking result carefully. It shows the smallest failing input; use it as your starting point for real fixes.
Add observability in your API: log incoming payloads, validation errors, and response times to help identify root causes.
If a property fails intermittently, ensure test isolation and deterministic seeds where possible.
When dealing with randomness, seed your tests and reproduce exact seeds when a failure occurs.

Part 7. Maintenance and evolution

As your API evolves, review and prune properties that no longer apply or add new properties for new features.
Keep data schemas in sync with tests. If you change field types or validation rules, adjust properties accordingly.
Use property-based tests alongside traditional unit tests to maximize coverage without an explosion of test cases.

Illustrative example: a failure and how to shrink it

Suppose fast-check reveals a failing case where creating a widget with name " a b " and price -5 triggers a 400, but your validation intended to reject negative prices. The input reveals an edge-case in normalization or validation order. Shrinking reduces whitespace and value ranges to a minimal reproducible case (e.g., name "A" price -0.01). You can then adjust validation logic to catch negative values before normalization or clarify error messages for the combination.

Best practices checklist

Start small: identify two or three core properties and implement them first.
Keep tests fast: generate reasonable input ranges; avoid expensive setup per test.
Validate error shapes: ensure error messages are stable and machine-readable when possible.
Use real endpoints if you can: testing against real services catches integration issues earlier.

What to implement next

If you’re ready, I can tailor a concrete setup for your stack (Express/NestJS or Python FastAPI) and provide a ready-to-run example with fast-check or Hypothesis, including a minimal CI workflow.
I can also help craft a small set of core properties specific to your API domain, including sample inputs and expected failure modes.

Would you like me to adapt this tutorial to your tech stack (JavaScript/TypeScript with Node.js, or Python with FastAPI/Django) and generate a complete runnable project scaffold? If yes, tell me your framework and testing library preferences, and I’ll provide a ready-to-run repository layout with code you can copy-paste.

Rizwan Saleem | https://rizwansaleem.co