DEV Community: Imran Al Munyeem

Contract Testing in 10 Lines: JSON Schema Validation in Postman

Imran Al Munyeem — Mon, 10 Aug 2026 10:00:00 +0000

Here's a bug your test suite probably wouldn't catch.

A backend developer refactors the user model. The id field — an integer since forever — starts coming back as a string: "42" instead of 42. Every value is still "correct". Your assertion pm.expect(user.id).to.eql(42) fails, sure — but only on the one endpoint you asserted id on, not the other nine that return users. Meanwhile three client apps that did user.id + 1 are now computing "421".

That's structural drift, and it's what actually breaks API consumers: renamed fields, changed types, properties that quietly vanish. Field-by-field value assertions catch it patchily and by accident. Schema validation catches it systematically — and in Postman it costs about ten lines, because the ajv JSON-schema validator is built into the script sandbox.

The ten lines

In Scripts → Post-response on any request that returns a user:

const userSchema = {
    type: "object",
    required: ["id", "name", "email"],
    properties: {
        id:    { type: "integer" },
        name:  { type: "string" },
        email: { type: "string", pattern: "@" }
    }
};

pm.test("Response matches the user schema", () => {
    pm.expect(pm.response.json()).to.be.jsonSchema(userSchema);
});

That single test now fails if id becomes a string, if email disappears, if name becomes an object — every structural mutation, whether or not you thought to assert on that field's value.

For an endpoint returning an array of users:

const userListSchema = {
    type: "array",
    minItems: 1,
    items: userSchema   // reuse the object schema
};

pm.test("List matches schema", () => {
    pm.expect(pm.response.json()).to.be.jsonSchema(userListSchema);
});

Share one schema across every endpoint

The real power move: your API returns users from /users, /users/:id, /login, /teams/:id/members… and they should all be the same shape. Store the schema once as a collection variable (JSON, stringified), and every request validates against the same contract:

// One-time setup — e.g. in the collection's Pre-request script:
pm.collectionVariables.set("userSchema", JSON.stringify({
    type: "object",
    required: ["id", "name", "email"],
    properties: {
        id:    { type: "integer" },
        name:  { type: "string" },
        email: { type: "string", pattern: "@" }
    }
}));

// In any request's Post-response script:
const schema = JSON.parse(pm.collectionVariables.get("userSchema"));

pm.test("User matches the shared contract", () => {
    pm.expect(pm.response.json()).to.be.jsonSchema(schema);
});

Now when the contract legitimately changes — a new required field, say — you update one variable and every endpoint's test updates with it. That's contract testing: not a new tool, just the discipline of asserting shape centrally.

Schema tips that save real debugging time

required is where the protection lives. JSON Schema ignores missing properties unless they're listed in required. A schema without required validates {} happily — the most common reason people think their schema test "isn't working".

Type-check numbers deliberately. integer vs number matters — prices and quantities have different rules. And remember the drift that motivates all this is usually number→string, so never write type: ["integer", "string"] to make a flaky test pass. That's deleting the alarm.

Formats catch bad data early. pattern for emails and IDs, enum for status fields:

status: { type: "string", enum: ["active", "suspended", "deleted"] }

When someone adds a fourth status without telling anyone, you find out from a red test instead of a production incident.

Start schemas from reality, not from scratch. Send the request once, copy the response, and derive the schema from it (an LLM does this conversion well — then review it against the spec, especially which fields are truly required; a schema generated from one happy response will mark optional fields as required and miss fields that were null that day).

Where this sits in a strategy

Schema validation is one layer of a complete endpoint checklist — protocol (status, headers, timing), contract (this), data values, behaviour (state actually changed), and security (auth negatives). It's the layer with the best effort-to-protection ratio in the whole list: ten lines per shape, and an entire category of consumer-breaking regressions becomes impossible to ship silently.

Adapted from Chapter 7 of my free, open-source book *API Testing Using Postman: The Practical Guide to Modern API Testing*. Read online, grab the PDF/EPUB, or contribute on GitHub.

I'm a PhD researcher in Computer Science at Nottingham Trent University working on cybersecurity and AI-assisted security testing. More at imranalmunyeem.com.

Still Testing APIs Like It's 2022? Here's Everything That Changed"

Imran Al Munyeem — Mon, 03 Aug 2026 10:00:00 +0000

If you learnt Postman a few years ago and have been coasting on that knowledge, an uncomfortable amount of it is now wrong — not "slightly dated" wrong, but "the tab you're looking for doesn't exist" wrong.

I know because I wrote a Postman guide in 2022 and recently rewrote it as a full book. Here's the changelog of everything that broke, moved, or appeared — the field guide I wish I'd had.

1. The Tests tab is gone

The old Tests and Pre-request Script tabs were merged into a single Scripts tab with two sections: Pre-request (runs before the request) and Post-response (runs after — this is where your tests live now).

Your code is unchanged — same pm.test, same Chai assertions, same snippets — only the geography moved:

// Exactly the same as it ever was, just under Scripts → Post-response
pm.test("Status code is 200", function () {
    pm.response.to.have.status(200);
});

Every tutorial screenshot showing a Tests tab is from the old world. The concepts transfer completely.

2. Newman has an official successor

Newman — the beloved open-source CLI runner — is no longer the only way to run collections from a terminal. The official Postman CLI authenticates with a Postman API key and can run collections straight from your workspace, no export step:

postman login --with-api-key $POSTMAN_API_KEY
postman collection run 12345678-abcd-efgh-ijkl-9876543210ab

It also pushes results back to Postman's cloud as a shareable run report.

Newman still works fine and its htmlextra HTML reports remain unmatched — but there's a hard compatibility line to know: Newman only supports collection format v2/v2.1. Postman v12 introduced a v3 (YAML) format for its Git-native workflows, and Newman cannot run it. New pipeline? Start with the Postman CLI. Existing Newman pipeline? It keeps working — just export v2.1.

3. Your Jenkins tutorial will not work

The classic "install JDK 8, run java -jar jenkins.war" instructions are dead: modern Jenkins requires Java 21 (recent LTS lines dropped Java 17 and older). If you're standing up a CI box today, grab Temurin 21 from adoptium.net first, or Jenkins simply won't start.

Better yet, notice that for many teams the Jenkins box is now optional — a 25-line GitHub Actions workflow runs your collection on every push and every night, with no server to patch:

name: API Tests
on:
  push:
  schedule:
    - cron: "0 2 * * *"
jobs:
  postman-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: "lts/*"
      - run: npm install -g newman newman-reporter-htmlextra
      - run: newman run collections/MyCollection.postman_collection.json -r cli,htmlextra

4. There's an AI in the footer

Postbot generates tests from plain-English prompts, adds baseline tests to a whole collection in one action, repairs failing scripts, documents requests, and visualises responses. It's genuinely useful and genuinely dangerous in the same specific way: it writes assertions from the response your API currently gives — so if the current behaviour is a bug, the bug becomes the expected result. Generate freely; review against the spec, not the response. (I wrote a whole piece on this failure mode.)

Postman also added AI request types (test your LLM-backed endpoints like any other), Agent Mode, and an MCP server — the platform is clearly betting that agents will be first-class API consumers.

5. Your practice APIs died

Heroku ended its free tier, taking a generation of tutorial APIs with it. If a guide points you at something.herokuapp.com, expect a dead link. JSONPlaceholder (jsonplaceholder.typicode.com) remains free, signup-less, and reliable — or create a mock server inside Postman itself from your saved examples.

6. The scripting API moved on politely

postman.setNextRequest(null) still works, but the current form is:

pm.execution.setNextRequest(null);   // stop the run
pm.execution.setNextRequest("Delete user");   // or jump to a request

Same story across the pm API: old code keeps running, new code should use the new names.

7. Secrets got first-class treatment

Two things every team should know: variables now have a secret type (masked on screen), and — the one that catches people — initial values sync to Postman's servers and are shared with collaborators, while current values stay on your machine. Real credentials go in current values only, or in Postman Vault (encrypted, local, never synced). The classic leak is a Bearer token pasted into an initial value "just for a second," then synced, then forked into a public workspace.

The takeaway

None of this changed what good API testing is — five layers of assertions, negative cases, deterministic suites, CI enforcement. What changed is the tooling around it, and the tooling changed enough that 2022 muscle memory now produces broken pipelines and missing tabs. Update the muscle memory; keep the principles.

This is the story behind my free, open-source book *API Testing Using Postman: The Practical Guide to Modern API Testing** — a 2022 guide rewritten end-to-end for how we test now. Read online, grab the PDF/EPUB, or contribute on GitHub.*

I'm a PhD researcher in Computer Science at Nottingham Trent University working on cybersecurity and AI-assisted security testing. More at imranalmunyeem.com.

AI Writes Your API Tests Now. Here's the Bug It Introduces.

Imran Al Munyeem — Mon, 27 Jul 2026 10:00:00 +0000

Ask an AI to write tests for your API endpoint and it will do something subtly dangerous: it will look at the response your API currently gives and write assertions that lock it in.

If that response is correct — great, you just saved twenty minutes. If that response is a bug — say, your API wrongly returns 200 OK for an invalid payload — the AI will happily generate pm.test("Status code is 200", ...) and your bug is now the expected behaviour, defended by a green test forever.

That's the whole tension of AI-assisted testing in one sentence: AI-generated tests validate what a response is, not what it should be. The specification lives in your head and your docs; the AI only sees the wire.

I use AI test generation daily and I'm not telling you to avoid it — I'm telling you how to use it without it quietly rotting your suite.

What the AI is genuinely great at

Postman's built-in assistant, Postbot, is the one most API testers will meet first (the pattern applies equally to Copilot, Claude, or whatever you use). Its strongest moves:

Boilerplate generation. Open a request, send it once so a response exists, open Scripts → Post-response, and prompt in plain English:

"Add tests for the status code, the content type, and that the response time is under 500 ms."

It writes idiomatic pm.test / pm.expect code straight into the editor. For an entire collection, Generate tests from the collection menu adds baseline tests to every request at once — hours of boilerplate become minutes of review.

Fixing broken scripts. When a test throws an error, Postbot's Fix script analyses it and proposes a correction. For syntax slips and stale JSON paths, this is close to magic.

Exploration. "What's wrong with this request?", "visualise this response as a table", "document this collection" — context-aware answers that used to mean twenty minutes of reading.

Test-case brainstorming. Outside Postman, feeding an OpenAPI spec to an LLM and asking for edge cases is genuinely productive: it will suggest boundary values, weird Unicode, and state transitions you hadn't listed. Treat the output as a checklist to evaluate, not a suite to paste.

Where it fails, specifically

Three failure modes recur:

1. The observed-behaviour trap (the big one, above). Every generated assertion needs review against the spec, not the response. The question is never "does this match what came back?" — the AI already guaranteed that. The question is "is what came back correct?"

2. Happy-path bias. Ask for tests and you'll get success cases. The negative suite — wrong passwords, missing fields, expired tokens, malformed bodies, the requests an attacker sends — still needs a human's adversarial instinct. My rule of thumb: a real endpoint needs five to ten negative/boundary cases per positive case, and the AI will volunteer roughly none of them unprompted.

3. Plausible nonsense. Occasionally the generated assertion is syntactically perfect and semantically empty — checking that an array "exists" rather than that it contains the right things. A suite full of these is worse than a small honest one, because it looks like coverage.

The working protocol

What this adds up to in practice:

Learn to write tests by hand first. Not for purity — because review is the whole job now, and you cannot review code you couldn't have written. A test you can't read is a test you can't trust.
Generate, then interrogate. For each AI assertion ask: does the spec say this? If the response were wrong in the most likely way, would this catch it?
Add the negatives yourself. Budget the time you saved on boilerplate for the adversarial cases the AI skipped.
Never paste secrets into prompts. Tokens, customer data, internal URLs — they don't belong in any AI conversation unless your plan contractually protects them. (Postbot on enterprise plans excludes inputs from training; check your tier's terms.)
Own the suite. If a generated test fails six months from now, someone has to understand it in ninety seconds. That someone is you.

The honest framing

Used this way, AI does for test authoring what Newman did for test execution: it removes the drudgery and leaves you the judgement. The testers who struggle with AI are the ones who hand over the judgement too — and then wonder why their green suite missed the bug that was green by design.

Machines for repetition, humans for meaning. That division of labour was true before LLMs and it will be true after whatever comes next.

Adapted from Chapter 14 of my free, open-source book *API Testing Using Postman: The Practical Guide to Modern API Testing** — 15 chapters from your first request to AI-assisted CI/CD pipelines. Read online, grab the PDF/EPUB, or contribute on GitHub.*

I'm a PhD researcher in Computer Science at Nottingham Trent University working on cybersecurity and AI-assisted security testing. More at imranalmunyeem.com.

How to Design an API Test Strategy (Not Just Accumulate Tests)

Imran Al Munyeem — Mon, 20 Jul 2026 10:00:00 +0000

Most API test suites are decorative.

They're green, they run on every build, and they check almost nothing that matters: a wall of status is 200 assertions over happy-path requests. When the API starts returning the wrong data, the wrong errors, or leaking internals in a stack trace, the suite stays green — because nobody ever decided what it was supposed to catch.

Tools execute tests; testers choose them. Given an endpoint, what should you actually verify? Here's the framework I use — five layers, three axes of case design, and a way to prioritise when you can't test everything first. Examples use Postman's test scripts, but the framework is tool-agnostic.

The layered checklist

For every endpoint that matters, work outside-in through five layers.

1. Protocol layer

The right status code for the right situation — and specific failures, not generic ones: 400 for malformed input, 401 for missing credentials, 403 for insufficient rights, 404 for absent resources. Plus the correct Content-Type, and a sensible response time.

pm.test("Status is 200", () => pm.response.to.have.status(200));

pm.test("Content-Type is JSON", () => {
    pm.expect(pm.response.headers.get("Content-Type"))
        .to.include("application/json");
});

pm.test("Responds within 500 ms", () => {
    pm.expect(pm.response.responseTime).to.be.below(500);
});

Cheap to write, catches an entire class of regressions. But if your suite stops here, it's decorative.

2. Contract layer

The response structure matches its schema: required fields present, types correct, formats valid. This is what protects the API's consumers — a renamed field or a number that quietly became a string breaks every client, while your value assertions may not even notice.

Postman ships the ajv JSON-schema validator, so contract testing is ten lines:

const userSchema = {
    type: "object",
    required: ["id", "name", "email"],
    properties: {
        id:    { type: "integer" },
        name:  { type: "string" },
        email: { type: "string", pattern: "@" }
    }
};

pm.test("Response matches the user schema", () => {
    pm.expect(pm.response.json()).to.be.jsonSchema(userSchema);
});

3. Data layer

The values are right: the created user has the name you sent; the filtered list contains only matching records; the total equals the sum of the parts; page 2 is actually different from page 1.

const body = pm.response.json();

pm.test("Filter returns only active users", () => {
    body.forEach(user => pm.expect(user.status).to.eql("active"));
});

4. Behaviour layer

State actually changed. After DELETE, a follow-up GET returns 404 — deletion that only claims to work is a classic bug. After PATCH, only the patched fields differ. POSTing the same order twice doesn't charge twice.

These require chaining requests — capture an ID from one response and use it in the next:

// In the POST's post-response script:
pm.test("User created", () => pm.response.to.have.status(201));
pm.collectionVariables.set("newUserId", pm.response.json().id);

// Next request: GET {{baseUrl}}/users/{{newUserId}}
// Then: DELETE, then GET again expecting 404

The behaviour layer is where the most expensive bugs live, and it's the layer happy-path suites never reach.

5. Security layer

Every negative auth case: no token (expect 401), an expired token (401), a valid token for the wrong user (403), a valid token with insufficient role (403). Plus: error responses that don't leak stack traces or internal hostnames.

Authentication isn't plumbing to get through on the way to your tests. It is a test surface — usually the highest-risk one you have.

Positive, negative, and boundary

For each input an endpoint accepts, generate cases along three axes:

Positive — valid, typical input succeeds. The case everyone writes.

Negative — invalid input fails correctly: missing required fields, wrong types, malformed JSON, unknown IDs, illegal state transitions. The assertion is not merely "it fails" but "it fails with the right status and a useful error body." An API that returns 500 for bad input has a bug even though it rejected the request — the server failed to validate.

Boundary — the edges: empty strings, zero, negative numbers, maximum lengths, page sizes of 0 and 1 and the documented limit and the limit plus one, Unicode in text fields.

A practical heuristic: for a typical endpoint, expect one or two positive cases and five to ten negative and boundary cases. If your suite is mostly green-path, it is mostly decorative.

In Postman, the clean way to industrialise this is a data file. One request, one parameterised test, a CSV of cases:

email,password,expectedStatus
valid@example.com,correct-pass,200
valid@example.com,wrong-pass,401
,correct-pass,400
not-an-email,correct-pass,400

pm.test("Status matches the data file", () => {
    pm.response.to.have.status(
        Number(pm.iterationData.get("expectedStatus"))
    );
});

Adding a case is now a spreadsheet edit — which means anyone on the team can extend coverage, not just the people who write JavaScript.

Prioritising: you cannot test everything first

Order the work by risk:

Endpoints whose failure costs most — auth, payment, anything that writes data
Endpoints that change most often — change is where regressions come from
Everything else

A ten-test suite on the login endpoint beats a hundred tests on a static reference lookup.

And adopt the single highest-value habit in test automation — the regression contract: every bug found in production earns a permanent test reproducing it. Over a year, this builds the most valuable suite you'll ever own, because it is provably aligned with how your system actually fails.

Keeping the suite honest

Three disciplines keep a growing suite trustworthy:

Deterministic tests. A test that sometimes fails without a code change is worse than no test — it trains the team to ignore red. Hunt flakiness down; it's usually shared state, missing teardown, or time-dependent assertions.

Independent tests where possible. Chained workflows are necessary, but keep chains short and self-contained. A suite where request 40 depends on request 3 is unmaintainable.

Readable failures. Name tests so a red line states the defect. "PATCH ignores read-only fields" tells the developer everything before they even open the tool. "Test 12 failed" tells them nothing.

The one-line summary

A test suite earns its keep not by the number of green ticks but by the decisions behind them: five layers per endpoint, five-to-one negative-to-positive ratio, risk-ordered, deterministic, and readable when red.

This article is adapted from Chapter 8 of my free, open-source book *API Testing Using Postman: The Practical Guide to Modern API Testing** — 15 chapters from your first request to AI-assisted CI/CD pipelines, with a 37-question interview FAQ. Read it online, grab the PDF/EPUB, or contribute on GitHub.*

I'm a PhD researcher in Computer Science at Nottingham Trent University working on cybersecurity and AI-assisted security testing. More at imranalmunyeem.com.