Most integration test suites check one thing: does our code do what we expect when we feed it a response we captured from the vendor six months ago. That's a useful test, but it answers the wrong question. It tells you nothing about whether the vendor's API still looks the way your fixture assumes it does today. Contract testing closes that gap, and it's a lot less work to set up than most teams assume.
What a Contract Test Actually Checks
A contract test verifies that a live API response still matches the shape your code expects, independent of whatever your regular unit tests are doing. Where a unit test asks "does my parsing function handle this JSON correctly," a contract test asks "is this JSON still the shape my parsing function was written against." Those are different questions, and most teams only ever automate the first one.
The distinction matters because a unit test against a stale fixture will happily pass forever, even after the vendor renames a field and your production code starts silently dropping data. A contract test hitting the vendor's real API, even a sandbox environment, catches the drift the moment it happens instead of the moment a customer notices.
Why Fixtures Alone Give False Confidence
It's worth being explicit about why a captured fixture fails at this job, because the failure mode is subtle. A fixture is a snapshot of the vendor's response at one point in time, frozen into your test suite. Your unit tests validate that your parsing logic handles that snapshot correctly, which is genuinely useful, but the test can't tell you anything about whether the live API still produces that shape today.
The dangerous part is that this kind of test suite gets more confident over time, not less. Every green build reinforces the assumption that the integration is healthy, right up until the day it silently isn't, because the fixture never asked the one question that mattered: does this still match reality.
Step 1: Define the Contract Explicitly
Before writing any test, write down what you actually expect from the vendor's response: which fields must exist, what type each one is, and which values are considered valid. JSON Schema is a natural fit here because it turns that expectation into a document you can validate against programmatically, rather than a set of assumptions scattered across your parsing code.
Keep this contract narrow. Only include fields your system actually reads. A contract that demands the entire vendor response match some sprawling schema will break on every cosmetic change the vendor makes, even ones that don't affect you, and your team will start ignoring the alerts.
Step 2: Run the Contract Against a Live Endpoint on a Schedule
A contract test that only runs at merge time misses drift that happens between deploys, which for most teams is most of the calendar. Run the same contract validation as a scheduled job, hourly or daily depending on how critical the integration is, hitting the vendor's real API or sandbox and checking the response against your schema.
Pact is purpose-built for this pattern on both sides of an integration, letting you define the expected interaction once and verify it independently of your normal test suite's cadence. For vendors where you don't control both ends, a lightweight scheduled script that fetches a real response and validates it against your JSON Schema captures most of the same protection without needing cooperation from the vendor's team.
Step 3: Fail Loud, Not Quiet
A contract test that logs a warning and continues is barely better than no contract test at all, because warnings get ignored the same way a green build gets ignored. When a contract check fails, it should page someone or block a deploy, the same way a failing unit test would. If your team doesn't trust the check enough to treat a failure as urgent, that's worth fixing before adding more contract tests, not after.
Document what a failure means when it fires. "The status field is now returning an integer instead of a string" is actionable. A generic "contract test failed" alert with no detail just trains the on-call engineer to dismiss it.
Step 4: Version the Contract Alongside Your Adapter
Every time you update your integration code to handle a new response shape, update the contract document in the same commit. Treating the contract as documentation that lives next to the code it describes, rather than a one-time artifact from initial setup, means the next engineer who touches the integration has an accurate reference instead of a stale schema nobody trusts.
The OpenAPI Specification works well for this even when documenting a vendor's undocumented API, because it gives your whole team one canonical description of what you're coding against, diffable in version control like any other code change.
Step 5: Treat Unexpected Fields as Signal, Not Noise
A well-designed contract test should also flag fields appearing in the response that weren't there before, not just missing or malformed ones. A new field showing up is often the first visible sign that a vendor is mid-rollout of a larger change, and catching it early gives your team time to investigate before the field becomes load-bearing somewhere in production without anyone deciding it should be.
Feeding contract test failures into an observability tool like Sentry turns a one-off test failure into a trend you can watch develop, which is often the difference between catching a vendor's migration in week one versus discovering it after a customer-facing incident in week six.
What to Do When a Contract Test Actually Fails
Have a runbook ready before the first failure happens, not after. At minimum it should answer: does the pipeline halt automatically, or does it keep running against the old adapter while someone investigates? Who gets paged? Is there a safe fallback behavior, or does this integration need to stop entirely until a human confirms what changed?
Teams that skip this step end up improvising a response under incident pressure the first time a contract test fires for real, which is a worse environment to make those decisions in than a calm planning session. The runbook doesn't need to be long. It needs to exist before you need it.
Common Mistake: Writing the Contract Too Broadly
A frequent first-attempt mistake is writing the contract against the vendor's entire response, every field they return, rather than just the fields your integration actually depends on. That approach guarantees the contract test fires on every cosmetic change the vendor makes, whether or not it affects you, and a test suite that cries wolf on harmless changes trains your team to stop trusting it within a few months.
Scope the contract to exactly what you read. If you add a new field dependency later, add it to the contract at the same time, in the same commit. This keeps the contract test's signal-to-noise ratio high enough that a failure is always worth investigating immediately, which is the entire point of building one.
The Payoff
Contract tests don't prevent vendors from changing their APIs. Nothing does. What they buy you is the difference between finding out from an automated check that runs every hour and finding out from a customer support ticket that took two weeks to trace back to its root cause. For an integration doing anything customer-facing or revenue-relevant, that difference alone justifies the setup cost many times over.
If you're maintaining more than a couple of vendor integrations without any contract testing in place, it's usually the first gap 137Foundry's integration engineers look for when auditing a client's data pipeline, and it's often the fastest fix relative to the risk it removes. For a longer walkthrough of the mapping layer this pairs with, see our guide to building a schema mapping layer that survives vendor API changes.
Top comments (0)