An API can be up, fast, and still break every client that depends on it.
A field changes from an object to an array. An optional property disappears. A path is renamed in the OpenAPI document.
The uptime check stays green because the server did exactly what it was asked to do: it returned 200 OK.
That gap is what I wanted to monitor.
Uptime answers only one question
A basic status probe can tell you whether:
- DNS resolved
- TLS completed
- the server responded
- the response arrived within a threshold
Those checks matter, but they do not tell you whether the response still means what your code expects.
Imagine yesterday's payload looked like this:
{
"customer": {
"id": "cus_42",
"status": "active"
}
}
Today it looks like this:
{
"customer": [
{
"id": "cus_42",
"status": "active"
}
]
}
Both responses are valid JSON. Both can arrive in 120 ms with a 200. Only one matches a client written for yesterday's contract.
Four kinds of API change
I find it useful to separate dependency failures into four buckets:
- Availability — status, timeout, TLS, or DNS failures.
- Shape — a field was added or removed, a type changed, or an object became an array.
- Contract — an OpenAPI path, method, parameter, or response schema changed.
- Behavior — the shape stayed the same, but the meaning changed.
The first three can be observed automatically. The fourth usually needs domain-specific assertions.
Keeping these separate also makes alerts easier to reason about. “The endpoint is down” and “the endpoint is up but incompatible” should not arrive as the same vague incident.
Raw diffs create their own problem
A generic JSON diff catches everything, including values that are supposed to change: timestamps, request IDs, counters, ordering, and rotating tokens.
The useful output is not just “these payloads differ.” It is closer to:
{
"rule": "type_changed",
"path": "$.customer",
"before": "object",
"after": "array",
"severity": "breaking"
}
That is small enough to use in a webhook, CI check, or incident timeline. More importantly, it tells the person responding where to look.
A practical monitoring loop
For each external API dependency:
- Probe a representative endpoint.
- Remove known volatile fields.
- Derive a normalized response shape.
- Compare it with the last accepted observation.
- Classify the change.
- Keep the evidence needed to reproduce it.
- Alert only when a rule says the change matters.
The phrase last accepted observation is important. If every result automatically becomes the new baseline, one malformed response can quietly normalize the failure.
I would also avoid testing with private credentials unless the monitor has a clear data-handling model. Public HTTPS targets and synthetic test accounts are a safer place to start.
What I built from this
I turned this pattern into a small developer API called DependSignal. It probes public APIs for status and latency, compares JSON response shapes and supported OpenAPI contracts, and returns field paths with rule evidence.
I kept the scope narrow on purpose. It is not a replacement for full contract tests or domain assertions. It is another layer for catching upstream drift before a user reports it.
The landing page includes a no-key sample, because a monitoring product should be inspectable before anyone creates credentials:
https://dependsignal.strigsapi.com/
The part I am still tuning is noise. Some teams want every added field; others only care about removals and type changes.
What does your team treat as a breaking API change: status, schema, behavior, or some combination?
Top comments (1)
This is such an important point! I've been burned by this exact issue when building API integrations. The key takeaway is that HTTP status codes don't tell the whole story — you need to validate the actual response structure and data types. I've started adding schema validation to all my API calls and it catches these subtle issues early. Thanks for sharing this insight!