The problem
We shipped a "deprecated" internal field, response.legacy_id, and left it in for backward compatibility — marked deprecated in the docs, with a removal date two releases out. Nobody consumed it directly; no client code referenced it anywhere. So we cut it.
Three unrelated services broke in production within about forty minutes. Not because they read the field. Because a shared caching proxy fingerprinted the entire response body to build cache keys, and removing a field changed every hash, which invalidated caches that three completely different systems depended on for reasons that had nothing to do with legacy_id.
Nobody used the field. Everybody depended on it anyway.
Why it happens
This has a name — Hyrum's Law, coined by Hyrum Wright at Google: "With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody."
The key word is observable, not documented. Your contract says "this field is deprecated, don't rely on it." Reality doesn't care what the contract says. If a behavior is observable — response ordering, field presence, error message text, timing, byte-for-byte JSON shape, even header casing — someone, somewhere, at scale, will build on it. Usually not maliciously. Usually accidentally, through a generic tool (a diffing layer, a schema validator, a caching proxy, a test snapshot) that treats the entire observable surface as the contract, because that's the only contract it can see.
The bigger your user base, the more certain this becomes. At 10 callers, a dependency on field ordering is a coincidence. At 10,000, it's a statistical certainty that it exists somewhere.
This is also why "nobody's using it, I checked the code" is a weaker guarantee than it feels. You checked the code that reads your API directly. You didn't check every downstream cache key, diff tool, monitoring rule, or test fixture built against your response shape.
What to do about it
You can't design your way out of Hyrum's Law — you can only manage the blast radius.
Version your contract explicitly, and make it narrower than what you actually return. If your spec only promises three fields, undocumented extra fields are fair game to change — but say so loudly, because "undocumented" doesn't mean "unobserved."
Make deprecation observable before it's real. Don't silently remove a field. First return it with a sentinel value for a full release cycle, log every caller that still reads a non-null value, and ship machine-readable deprecation signals (like Sunset/Deprecation headers) instead of a line in a changelog nobody reads.
Change shape deliberately, not incidentally. If you're removing a field, make it its own atomic change — not bundled with an unrelated refactor — so if something breaks, the diff that caused it is one commit, not forty.
Treat internal consumers with the same discipline as external ones. The break above wasn't a public API — it was internal service-to-service traffic, which teams often skip versioning discipline on because "we control both sides." You control both codebases. You don't control every cache and proxy sitting between them.
Assume the null hypothesis is wrong. Before removing any observable behavior, default to "something depends on this," not "probably nothing depends on this." That's cheap to disprove with real production traffic sampling, and expensive to discover the hard way.
Key takeaways
- Hyrum's Law: at scale, every observable behavior of your system becomes a dependency, whether you documented it or not.
- "Nobody calls this field" only accounts for direct callers — indirect consumers (caches, diff tools, snapshots, monitors) depend on the whole observable shape.
- Deprecation should be logged and observable before it's enforced; silent removal is where the surprises live.
- Internal APIs need the same versioning discipline as public ones — "we control both sides" isn't the same as "we control every intermediary."
- Default to assuming something depends on the behavior you're about to change, and verify against real traffic before you cut it.
Top comments (0)