We shipped a new API version and added a field called region to the customer object. It was optional. The spec said optional, the schema said optional, and the release notes said optional. Nine months later we stopped populating it for a category of legacy accounts, because the value was meaningless for them, and four downstream integrations broke inside an hour.
Every one of those consumers had written code that read customer.region and used it directly. Not because they were careless, but because the field had been present in every single response any of them had ever seen. Optional in a schema means "may be absent." Optional in practice means "is always there until the day it isn't," and there is no way for a consumer to tell those apart by observation. They tested against reality, and reality lied to them.
That is the part I keep coming back to. A contract is not what the document says. It's the intersection of what the document says and what the system actually does, and consumers will always couple to the second one, because that's the thing they can see. If you emit a field on one hundred percent of responses for nine months, you have made a promise regardless of what the schema declares.
We changed two things. First, if a field is genuinely optional, prove it: our contract tests now generate responses with every optional field absent, and we run consumer-facing integration tests against that shape, so absence is exercised on day one rather than in month nine. A staging endpoint deliberately omits optional fields at random. Second, if a field turns out to be effectively mandatory in practice, we stop pretending and mark it required, then treat any future removal as a breaking version change with the deprecation process that implies.
The deprecation process is the boring half and the one that actually saved us afterwards. Log usage per consumer per field, so you know who reads what. Announce with a date. Then dark-launch the removal on a small percentage of traffic and watch their error rates, because that tells you what a mailing list never will.
Your API's real contract is your production traffic. Test the shapes you claim are legal, or your consumers will define the contract for you.
– Sergey Shinder
Top comments (0)