Quick Tip
Your integration tests pass because the API returns 200. Then users report the app is broken because user.profile.name became user.display_name. Status codes don't catch schema drift.
Eight lines with deepdiff catches it:
import requests
from deepdiff import DeepDiff
old = requests.get("https://api.example.com/v1/user/1").json()
new = requests.get("https://api.staging.example.com/v1/user/1").json()
diff = DeepDiff(old, new, ignore_order=True)
if diff:
print(diff.pretty()) # dictionary_item_removed, type_changes, etc.
Run this in CI against staging vs production and you get a readable list of every removed key, added key, and type flip (int → str is the silent killer) before deploy.
Real catch from last week: a payments API flipped amount_cents from int to str in a minor version bump. Every status-code test was green. This diff caught it in CI in 0.4 seconds.
pip install deepdiff — one dependency, no config.
I generated the first version of this with MonkeyCode (free tier: https://ly.cyberserval.tech/iIETXiF) by literally pasting two sample responses and asking "what changed?" — then made it a permanent CI gate.
What's the sneakiest breaking API change that ever shipped past your tests?
Top comments (0)