A JSON response can be syntactically valid and still break your application.
When building API integrations or AI-agent workflows, validating the JSON format is only the first step. You also need to check whether the response matches the contract your application expects.
Here are seven checks worth including in your workflow.
- Required fields
Are all required keys present in successful responses and error responses?
A missing field can break downstream processing, especially when an API response feeds another service or an AI agent.
- Field types
Is a number unexpectedly returned as a string? Are booleans, arrays, and objects consistent with the expected contract?
For example, ""count": "5"" and ""count": 5" are not equivalent for every application.
- Null vs. missing
Does your application distinguish between "null", a missing key, and an empty value?
These cases can have different meanings and should be tested explicitly.
- Enums and status values
What happens when an API returns an unexpected status or a new enum value?
Your application should handle unknown values deliberately rather than failing unpredictably.
- Unexpected fields and sensitive data
Could debug information, internal metadata, or sensitive values accidentally appear in a public response?
Check both the fields your application requires and any unexpected data that should not be exposed.
- Edge cases
Have you tested empty arrays, missing nested objects, malformed input, timeouts, and rate limits?
A contract that works for the happy path may still fail under ordinary edge cases.
- Schema drift
Do you rerun representative payloads after an API version, model, or prompt changes?
Even small changes can alter the structure of an AI-generated response or an upstream service's output.
Free tools to help
I built a small, free browser-based JSON Contract Tester and a practical checklist to help developers review common contract mismatches.
- "Try the free JSON Contract Tester" (https://falconprojectai-arch.github.io/falcon-ai-decision-engine/products/json-contract-tester/)
- "Read the seven-point contract review checklist" (https://falconprojectai-arch.github.io/falcon-ai-decision-engine/products/dev-agent-toolbox/contract-review-checklist.html)
A note about limitations: The tester currently supports only a subset of JSON Schema. A passing result does not guarantee full schema compliance, application correctness, or security. For production systems, use comprehensive automated tests and a validator appropriate to your schema and requirements.
These tools are intended as a practical starting point, not a replacement for a full testing strategy.
I'm sharing them to get feedback from other developers.
Which JSON contract mismatch has caused you the most trouble in a real project?
Disclosure: These tools are part of the Falcon project. I used AI assistance during development and preparation of this post. The tools are free to try; an optional human-reviewed contract review is also available.
Top comments (0)