JSON Schema default that complicates compatibility checks

Automating API contract checking can save you from human mistakes. The syntax part of API contracts is a low-hanging fruit that can be improved quickly by centralising the schemas on a schema registry.

Schema registries not only store the schemas, they can also perform automated compatibility checks on them.

Talking about API integrations, a bullet-proof, yet practical, definition of what is a non-breaking (compatible) change vs breaking (incompatible) is not trivial. It does not involve just the provider/owner of the API, but also all its consumers. For some consumers the change may be a breaking one, for others not.

JSON is very popular because it is simple and because JSON documents can be read by humans directly. But when it comes to automating schema compatibility checks, it shows some weaknesses.

Yes, you can tame the JSON data sent over your APIs to some degree using OpenAPI or AsyncAPI.

Let's take the Petstore example. If you pretty print the OpenAPI spec and scroll down to the section starting with

        "components": {
            "schemas": {

you can find JSON schemas like this one:

        "Category": {
            "type": "object",
            "properties": {
                "id": {
                    "type": "integer",
                    "format": "int64",
                    "example": 1
                },
                "name": {
                    "type": "string",
                    "example": "Dogs"
                }
            }
        }

Based on this you can tell that { "id": 1234 } is valid, but { "id": "1234" } (using a JSON string instead of the number) is invalid.

However, what the schema does not tell explicitly is, that a document with any extra fields is valid, too.

In this document

        { "id": 1234, "name": "my category", "newAwesomeField": true }

I added the "newAwesomeField" that is not specified by the schema, but any application that reads the JSON document should accept it. This usually works because deserializers like Jackson ignore unknown fields by default.

Actually, the above JSON Schema is equivalent to this one:

        "Category": {
            "type": "object",
            "properties": {
                "id": {
                    "type": "integer",
                    "format": "int64",
                    "example": 1
                },
                "name": {
                    "type": "string",
                    "example": "Dogs"
                }
            },
            "additionalProperties": true
        }

The "additionalProperties" JSON Schema attribute is commonly omitted. Its default value is true, meaning that documents with any properties on top of those explicitly listed are valid.

Now what if someone decides to create a new version of the schema, making the "newAwesomeField" official? Let's say the field is defined as follows:

        "newAwesomeField": {
            "type": "string",
            "example": "Some awesome free text"
        }

Now the document { "id": 1234, "name": "my category", "newAwesomeField": true } that was valid with the old schema becomes invalid! It is because the new schema defines newAwesomeField as a string, whereas in the document we use it with a boolean value.

This default openness of JSON Schema creates a barrier for a safe evolution of schemas.

The other two commonly used schema languages (Avro and Protobuf) are stricter, making the schema evolution with them more controlled. It makes sense for them because they are designed for efficient binary serialization. Too much flexibility (including for example that you can shuffle JSON fields in any order) is not good for efficient processing.

If you are using JSON in your API payloads and want to make compatibility checks safer, it may be a good idea to include "additionalProperties": false in your JSON Schemas.

If I triggered your interest, you can read all the details on how the additionalProperties work.