Robustness principle

"Be conservative in what you send, be liberal in what you accept"

This wise advice, called also Robustness principle or Postel's law, turns out to be really useful in all those use cases in which applications send messages between them. Often those messages have a Json payload sent over HTTP.
A typical scenario involves:

1. Client serializes the model in a Json and sends it over HTTP to a server.
2. On the other side, the server gets the message, extracts the body of the request which is our Json, deserializes it back to a model (which can be different from the model of the client) and then performs operations with it.

Nothing really new here.
Even in this simple and common scenario, there is quite an important decision that must be made: the format of the data. In this case, the json structure of the message.
Let's assume, we are designing a rest endpoint which performs operations on a User. Based on the requirements we have, it looks that in order to model our user, we only need first name and last name.
The json for our user could look like this:

{ "firstname" : "Diego", "lastname" : "Maradona"}

We basically defined an implicit contract between our two pieces of the system (client and server). In order for them to work as a whole, they need to agree on the format of the data.
Even though our decision looks reasonable, we have violated another important principle:

"Increase the number of decision not made"

By defining the structure of the message, we have decided that the json which is going to be sent by the client to the server has two fields ("first name" and "last name") and it can only contain those two fields. It's exactly the second part of the sentence which sounds like an unnecessary decision. Indeed we are making our system hard to change for no good reason.
There a couple of cons associated with it:

1. If in the future we have to add a new field to our User, that can only be done by changing the client and the server at the same time and deploying them together.
2. Either the client or the server might change its own model, which is going to cause an error only detectable at run-time (unless we do additional work to test our contract)

Schema evolution

The problem is pretty common and goes under the general name of Schema evolution, definition which includes also stuff like Database schema evolution.
So it's not a big surprise that there are libraries out there which help doing that (for example Protocol buffer or Avro).
Even though I have experimented a bit with ProtoBuf, I rarely had the need to use it (mostly because it contains a super set of the features that I actually needed). Indeed, most of the java applications I have worked on already had Jackson or Gson as a dependency. It turns out that both of them can be tuned to become a tolerant reader/writer and support schema evolution.
If you are a Java programmer, you are probably familiar with Jackson or Gson, which are two popular libraries to convert Java POJO from/to Json. In case you are not, you might want to check out this tutorial. Either way, it should be pretty easy to follow the examples.

Example

The following examples show how to tune Jackson to be resilient to schema change. The examples are written in Java and you can find all the code used in the article in github.com/napicella/java-jackson-tolerant-reader.

This is how the model for our User looks like for our rest api:

public class User {
    private String name = "";
    private String surname = "";

    public User() {
    }

    public User(String name, String surname) {
        this.name = name;
        this.surname = surname;
    }
    // setters and getters …     
}

The server gets the json and deserializes it to an User type.

public void handle(Request request) throws IOException {
  ObjectMapper mapper = new ObjectMapper();  
  User user = mapper.readValue(request.body(), User.class);
   // do something with User
}

In the example above, Jackson object mapper maps the String in the request.body to the User class. It throws an exception if the json does not match the class.
For example, say the client sends us the following json:

{ "firstname" : "Diego", "lastname" : "Maradona", "middlename": "Armando" }

Jackson is going to throw an exception because the property "middlename" is not defined in the User class.

A tolerant reader - 1

What can we do about it?
Let's try to increase the number of decisions not made by defining that the json must contain at least "firstname" and "lastname".
Jackson allows to define that in two ways: programmatically and with Java annotations. Without loss of generality, we are going to do that with annotations:

@JsonIgnoreProperties(ignoreUnknown = true)
public class User {
//…

The @JsonIgnoreProperties(ignoreUnknown = true) tells Jackson to ignore any unknown properties in JSON input without exception.

    @Test
    public void test() throws IOException {
        ObjectMapper mapper = new ObjectMapper();
        User user =
                mapper.readValue("{ \"name\" : \"Diego\", \"surname\" : \"Maradona\", \"middlename\" : \"Armando\"}",
                        User.class);

        assertThat("TOLERANT-READER. The json might contain property that are not defined in the pojo. " +
                        "Ignore them!" +
                        "How: use @JsonIgnoreProperties(ignoreUnknown = true) annotation on the POJO",
                user.getName(), is("Diego"));
    }

By doing that, the server is not going to complain anymore about a field which is not defined in the User class.

A tolerant reader - 2

In many cases the server could deal with the fact that a property is missing by assigning it a default value. For the sake of the example, say that the server could accept an empty user, in which case it will assign default values for first name and last name.
This requires to set up two things:
1 - Set defaults for first name and last name in the model

@JsonIgnoreProperties(ignoreUnknown = true)
public class User {
    private String name = "Dries";
    private String surname = "Mertens";
    // rest of the class as before

2 - Tell Jackson not to serialize null values using the @JsonInclude(JsonInclude.Include.NON_NULL) annotation.

@JsonInclude(JsonInclude.Include.NON_NULL)
@JsonIgnoreProperties(ignoreUnknown = true)
public class User {
    private String name = "Dries";
    private String surname = "Mertens";
    // rest of the class as before

The JsonInclude.Include.NON_NULL guarantees that null values are not going to be serialized, so the json received by the server won't feature those properties at all.
Setting the defaults causes Jackson to use that value in case the property is missing from the Json.
As a result, the following test is green:

    @Test
    public void test2() throws IOException {
        ObjectMapper mapper = new ObjectMapper();
        User user = new User("Michael", null);
        String json = mapper.writeValueAsString(user);

        User deserializedUser = mapper.readValue(json, User.class);

        assertThat("TOLERANT-WRITER. Don't serialize null values. A tolerant reader prefers no value at all, " +
                        "because in that case can provide a default." +
                        "If you serialize null, there is no way for the reader to understand that actually the property" +
                        "is missing." +
                        "How: use @JsonInclude(JsonInclude.Include.NON_NULL) annotation on the POJO",
                deserializedUser.getSurname(), is("Mertens"));

        assertThat(deserializedUser.getName(), is("Michael"));
    }

Conclusions

There is definitely much more to say about schema evolution in the world of rest apis. Also interesting are the ways we can use to tests such contracts, but that's a discussion for another time.
Of course, any feedback would be greatly appreciated!