Yann Normand

Posted on Aug 18, 2020 • Updated on Aug 20, 2020

JSON Merge Patch in .NET Core

#csharp #dotnet #webdev #javascript

The other day, I was looking at how I would implement a REST API in .NET Core that accepts PATCH requests.

The many ways to patch a resource

I know of two RFCs that describe a format to represent a partial JSON update:

While the similar name is prone to confusion, they describe different structures to represent a partial JSON update.

Given the following sample JSON representation of a person:

{
  "name": "Joe",
  "email": "joe@example.com",
  "physicalAttributes": { "weight": 75, "height": 175 },
  "favoriteColors": ["blue", "red"]
}

A JSON Merge Patch request body to remove the email address and to add "black" as a favourite colour would look like:

{
  "email": null,
  "favoriteColors": ["blue", "red", "black"]
}

The same with JSON Patch would be:

[
   { "op": "remove", "path": "/email" },
   { "op": "add", "path": "/favoriteColors/-", "value": "black" }
]

It is interesting to note that JSON Merge Patch doesn't support partial updates of array items. Arrays must instead be entirely replaced.

On the other hand, JSON Patch is designed to allow the mutation of arrays.

Because the JSON Merge Patch structure mimics the target resource, it makes it easier for client applications to craft a JSON Merge Patch request than a JSON Patch request.

Most popular public REST APIs tend to align more closely to the JSON Merge Patch spec.

Patching a resource in .NET Core

Let us come back to partial JSON updates in the context of .NET Core.

Last year, The .NET Core team announced their own JSON serializer as part of System.Text.Json, moving away from the popular NewtonSoft.Json library.

If you search for "PATCH ASP.NET Core", one of the first results is JSONPatch.

However, JSONPatch still relies on the NewtonSoft.Json package, and, as its name implies, implements the JSON Patch spec.

To support JSON Merge Patch, the first approach that comes to mind is to define a DTO class and to use model binding to deserialize the request into a DTO instance.

This is what is traditionally done for PUT and POST (Create) requests.

Using the prior sample request, we would define the following:

public class Person
{
  public string Name { get; set; }
  public string Email { get; set; }
  public PhysicalAttributes PhysicalAttributes { get; set; }
  public List<string> FavoriteColors { get; set; }
}

public class PhysicalAttributes
{
   public decimal? Weight { get; set; }
   public decimal? Height { get; set; }
}

We can then call it a day, right?

...not so quick.

This works for the most part but there is an aspect of the spec that is not fulfilled.

null values in the merge patch are given special meaning to indicate the removal of existing values in the target.

This information gets lost after deserializing the request.
A null property on the C# model can represent either a null JSON value or the absence of the JSON property.

In the preceding patch example, the intent is to remove the email but such an implementation would instead ignore it, as if the property was absent from the request.

JSON Merge Patch with System.Text.Json

I stumbled upon this issue in the dotnet/runtime Github repo.

All credits for the solution below go to Ahson Khan.

As I was explaining, the naive initial approach was losing information about what properties were explicitly sent with null.

One thing we can do is to parse the patch document and extract all those null properties.

Combined with a DTO that contains all the non-null properties, there is now enough information to correctly patch the resource.

This can be done with the few lines of code below:

public static List<string> ExtractNullProperties(string patch)
{
  using var patchDoc = JsonDocument.Parse(patch);
  if (patchDoc.RootElement.ValueKind != JsonValueKind.Object)
  {
    throw new InvalidOperationException($"The patch JSON document must be an object type. Instead it is {patchDoc.RootElement.ValueKind}.");
  }

  return ExtractNullPropertiesFromObject(patchDoc.RootElement).ToList();
}

private static IEnumerable<string> ExtractNullPropertiesFromObject(JsonElement patch)
{
  Debug.Assert(patch.ValueKind == JsonValueKind.Object);
  foreach (var property in patch.EnumerateObject())
  {
    if (property.Value.ValueKind == JsonValueKind.Null)
    {
      yield return property.Name;
    }
    else if (property.Value.ValueKind == JsonValueKind.Object)
    {
      foreach (var field in ExtractNullPropertiesFromObject(property.Value))
      {
        yield return String.Join('.', property.Name, field);
      }
    }
  }
}

Here, we recursively identify all null properties and return a list of property names, with nested property names separated by . i.e physicalAttributes.weight.

What I like about this solution is that it doesn't preclude the API from using any other JSON serializer for deserializing the non-null properties.

However, notice here that the request will need to be read twice

once to populate the DTO with the serializer of choice
once to extract all null properties.

The API is then responsible for taking into account the list of properties that need to be deleted.

The second approach is to serialize the original resource, apply the patch, and then to deserialize the JSON result into an object that represents the patched resource.

public static T MergeModel<T>(T original, string patch, JsonSerializerOptions options = null)
{
    var originalJson = JsonSerializer.Serialize(original, options);
    return JsonSerializer.Deserialize<T>(Merge(originalJson, patch), options);
}

public static string Merge(string original, string patch)
{
    var outputBuffer = new ArrayBufferWriter<byte>();
    using (var originalDoc = JsonDocument.Parse(original))
    using (var patchDoc = JsonDocument.Parse(patch))
    using (var jsonWriter = new Utf8JsonWriter(outputBuffer))
    {
        var originalKind = originalDoc.RootElement.ValueKind;
        var patchKind = patchDoc.RootElement.ValueKind;
        if (originalKind != JsonValueKind.Object)
        {
            throw new InvalidOperationException($"The original JSON document to merge new content into must be an object type. Instead it is {originalKind}.");
        }
        if (patchKind != JsonValueKind.Object)
        {
            throw new InvalidOperationException($"The patch JSON document must be an object type. Instead it is {originalKind}.");
        }
        if (originalKind != patchKind)
        {
            return original;
        }
        MergeObjects(jsonWriter, originalDoc.RootElement, patchDoc.RootElement);
    }
    return Encoding.UTF8.GetString(outputBuffer.WrittenSpan);
}

private static void MergeObjects(Utf8JsonWriter jsonWriter, JsonElement original, JsonElement patch)
{
    Debug.Assert(original.ValueKind == JsonValueKind.Object);
    Debug.Assert(patch.ValueKind == JsonValueKind.Object);
    jsonWriter.WriteStartObject();
    // Write all the properties of the original document.
    // If a property exists in both documents, either:
    // * Merge them, if they are both objects
    // * Completely override the value of the original with the one from the patch, if the value kind mismatches (e.g. one is object, while the other is an array or string)
    // * Ignore the original property if the patch property value is null
    foreach (var property in original.EnumerateObject())
    {
        if (patch.TryGetProperty(property.Name, out JsonElement patchPropValue))
        {
            if (patchPropValue.ValueKind == JsonValueKind.Null)
            {
                continue;
            }
            jsonWriter.WritePropertyName(property.Name);
            var propValue = property.Value;
            if (patchPropValue.ValueKind == JsonValueKind.Object && propValue.ValueKind == JsonValueKind.Object)
            {
                MergeObjects(jsonWriter, propValue, patchPropValue); // Recursive call
            }
            else 
            {
                patchPropValue.WriteTo(jsonWriter);
            }
        }
        else
        {
            property.WriteTo(jsonWriter);
        }
    }
    // Write all the properties of the patch document that are unique to it (beside null values).
    foreach (var property in patch.EnumerateObject())
    {
        if (!original.TryGetProperty(property.Name, out JsonElement patchPropValue) && patchPropValue.ValueKind != JsonValueKind.Null)
        {
            property.WriteTo(jsonWriter);
        }
    }
    jsonWriter.WriteEndObject();
}

Example:

var originalModel = new Person { Email = "joe@example.com", Name = "Joe", PhysicalAttributes = new PhysicalAttributes (75, 175), FavoriteColors = new List<string> { "blue", "red" } };

string patch = @"{ 
""favoriteColors"": [""black""], 
""email"": null, 
""physicalAttributes"": {
    ""weight"": 80
}}";

var options = new JsonSerializerOptions
{
  PropertyNameCaseInsensitive = true,
  PropertyNamingPolicy = JsonNamingPolicy.CamelCase
};

var patchedModel = JsonMergeUtils.MergeModel(originalModel, patch, options);
// patchedModel { Name = "Joe", Email = null, FavoriteColors = ["black"], PhysicalAttributes = { Weight = 80, Height = 175 } }

MergeModel uses the System.Text.Json serializer here but it could easily be swapped with another JSON serializer.

There it is, you now have the building blocks to process JSON Merge requests.

The code above, along with some async overrides, can be found in this gist.

Top comments (4)

shanchin2k • Sep 24 '20

Thanks for the good article. I am trying a JSON merge PATCH and my idea was to set FieldStatus for all the fields whether they are supplied with null values with the request. API handles patch accordingly. I can populate to determine whether values are set to null by using the approach one in the article. But what is the best way to make the field statuses updated in the model? A model binder?

Yann Normand • Sep 30 '20 • Edited

To achieve what you want, it would require a way to map an incoming null JSON property to a field on the associated model.

I don't know how that could be done with System.Text.Json.
It wouldn't be trivial to have a generic solution because every .NET type and property can define custom deserialization rule.

A JSON property from a request doesn't necessarily translate to a field or property on the .NET type it would deserialize into.

I initially went down that path but couldn't wrap my head around it, notably when dealing with nested fields.
There is also a lot of metadata that System.Text.Json keeps internal.

I am afraid the best you could do is to extract the list of null properties with ExtractNullProperties and work with that (and maybe use reflection to map the property name to an object's property, keeping in mind all the caveats I mentioned above).