Klemen Slavič

Posted on Oct 28, 2018

Extending JSON for fun and profit

#javascript

In the early days of JavaScript when asynchronous requests first enabled web authors to make requests to HTTP servers and receive a readable response, everyone was using XML as the standard for data exchange. The problem with that was usually parsing; you'd have to have a beefy parser and serializer to safely communicate with a server.

That changed as Douglas Crockford introduced JSON as a static subset of the JavaScript language that only allowed strings, numbers and arrays as values, and objects were reduced to just key and value collections. This made the format robust while providing safety, since unlike JSONP, it would not allow you to define any executable code.

Web authors loved it [citation needed], API developers embraced it, and soon, standardization brought the JSON API into the fold of web standards.

Parsing `JSON`

The parse method takes just two arguments: the string representing a JSON value, and an optional reviver function.

With parsing, you may only have used the first argument to parse a function, which works just fine:

But just what does that reviver argument do, exactly?

Per MDN, the reviver is a function that will be passed every key and value during parsing and is expected to return a replacement value for that key. This gives you the opportunity to replace any value with anything else, like an instance of an object.

Let's create an example to illustrate this point. Say you have a fleet of drones that you'd like to connect to, and the API responds with an array of configuration objects for each drone. Let's start by looking at the Drone class:

For simplicity, all the class does is provide the name property. The symbols defined are there to hide the private members from public consumers. Let's see if we can make a factory function that will convert the configurations into actual objects.

Our imaginary API server responds with the following JSON object:

[
  { "$type": "Drone", "args": ["George Droney", { "id": "1" } ] },
  { "$type": "Drone", "args": ["Kleintank", { "id": "2" } ] }
]

We want to turn each entry that has a $type property into an instance by passing the arguments to the constructor of the appropriate object type. We want the result to be equal to:

const drones = [
  new Drone('George Droney', { id: '1' }),
  new Drone('Kleintank', { id: '2' })
]

So let's write a reviver that will look for values that contain the $type property equal to "Drone" and return the object instance instead.

The nice thing about the reviver function is that it will be invoked for every key in the JSON object while parsing, no matter how deep the value. This allows the same reviver to run on different shapes of incoming JSON data, without having to code for a specific object shape.

Serializing into `JSON`

At times, you may have values that cannot be directly represented in JSON, but you need to convert them to a value that is compatible with it.

Let's say that we have a Set that we would like to use in our JSON data. By default, Set cannot be serialized to JSON, since it stores object references, not just strings and numbers. But if we have a Set of serializable values (like string IDs), then we can write something that will be encodable in JSON.

For this example, let's assume we have a User object that contains a property memberOfAccounts, which is a Set of string IDs of accounts it has access to. One way we can encode this in JSON is just to use an array.

const user = {
  id: '1',
  memberOfAccounts: new Set(['a', 'b', 'c'])
};

We'll do this by using the second argument in the JSON API called stringify. We pass the replacer function

In this way, if we want to parse this back into its original state, we can apply the reverse as well.

Completing the cycle

But before we verify that the reverse mapping works, let's extend our approach so that the $type can be dynamic, and our reviver will check to the global namespace to see if the name exists.

We need to write a function that will be able to take a name of a class and return that class' constructor so that we can execute it. Since there is no way to inspect the current scope and enumerate values, this function will need to have its classes passed to into it:

const createClassLookup = (scope = new Map()) => (name) =>
  scope.get(name) || (global || window)[name];

This function looks in the given scope for the name, then falls back onto the global namespace to try to resolve built-in classes like Set, Map, etc.

Let's create the class lookup by defining Drone to be in the scope for resolution:

const classes = new Map([
  ['Drone', Drone]
]);

const getClass = createClassLookup(classes);

// we can call getClass() to resolve to a constructor now
getClass('Drone');

OK, so let's put this all together and see how this works out:

Et voilá! We've successfully parsed and revived the objects back into the correct instances! Let's see if we can make the dynamic class resolver work with a more complicated example:

const jsonData = `[
  {
    "id": "1",
    "memberOf": { "$type": "Set", "args": [["a"]] },
    "drone": { "$type": "Drone", "args": ["George Droney", { "id": "1" }] }
  }
]`;

Ready, set, parse!

If you drill down into the object structure, you'll notice that the memberOf and drone properties on the object are actual instances of Set and Drone!

Wrapping up

I hope the examples above give you a better insight into the parsing and serializing pipeline built into the JSON API. Whenever you are dealing with data structures for incoming data objects that need to be hydrated into class instances (or back again), this provides a way to map them both ways without having to write your own recursive or bespoke functions to deal with the translation.

Happy coding!

Top comments (13)

Guillaume Martigny • Oct 29 '18

Very neat article with inspiring code. I never thought of your "private class member" with symbols, this is amazing (Why use Symbol.for tho ? It would allow access with instance[Symbol.for("name")] ?!.

Another question, why use Map for the classes dictionary ? Object literal would make a more readable code (IMO of course).

const createClassLookup = (scope = {}) => (name) =>
  scope[name] || (global || window)[name];
const classes = {
  Drone,
};

Klemen Slavič • Oct 29 '18

Actually, yeah, you're right... I should've used Symbol('name') instead. My bad! I'll fix the examples.

In the case of the classes lookup, you might have to have some mapping between the value of the $type property and the actual classname if they happen to not coincide. But really, there's no reason to pick Map over an object if you're just mapping strings to objects. I just like to use Map instead of POJOs as hashmaps when they become highly polymorphic, as it doesn't create hidden classes on mutation.

Jaimie Malvi • Oct 28 '18

Awesome Article Klemen.

Would love to share JSON parser tool jsonformatter.org/json-parser

Ben Halpern • Oct 28 '18

Jaimie, you appear to be a sort of low-key spam account. But you've been on-platform for about a year and a half and you only have nine comments, all like this. So I'm not even mad, I'm impressed.

John Carroll • Oct 29 '18

Your comment made me click on Jaimie's profile. I don't see how you reached the spam conclusion. Clearly, a JSON parser tool is on-topic here, and past comments about the tool also seem to be on-topic.

Klemen Slavič • Oct 29 '18

I would be wary of these kinds of online tools; you may be exposing sensitive data to a 3rd party. It's easy to mistakenly leak configuration, passwords or keys accidentally this way.

Vaibhav • Oct 29 '18

TIL I qualify as a spam account.

Guillaume Martigny • Oct 29 '18

TIL Kate Upton really like JSON :D
pinterest.fr/pin/139470919694956648/

Klemen Slavič • Oct 29 '18

True, I could have framed it as, say, a saving mechanism for a game using *Storage. I just hope the idea carries over rather than the implementation details in the reader's mind. :)