DEV Community

loading...
Cover image for Extending JSON for fun and profit

Extending JSON for fun and profit

krofdrakula profile image Klemen Slavič ・6 min read

header image by Steel's Fudge

In the early days of JavaScript when asynchronous requests first enabled web authors to make requests to HTTP servers and receive a readable response, everyone was using XML as the standard for data exchange. The problem with that was usually parsing; you'd have to have a beefy parser and serializer to safely communicate with a server.

That changed as Douglas Crockford introduced JSON as a static subset of the JavaScript language that only allowed strings, numbers and arrays as values, and objects were reduced to just key and value collections. This made the format robust while providing safety, since unlike JSONP, it would not allow you to define any executable code.

Web authors loved it [citation needed], API developers embraced it, and soon, standardization brought the JSON API into the fold of web standards.

Parsing JSON

The parse method takes just two arguments: the string representing a JSON value, and an optional reviver function.

With parsing, you may only have used the first argument to parse a function, which works just fine:

const json = '{"hello": "world"}'; const value = JSON.parse(json);

But just what does that reviver argument do, exactly?

Per MDN, the reviver is a function that will be passed every key and value during parsing and is expected to return a replacement value for that key. This gives you the opportunity to replace any value with anything else, like an instance of an object.

Let's create an example to illustrate this point. Say you have a fleet of drones that you'd like to connect to, and the API responds with an array of configuration objects for each drone. Let's start by looking at the Drone class:

const _name = Symbol('name'); const _config = Symbol('config'); class Drone { constructor(name, config) { Object.defineProperties( this, { [_name]: { value: name, configurable: false, enumerable: false }, [_config]: { value: config, configurable: false, enumerable: false } } ); } get name() { return this[_name]; } } const d = new Drone('George Droney', { id: 1 });

For simplicity, all the class does is provide the name property. The symbols defined are there to hide the private members from public consumers. Let's see if we can make a factory function that will convert the configurations into actual objects.

Our imaginary API server responds with the following JSON object:

[
  { "$type": "Drone", "args": ["George Droney", { "id": "1" } ] },
  { "$type": "Drone", "args": ["Kleintank", { "id": "2" } ] }
]

We want to turn each entry that has a $type property into an instance by passing the arguments to the constructor of the appropriate object type. We want the result to be equal to:

const drones = [
  new Drone('George Droney', { id: '1' }),
  new Drone('Kleintank', { id: '2' })
]

So let's write a reviver that will look for values that contain the $type property equal to "Drone" and return the object instance instead.

const _name = Symbol('name'); const _config = Symbol('config'); class Drone { constructor(name, config) { Object.defineProperties( this, { [_name]: { value: name, configurable: false, enumerable: false }, [_config]: { value: config, configurable: false, enumerable: false } } ); } get name() { return this[_name]; } } const jsonData = [ '[', ' { "$type": "Drone", "args": ["George Droney", { "id": "1" } ] },', ' { "$type": "Drone", "args": ["Kleintank", { "id": "2" } ] }', ']' ].join('\n'); const reviver = (key, value) => { switch(value.$type) { case 'Drone': { return new Drone(...value.args); } default: { return value; } } }; const drones = JSON.parse(jsonData, reviver);

The nice thing about the reviver function is that it will be invoked for every key in the JSON object while parsing, no matter how deep the value. This allows the same reviver to run on different shapes of incoming JSON data, without having to code for a specific object shape.

Serializing into JSON

At times, you may have values that cannot be directly represented in JSON, but you need to convert them to a value that is compatible with it.

Let's say that we have a Set that we would like to use in our JSON data. By default, Set cannot be serialized to JSON, since it stores object references, not just strings and numbers. But if we have a Set of serializable values (like string IDs), then we can write something that will be encodable in JSON.

For this example, let's assume we have a User object that contains a property memberOfAccounts, which is a Set of string IDs of accounts it has access to. One way we can encode this in JSON is just to use an array.

const user = {
  id: '1',
  memberOfAccounts: new Set(['a', 'b', 'c'])
};

We'll do this by using the second argument in the JSON API called stringify. We pass the replacer function

const user = { id: '1', memberOfAccounts: new Set(['a', 'b', 'c']) }; const replacer = (key, value) => { if (value instanceof Set) { return { $type: 'Set', args: [Array.from(value)] }; } else { return value; } }; const jsonData = JSON.stringify(user, replacer, 2);

In this way, if we want to parse this back into its original state, we can apply the reverse as well.

Completing the cycle

But before we verify that the reverse mapping works, let's extend our approach so that the $type can be dynamic, and our reviver will check to the global namespace to see if the name exists.

We need to write a function that will be able to take a name of a class and return that class' constructor so that we can execute it. Since there is no way to inspect the current scope and enumerate values, this function will need to have its classes passed to into it:

const createClassLookup = (scope = new Map()) => (name) =>
  scope.get(name) || (global || window)[name];

This function looks in the given scope for the name, then falls back onto the global namespace to try to resolve built-in classes like Set, Map, etc.

Let's create the class lookup by defining Drone to be in the scope for resolution:

const classes = new Map([
  ['Drone', Drone]
]);

const getClass = createClassLookup(classes);

// we can call getClass() to resolve to a constructor now
getClass('Drone');

OK, so let's put this all together and see how this works out:

const _name = Symbol('name'); const _config = Symbol('config'); class Drone { constructor(name, config) { Object.defineProperties( this, { [_name]: { value: name, configurable: false, enumerable: false }, [_config]: { value: config, configurable: false, enumerable: false } } ); } get name() { return this[_name]; } } const user = { id: '1', memberOfAccounts: new Set(['a', 'b', 'c']) }; const replacer = (key, value) => { if (value instanceof Set) { return { $type: 'Set', args: [Array.from(value)] }; } else { return value; } }; const jsonData = JSON.stringify(user, replacer, 2); const createClassLookup = (scope = new Map()) => (name) => scope.get(name) || (global || window)[name]; const classes = new Map([ ['Drone', Drone] ]); const getClass = createClassLookup(classes); const reviver = (key, value) => { const Type = getClass(value.$type); if (Type && typeof Type == 'function') { return new Type(...value.args); } else { return value; } } const parsedUser = JSON.parse(jsonData, reviver);

Et voilá! We've successfully parsed and revived the objects back into the correct instances! Let's see if we can make the dynamic class resolver work with a more complicated example:

const jsonData = `[
  {
    "id": "1",
    "memberOf": { "$type": "Set", "args": [["a"]] },
    "drone": { "$type": "Drone", "args": ["George Droney", { "id": "1" }] }
  }
]`;

Ready, set, parse!

const _name = Symbol('name'); const _config = Symbol('config'); class Drone { constructor(name, config) { Object.defineProperties( this, { [_name]: { value: name, configurable: false, enumerable: false }, [_config]: { value: config, configurable: false, enumerable: false } } ); } get name() { return this[_name]; } } const jsonData = [ '[', ' {', ' "id": "1",', ' "memberOf": { "$type": "Set", "args": [["a"]] },', ' "drone": { "$type": "Drone", "args": ["George Droney", { "id": "1" }] }', ' }', ']' ].join('\n'); const createClassLookup = (scope = new Map()) => (name) => scope.get(name) || (global || window)[name]; const classes = new Map([ ['Drone', Drone] ]); const getClass = createClassLookup(classes); const reviver = (key, value) => { const Type = getClass(value.$type); if (Type && typeof Type == 'function') { return new Type(...value.args); } else { return value; } } const data = JSON.parse(jsonData, reviver, 2);

If you drill down into the object structure, you'll notice that the memberOf and drone properties on the object are actual instances of Set and Drone!

Wrapping up

I hope the examples above give you a better insight into the parsing and serializing pipeline built into the JSON API. Whenever you are dealing with data structures for incoming data objects that need to be hydrated into class instances (or back again), this provides a way to map them both ways without having to write your own recursive or bespoke functions to deal with the translation.

Happy coding!

Discussion (14)

pic
Editor guide
Collapse
asparallel profile image
AsParallel • Edited

If I may, I think a better/more real-world use case for this method would be a content-aware facade using local/session storage. You definitely put an admirable amount of work into staging the experiment, and explained it quite well. It just feels like the API example, while clever, is a bit of a stretch.

Collapse
krofdrakula profile image
Klemen Slavič Author

True, I could have framed it as, say, a saving mechanism for a game using *Storage. I just hope the idea carries over rather than the implementation details in the reader's mind. :)

Collapse
gmartigny profile image
Guillaume Martigny

Very neat article with inspiring code. I never thought of your "private class member" with symbols, this is amazing (Why use Symbol.for tho ? It would allow access with instance[Symbol.for("name")] ?!.

Another question, why use Map for the classes dictionary ? Object literal would make a more readable code (IMO of course).

const createClassLookup = (scope = {}) => (name) =>
  scope[name] || (global || window)[name];
const classes = {
  Drone,
};
Collapse
krofdrakula profile image
Klemen Slavič Author

Actually, yeah, you're right... I should've used Symbol('name') instead. My bad! I'll fix the examples.

In the case of the classes lookup, you might have to have some mapping between the value of the $type property and the actual classname if they happen to not coincide. But really, there's no reason to pick Map over an object if you're just mapping strings to objects. I just like to use Map instead of POJOs as hashmaps when they become highly polymorphic, as it doesn't create hidden classes on mutation.

Collapse
jamesmalvi profile image
Jaimie Malvi

Awesome Article Klemen.

Would love to share JSON parser tool jsonformatter.org/json-parser

Collapse
ben profile image
Ben Halpern

Jaimie, you appear to be a sort of low-key spam account. But you've been on-platform for about a year and a half and you only have nine comments, all like this. So I'm not even mad, I'm impressed.

Collapse
johncarroll profile image
John Carroll

Your comment made me click on Jaimie's profile. I don't see how you reached the spam conclusion. Clearly, a JSON parser tool is on-topic here, and past comments about the tool also seem to be on-topic.

Thread Thread
krofdrakula profile image
Klemen Slavič Author

I would be wary of these kinds of online tools; you may be exposing sensitive data to a 3rd party. It's easy to mistakenly leak configuration, passwords or keys accidentally this way.

Collapse
the_doctor profile image
Vaibhav

TIL I qualify as a spam account.

Thread Thread
gmartigny profile image
Collapse
ben profile image
Ben Halpern

Fascinating

Collapse
carlosmgspires profile image
Carlos Pires

Great article. Concise and precise. Keep it up!

Collapse
philnash profile image
Phil Nash

Thanks for writing this! I had no idea about the reviver and replacer functions and I reckon they will come in handy in the future.

Collapse
mikaturk profile image
mikat

Very interesting article, thanks for the great read :D