Querying JSON using JSONPath and JSONata

JavaScript Object Notation (JSON) is a robust medium for storing and transferring data.

It is the de facto standard for web-based client-to-server communication and is commonly used in REST and GraphQL APIs. Data can also be stored as JSON files. Many NoSQL databases like MongoDB and CouchDB rely on this mechanism as their primary storage engines.

A common approach in handling JSON-encoded data is parsing it into an object and searching for specific paths, keys, and values. However, this can be potentially redundant and time-consuming to code. What if we could directly query the JSON data just like SQL queries? That’s exactly what libraries like JSONPath and JSONata do.

If you’re a fan of Reddit, the self-proclaimed “front page of the internet”, you may know of its numerous sub-reddits like r/wallpapers, dedicated to sharing wallpapers.

Reddit has many nifty embedded features. For instance, you can browse multiple sub-reddits at once by combining their names using + characters: r/wallpapers+multiwall+spacewallpapers

To make developers happy, they also allow retrieving a JSON version of the page by appending .json to the end of the URL: r/SpaceWallpapers+multiwall+wallpapers.json

We can then fetch these pages in JavaScript, filter out the Not Safe For Work (NSFW) posts, and get the image URL of a random wallpaper.

We’ll use JSONata since it has more advanced features. They have an online sandbox where you can practice using this library.

JSONata has various built-in functions like $shuffle() that allows us to randomize the wallpapers. Using path operators, we can specify which nested key path to select in order to retrieve the wallpaper URL. We can also filter out unwanted list elements using expressions within square brackets.

The final JSON query looks like this:

$shuffle(data.children[data.over_18 = false])[0].data.preview.images[0].source.url

Let’s take a look at an example in Node.js. You can try it online using O’Reilly’s Katacoda playground.

const jsonata = require("jsonata");
const fetch = require("node-fetch");

async function fetchWallpapers(subreddits) {
    const url = `https://www.reddit.com/r/${subreddits.join('+')}.json`;
    const response = await fetch(url);
    const data = await response.json();
    return data;
}

async function getRandomWallpaper(subreddits) {
    const data = await fetchWallpapers(subreddits);
    const expression = jsonata("$shuffle(data.children[data.over_18 = false])[0].data.preview.images[0].source.url");
    const randomWallpaper = expression.evaluate(data);
    if (randomWallpaper.length) {
        // Reddit encodes HTML entities such as & as & in its URLs
        return randomWallpaper.replace(/&/g, "&");
    } else {
        return null;
    }
}

getRandomWallpaper(['wallpapers', 'multiwall', 'spacewallpapers']).then(function(randomWallpaper){
    console.log(randomWallpaper);
})

/*
Output:
https://preview.redd.it/7ocoihoonlm61.jpg?auto=webp&s=24b52119db8311f4963147349d870a7e0f7f355a
*/

It prints out a random wallpaper URL, which happened to be the cute kitty featured on this post.

This post was originally published on my blog where I write all about tech.