DEV Community

Vee Satayamas
Vee Satayamas

Posted on

JSON Manipulation

There are many ways to manipulate JSON. I reviewed a few rapid ways today, which are using a command line tool called jq and libraries that support JSONPath query language.

jq

Awk is a powerful domain-specific language for text processing, but it lacks built-in support for manipulating hierarchical data structures like trees. jq fills this gap by providing a tool for transforming JSON data. However, one potential drawback is that jq requires a separate installation, which may add complexity to my workflow.

So I write a shell script using jq to extract user's names and user's urls from a cURL response, and then output it in TSV format.

curl -s 'https://mstdn.in.th/api/v1/timelines/public?limit=10' | jq '.[].account | [.username, .url] | @tsv' -r
Enter fullscreen mode Exit fullscreen mode

The result looks like this:

kuketzblog      https://social.tchncs.de/@kuketzblog
cats    https://social.goose.rodeo/@cats
AlJazeera       https://flipboard.com/@AlJazeera
TheHindu        https://flipboard.com/@TheHindu
GossiTheDog     https://cyberplace.social/@GossiTheDog
kuketzblog      https://social.tchncs.de/@kuketzblog
weeklyOSM       https://en.osm.town/@weeklyOSM
juanbellas      https://masto.es/@juanbellas
noborusudou     https://misskey.io/@noborusudou
jerryd  https://mastodon.social/@jerryd
Enter fullscreen mode Exit fullscreen mode

With a TSV file, we can use Awk, sed, etc. to manipulate them as usual.

JSONPath

JSONPath, which was explained in RFC 9535, is supported by many libraries and applications, e.g. PostgreSQL. Still, I try to it in Python by the jsonpath_nq library.

from jsonpath_ng import parse
import requests


res = requests.get("https://mstdn.in.th/api/v1/timelines/public?limit=10")

compiled_path = parse("$[*].account")


for matched_node in compiled_path.find(res.json()):
    print(matched_node.value["username"] + "\t" + matched_node.value["url"])
Enter fullscreen mode Exit fullscreen mode

It gave the same result to the shell script above. The code is a bit longer than the shell script. However, it can be integrated with many Python libraries.

API Trace View

Struggling with slow API calls?

Dan Mindru walks through how he used Sentry's new Trace View feature to shave off 22.3 seconds from an API call.

Get a practical walkthrough of how to identify bottlenecks, split tasks into multiple parallel tasks, identify slow AI model calls, and more.

Read more →

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

👋 Kindness is contagious

Please leave a ❤️ or a friendly comment on this post if you found it helpful!

Okay