From JSONL to a normalized relational database. Is a tool necessary?

Keto Man — Fri, 11 Jul 2025 13:36:51 +0000

This is my first post, please forgive me if it’s not the most beautiful post you’ll read. 😃

In my career as a developer, I often find myself spending a lot of time on the lengthy and resource-intensive task of extracting data from JSON files and placing them into relational databases so that data querying for analysts and others is facilitated by the use of SQL.

These are very time-consuming activities with a fairly high risk of making errors if not handled correctly.

For this reason, I have created a tool that automatically performs this task.🥳

What does this tool consist of?
In practice, JSONs are saved as JSONL within a text file.
Each individual line is analyzed to understand the structure of all the JSON paths within them.
Once this analysis is complete, a script creates the necessary structures for a PostgreSQL database (and soon for Oracle, Snowflake, etc.).

The result is an SQL script to be executed on the database, providing ready-to-use structures for the JSON data.

I have saved countless hours compared to before, and above all, if I receive any JSON input today, I no longer have to worry about its structure or writing the linking keys between the various tables.

You should know, in fact, that I have very complex JSONs, with up to 15 hierarchical levels, and... it works perfectly.

The interesting aspect of this utility is that it also automatically adapts to changes. Any new branches or variations are automatically detected, maintaining backward compatibility with already loaded data. You just need to rerun the script if changes are reported.

This utility also works if the input refers to XML!

Now the question I have for you is this: is this utility only useful to me, or are there other developers who might find it necessary? Let me know because I'm curious. 😃

Hello everyone. In your opinion, does it make sense to have a tool that automatically extracts data from JSON files, without first knowing the structure of the JSON schema, and transforms this data into a normalized relational database? How many of you mig

Keto Man — Fri, 11 Jul 2025 13:08:00 +0000

DEV Community: Keto Man

From JSONL to a normalized relational database. Is a tool necessary?

Hello everyone. In your opinion, does it make sense to have a tool that automatically extracts data from JSON files, without first knowing the structure of the JSON schema, and transforms this data into a normalized relational database? How many of you mig