There may be times when you have to work with some 'data' that is stored in a local JSON file as opposed to having a RESTful API. One such occurrence of this might be if someone exported a CSV from some spreadsheet and then it was converted to JSON. Now, as a JS Dev you might be tasked with performing some sort of data transformation.
For this article, we will have some 'catalog data' in JSON like so:
[
{
"name": "Hammer",
"desc": "A 🔨",
"price": 1.5
},
{
"name": "Chainsaw",
"desc": "Cut up 🧟♂️s.",
"price": 11.5
},
{
"name": "Frying Pan",
"desc": "For 👨🏽🍳ing 🥘.",
"price": 10.5
},
{
"name": "Spatula",
"desc": "Use it for grilling.",
"price": 7.5
},
{
"name": "Airplane",
"desc": "For flying around!",
"price": 100000000.5
},
{
"name": "Car",
"desc": "For driving.",
"price": 10000.5
}
]
Our task is to read this data from the JSON file, iterate over this data and add an id 🔑to each 'item.' We then need to write the updated JSON back to the file. In order to accomplish this task (and for the purposes of understanding the rest of this post), we need to already be somewhat familiar with:
- Using
forEach()to iterate and mutate Arrays. - functions written in arrow syntax.
We'll also be using import and reading and writing to the file using NodeJS, but as long as you understand how to work with Promises and async, it should be 🙆🏽♂️.
Along the way, I'll explain IIFEs and show when we need to use the 'mjs' file extension.
Read The Data from the Local JSON File
Our data is in './db.json.' Let's read this data with some help from Node!
import the 'Promises-based' fs from Node
We'll need to import Node's File System modules to allow us to read/write the JSON from 'db.json.' We'll specify that we want to use the more modern 'Promise-based' 'fs'. An explanation of asynchronicity and promises is well beyond the scope of this post.
Suffice to say that we will use Promises via the keywords: async and await to await the results of our read/write operations without blocking any other operations that may need to take place in the meanwhile. Generally, this approach is favored over traditional callbacks in modern JS, but was unavailable n Node until recently.
For convenience we are going to 'rename' that import fs (that's what as does).
import { promises as fs } from "fs";
Summarily, this says, "👋🏽JS! Go Look in Node's fs module and only give me the part named promises. Also, when you import that in, I want to refer to that module as fs directly and not promises"
Immediately Invoked Function Expression (IIFE)
As soon as we run our script, we want our function to get to work (be invoked)...immediately. And, since we are going to be using await, we need to specify that our function expression will be running asynchronously by prefacing it with the keyword: async.
(async function() {})();
This function is anonymous (no name specified/needed) and it's body is currently empty. There is no code (yet) inside of its scope (established by the {}s).
Read in the JSON and parse() it into a JS Object
We're going to read the JSON in asynchronously using async await. As soon as the file is read, we'll use JSON.parse() to turn the 'raw' JSON into a 'real' JS Object, and assign the results to a variable data. We'll log that to confirm that it works.
(async function() {
const data = JSON.parse(await fs.readFile("./db.json"));
console.log(data);
})();
'mjs' file vs 'js'
If we were to run this from our command line with: node index.js, we'll get yelled for trying to use an import: SyntaxError: Cannot use import statement outside a module. Since we are building up a whole app here, the simplest way to solve this is to rename 'index.js' to 'index.mjs.' This will allow us to work with 'experimental Node stuff' like imports.
[
{ name: 'Hammer', desc: 'A 🔨', price: 1.5 },
{ name: 'Chainsaw', desc: 'Cut up 🧟♂️s.', price: 11.5 },
{ name: 'Frying Pan', desc: 'For 👨🏽🍳ing 🥘.', price: 10.5 },
{ name: 'Spatula', desc: 'Use it for grilling.', price: 7.5 },
{ name: 'Airplane', desc: 'For flying around!', price: 100000000.5 },
{ name: 'Car', desc: 'For driving.', price: 10000.5 }
]
Mutate our Data
We'll use forEach to iterate over data and add a new 🔑, id to each one. Here, that id will be 1 more than the index of the item. So, the first item's id will be 1, and so on.
data.forEach((d, i) => {(d.id = i + 1)});
forEach takes a callback function with the first parameter, d representing each individual item inside of our data Array. The second parameter, i represents the current index of each item in the Array as we iterate. This starts at 0, so that's why we add 1 to each i.
We iterate over data one item at a time (forEach) ( d) and also look at its index, i. We then add a new 🔑, id and set (=)its value to be...one more than the current index with: i + 1.
We are using arrow syntax, we can omit the function keyword.
[
{ name: 'Hammer', desc: 'A 🔨', price: 1.5, id: 1 },
{ name: 'Chainsaw', desc: 'Cut up 🧟♂️s.', price: 11.5, id: 2 },
{
name: 'Frying Pan',
desc: 'For 👨🏽🍳ing 🥘.',
price: 10.5,
id: 3
},
{ name: 'Spatula', desc: 'Use it for grilling.', price: 7.5, id: 4 },
{
name: 'Airplane',
desc: 'For flying around!',
price: 100000000.5,
id: 5
},
{ name: 'Car', desc: 'For driving.', price: 10000.5, id: 6 }
]
Write The Data Back to the Local JSON File
Now we need to get this data written back to './db.json.' But we want to write JSON back - not a JS Object. To 'convert' our Object into JSON, we use JSON.stringify().
fs.writeFile("./db.json", JSON.stringify(data))
.then(() => {
console.log("Rote new data!");
})
.catch(error => {
console.error(`Error riting data: ${error}`);
});
writeFile needs to know where to write to "./db.json" and what we want to write (the 'string-ified' data).
As mentioned previously 👆🏽, we are using Promises. We don't need to assign our results to any variable, so rather than awaiting, we'll chain a typical then() and catch() and just 'print' some feedback to the console.
🏃🏽♂️our script with node index.js updates './db.json':
[{"name":"Hammer","desc":"A 🔨","price":1.5,"id":1},{"name":"Chainsaw","desc":"Cut up 🧟♂️s.","price":11.5,"id":2},{"name":"Frying Pan","desc":"For 👨🏽🍳ing 🥘.","price":10.5,"id":3},{"name":"Spatula","desc":"Use it for grilling.","price":7.5,"id":4},{"name":"Airplane","desc":"For flying around!","price":100000000.5,"id":5},{"name":"Car","desc":"For driving.","price":10000.5,"id":6}]
Format JSON
We can improve the readability of our JSON by utilizing stringify()'s optional parameters: JSON.stringify(data, null, 2). The second argument null just accepts the 'default implementation' of the method, which 'converts' all of our 'data.' The third argument, 2 specifies a '2 space tab' in our resulting JSON, cleaning things up! 🎨
Now, after re-🏃🏽♂️our script, './db.json' looks like:
[
{
"name": "Hammer",
"desc": "A 🔨",
"price": 1.5,
"id": 1
},
{
"name": "Chainsaw",
"desc": "Cut up 🧟♂️s.",
"price": 11.5,
"id": 2
},
{
"name": "Frying Pan",
"desc": "For 👨🏽🍳ing 🥘.",
"price": 10.5,
"id": 3
},
{
"name": "Spatula",
"desc": "Use it for grilling.",
"price": 7.5,
"id": 4
},
{
"name": "Airplane",
"desc": "For flying around!",
"price": 100000000.5,
"id": 5
},
{
"name": "Car",
"desc": "For driving.",
"price": 10000.5,
"id": 6
}
]
Here's the code for this post: https://github.com/manavm1990/read-transform-write-json-file
In the repo I wrap up our await in a try-catch as is a common practice for catching any errors. That's not overly relevant to the purpose of this post, so I won't mention it again.
Top comments (1)
Hii
You can also use onlinejsontools.org/ for json validator,beautify,minify,xml,yaml,CSV,bson,plain text,base64,tsv.
Do checkout this site!