Reading From CSV and Convert To Json. With Vanilla Node.js

Oluwafemi Durodola — Fri, 04 Nov 2022 04:34:52 +0000

First the CSV file we are reading from looks like this

data.csv

To read the any kind of fille in Node.js we can use fs module and also fs module provide **createReadStream **method which we can use to read data piece by piece, meaning we don’t need to wait for all the file to finished reading before making use of the data.

Now import fs module and access the createReadStream *method inside :
*Server.js

"use strict";

import fs from "fs";

const userDataReadStream = fs.createReadStream(`${process.cwd()}/data.csv`);

Now we are going to add “data” event listener to our userDataReadStream, so that whenever data is read we append/put the data value to a variable .

// ...
let rawData = ``;

userDataReadStream.on("data", (data) => {
  rawData += data;
});

Again we will assign “end” event listener to the same userDataReadStream, inside this “end” listener the data read will be available to us and we can use that data for anything we want here.

// ...
userDataReadStream.on("end", () => {
  console.log(rawData);
});

Now run node server.js inside your terminal to see the result, make sure you are in project directory.

Result

  /* 
    id,name,job          ---> HEAD
    1,john,web           ---> BODY
    2,wale,designer      ---> BODY
    3,femi,printer       ---> BODY
  */

As we can see that each column of the excel file is the result of each separate new line

The first line of the result will be the HEAD and others new line will be the BODY.

Next step is to convert our data to array using split() method and log it out to the console.

// ...
userDataReadStream.on("end", () => {
  const arrayData = rawData.split("\n");

  console.log(arrayData); // [ 'id,name,job', '1,john,web', '2,wale,designer', '3,femi,printer' ]
});

Now that our data’s in array format, lets separate HEAD from the BODY using destructuring and also log it out

// ...
userDataReadStream.on("end", () => {
  const arrayData = rawData.split("\n");
  const [stringHeadData, ...arrayBodyData] = arrayData;

  console.log(stringHeadData); // id,name,job
  console.log(arrayBodyData); // [ '1,john,web', '2,wale,designer', '3,femi,printer' ]
});

Notice our HEAD data comes back as string lets change the HEAD back to an array and log it out

// ...
userDataReadStream.on("end", () => {
  const arrayData = rawData.split("\n");
  const [stringHeadData, ...arrayBodyData] = arrayData;

  const arrayHeadData = stringHeadData.split(",");

  console.log(arrayHeadData); // [ 'id', 'name', 'job' ]
});

Also if you check the BODY array, you will see that each index of the array consist of the column inside of string, so now lets make each column to be in there own array and log it out also, the map method will return array and the split method will put each value in there own index wherever the “,” end.

// ...
userDataReadStream.on("end", () => {
  //...
  const bodyDatas = arrayBodyData.map((body) => body.split(","));
  console.log(bodyDatas);
});

Result

[
  [ '1', 'john', 'web' ],
  [ '2', 'wale', 'designer' ],
  [ '3', 'femi', 'printer' ]
]

Now we done preparing our data’s, so now lets create a file name “lib.js” this is where we will put our function to convert the HEAD array to OBJECT KEY and the BODY array to OBJECT VALUE

lib.js

export function arrToObjectData(arrHeader, arrBody) {
  const data = arrHeader.reduce((prevValue, curValue, curIndex, arr) => {
    return { ...prevValue, [curValue]: arrBody[curIndex] };
  }, {});

  return data;
}

We are using reduce method here and we loop through the HEAD **array and make each index value of the array to be the **OBJECT KEY and also our BODY **array we take each index value to be our **OBJECT VALUE.

I wish I can explain better 💔.

Now lets import this functions to our server.js

server.js

"use strict";

import fs from "fs";

import { arrToObjectData } from "./lib";

Lets create an empty array, this is where all our data object will be. And also we will loop through our BODY array, so that we can able to repeat our **arrToObjectData() **method for the length our array and push each object to our empty array variable we created and log the array out.

// ...
userDataReadStream.on("end", () => {
  const arrayData = rawData.split("\n");
  const [stringHeadData, ...arrayBodyData] = arrayData;
  const arrayHeadData = stringHeadData.split(",");
  const bodyDatas = arrayBodyData.map((body) => body.split(","));

  // Create empty array to put all our data object
  const userData = [];

  // Loop through the BODY array and pass the array at each index to our arrToObjectData method then push it back to our userData empty array variable
  for (let i = 0; i < bodyDatas.length; i++) {
    userData.push(arrToObjectData(arrayHeadData, bodyDatas[i]));
  }

  // log the array out
  console.table(userData);
});

Result

┌─────────┬─────┬────────┬────────────┐
│ (index) │ id  │  name  │    job     │
├─────────┼─────┼────────┼────────────┤
│    0    │ '1' │ 'john' │   'web'    │
│    1    │ '2' │ 'wale' │ 'designer' │
│    2    │ '3' │ 'femi' │ 'printer'  │
└─────────┴─────┴────────┴────────────┘

Now lets save the data object to a json file

// ...
userDataReadStream.on("end", () => {
  //...

  // Save the users data to a json file
  fs.writeFile(`${process.cwd()}/users_data/userdatas.json`, JSON.stringify(userData), (err) => {
    if (err) return console.error(err);

    console.log(`Data written successfully`);
  });
});

Result
userdatas.json

[
    {
        "id": "1",
        "name": "john",
        "job": "web"
    },
    {
        "id": "2",
        "name": "wale",
        "job": "designer"
    },
    {
        "id": "3",
        "name": "femi",
        "job": "printer"
    }
]

THANK YOU READING, please I need feedback for any mistake or wrong use of code.

DEV Community: Oluwafemi Durodola

Reading From CSV and Convert To Json. With Vanilla Node.js