Node.js: Unzip Async Await

#javascript #node #webdev #programming

I am developing a new feature of DeckDeckGo for which I have to unzip a data in Firebase Functions.

It took more time than expected to code such a Node.js function, that’s why I am sharing this solution, hoping it might help you some day too 😇.

Unzipper

Node.js provides a compression module Zlib but, it does not support ZIP files. Luckily, we can use the library unzipper to handle these.

npm i unzipper --save

Unzip With Async Await

My new feature reads and writes data uploaded in Firebase Storage through streams. I also develop my code with a promises (async / await) approach. Therefore, both have to coexist.

To narrow down the following example, I replaced the cloud storage with local files handled with file system streams (fs ).

The function unzip instantiates a stream on the zip data which is piped with unzipper . Each entry are iterated and piped themselves to writable outputs. Summarized: the zip is opened and each files it contains are extracted.

unzip is called in a retro compatible top await function and, that’s basically it 🥳.

const {Parse} = require('unzipper');
const {createWriteStream, createReadStream} = require('fs');

const unzip = () => {
  const stream = 
    createReadStream('/Users/david/data.zip').pipe(Parse());

  return new Promise((resolve, reject) => {
    stream.on('entry', (entry) => {
      const writeStream = 
        createWriteStream(`/Users/david/${entry.path}`);
      return entry.pipe(writeStream);
    });
    stream.on('finish', () => resolve());
    stream.on('error', (error) => reject(error));
  });
};

(async () => {
  try {
    await unzip();
  } catch (err) {
    console.error(err);
  }
})();

Read To String With Async Await

I had to read files with streams too. Consequently and cherry on top, here is how I integrated these in my code.

const {createReadStream} = require('fs');

const read = () => {
  const stream = 
    createReadStream('/Users/david/meta.json');

  return new Promise((resolve, reject) => {
    let data = '';

    stream.on('data', (chunk) => (data += chunk));
    stream.on('end', () => resolve(data));
    stream.on('error', (error) => reject(error));
  });
};

(async () => {
  try {
    const meta = await read();

    console.log({meta});
  } catch (err) {
    console.error(err);
  }
})();

It follows the same approach as previously and read the file content to an in memory string.

Summary

Coding is like a box of chocolates. You never know what you’re gonna get. Sometimes it should be quick, it takes time. Sometimes it should take so much time but, it goes fast — For-dev-rest Gump

To infinity and beyond!

David

You can reach me on Twitter or my website.

Give a try to DeckDeckGo for your next slides!

Discussion (4)

Roland Csibrei • Mar 15 '21

Hello! Thanks a lot for sharing! I think all of us will need to zip/unzip data sooner or later. I am working on a PWA which needs to poll data from an API every 5 minutes. When I started to work on the application, the payload was around 20 kB. Nowadays it is around 400 kB and mobile data users started to complain. I can't change the API so I created an AWS lambda, which polls the data and I used your zip function to zip the data and store in a S3 bucket. The PWA now polls the data from this bucket. The zipped file is around 13 kB. And again, unzip with your function. I was already thinking about stripping or packing the payload so your article saved me time. I owe you a drink dude 😂 thanks again and keep posting! R.