DEV Community

loading...

AWS SDK with Javascript: Multi-Files Download from S3

ldsrogan
I'm super interested in cool technologies!
・2 min read

If you want to download multiple files as zipped from AWS S3, and you have your server to take care of the file metadata handlings, then this article may help you understand how the process works. AWS doesn't provide the default multi-files download, so in order to achieve you may want to add lambda function or to use your own implemented service. This article uses the custom service to support multi-files download function.

To know the basic setup of the code (AWS keys, the meaning of Key in the code, bucket, etc.), please refer to another article that I worte AWS SDK with Javascript: Download File from S3.

import * as AWS from 'aws-sdk';
import archiver from 'archiver';
import path from 'path'
import { PassThrough } from 'stream';

const s3bucket = new AWS.S3({
  accessKeyId: process.env.AWS_ACCESS_KEY,
  secretAccessKey: process.env.AWS_SECRET_KEY,
  signatureVersion: 'v4',
  region: process.env.AWS_REGION, // ex) us-west-2
});
Enter fullscreen mode Exit fullscreen mode

Here is the main function that allows to receive file infos as an input parameter and to create stream object that can be used to write or pass the data as zipped format.

const multiFilesStream = (infos) => {
  // using archiver package to create archive object with zip setting -> level from 0(fast, low compression) to 10(slow, high compression) 
  const archive = archiver('zip', { zlib: { level: 5 } });

  for (let i = 0; i < infos.length; i += 1) {
    // using pass through stream object to wrap the stream from aws s3
    const passthrough = new PassThrough();
    s3bucket
      .getObject({
        Bucket: process.env.AWS_BUCKET_NAME,
        Key: path.join(infos[i].path, infos[i].filename);
      })
      .createReadStream()
      .pipe(passthrough);
    // name parameter is the name of the file that the file needs to be when unzipped.
    archive.append(passthrough, { name: infos[i].filename });
  }
  return archive;
};

Enter fullscreen mode Exit fullscreen mode
const files = [
   {path: '/drive', filename: 'file1.jpg'},
   {path: '/drive', filename: 'file2.jpg'},
   {path: '/drive', filename: 'file3.jpg'},
   {path: '/drive', filename: 'file4.jpg'},
];

const mfStream = multiFilesStream(files);

// res is the response object in the http request. You may want to create your own write stream object to write files in your local machine
mfStream.pipe(res);

// use finalize function to start the process
mfStream.finalize();
Enter fullscreen mode Exit fullscreen mode

Discussion (0)