DEV Community

loading...

Uploading files to AWS S3 using NodeJs, understand thought sequence diagram.

Rahul kumar
I am a computer science engineer by passion.
・3 min read

Amazon S3 or Amazon Simple Storage Service is a service offered by Amazon Web Services that provides object storage through a web service interface. Amazon S3 uses the same scalable storage infrastructure that Amazon.com uses to run its global e-commerce network.

The scope of this article is to understand and learn how can we upload files on S3 using nodejs

Github Repo

Prerequisites

  1. NodeJs
  2. ExpressJs
  3. Multer

sequence-diagram

Here, as you can see in the above diagram. The request will be initiated by the browser by selecting the file from local file system, then send it to the server. Then server will receive the file and it also sends it to the AWS S3 bucket. Once S3 will process the request then it'll send the status to server, and server will send status to the bowser.

NodeJs Server

I am going to use express.Router(), if you don't know then you can refer here. To receive the file at nodejs server, i'll use Multer

let multer  = require('multer')
let upload = multer({dest: 'kyc/'});
const S3 = require('./S3');

router.post("/upload",upload.single('user__profile__image'),async(req,res)=>{
    try{
        let docLoc = await S3.uploadDoc(req.file,'client/profile',req.user.cid);
        res.send(docLoc);
    }catch(e){
        console.log(e);
        res.status(500).send(err);
    }
});
Enter fullscreen mode Exit fullscreen mode

You can see i have defined a /upload route to upload the file and S3.uploadDoc is a function which will upload the given file to S3. Once S3.uploadDoc executes without any error, then it'll resolve to a string docLoc location of the file inside the S3 bucket.

S3.uploadDoc

Now, let's build the function to upload the file into S3 bucket. I'll use aws-sdk to work with AWS services. There is some sequence of steps involved while uploading the fine into S3 bucket.

  1. update AWS config
  2. get s3 bucket
  3. create uploadParams and select location
  4. createReadStreamto read the file
  5. upload the file
  6. return the result

update AWS config

AWS.config.update({
    region: process.env.REGION,
    accessKeyId: process.env.ACCESS_KEY,
    secretAccessKey: process.env.SECRET_KEY,
});
Enter fullscreen mode Exit fullscreen mode

get s3 bucket
Select the S3 bucket where you want to upload the file.

  const s3 = new AWS.S3({
    apiVersion: "2006-03-01",
    params: { Bucket: "resources-logistics" },
  });
Enter fullscreen mode Exit fullscreen mode

create uploadParams and select location

const uploadParams = {
  // the directory where the file will be uploaded
  Bucket: `resources-logistics/profile`,
  Key: "",
  Body: "",
};
Enter fullscreen mode Exit fullscreen mode

createReadStreamto read the file

// doc.path is the file path in you local file system
let fileStream = fs.createReadStream(doc.path);
Enter fullscreen mode Exit fullscreen mode

Full code

const AWS = require("aws-sdk");
const fs = require("fs");
function uploadDoc(doc, preset, cid) {
  let originalname = doc.originalname.split(".");

  // get the file extenstion ,like .jpg, .img, etc
  const docExt = originalname[originalname.length - 1];

  // update AWS config
  AWS.config.update({
    region: process.env.REGION,
    accessKeyId: process.env.ACCESS_KEY,
    secretAccessKey: process.env.SECRET_KEY,
  });

  const s3 = new AWS.S3({
    apiVersion: "2006-03-01",
    params: { Bucket: "resources-logistics" },
  });

  return new Promise((resolve, reject) => {
    const uploadParams = {
      Bucket: `resources-logistics/${preset}`,
      Key: "",
      Body: "",
    };

    // this fileStream will stream the file from local file system to the S3
    let fileStream = fs.createReadStream(doc.path);

    // if there is any error while reading the file from local file system or streaming it
    // then reject the promise with the err object
    fileStream.on("error", function (err) {
      reject(err);
    });

    // set body as stream
    uploadParams.Body = fileStream;

    // for key check the docs
    uploadParams.Key = cid
      ? `${cid}${originalname
          .slice(0, originalname.length - 1)
          .join("-")}.${docExt}`
      : doc.originalname;

    // upload the file
    s3.upload(uploadParams, function (err, data) {
      if (err) {
        reject(err);
      }
      if (data) {
        resolve(data);
      }
    });
  });
}
module.exports = {
  uploadDoc,
};
Enter fullscreen mode Exit fullscreen mode

If you liked it then give it a star https://github.com/ats1999/AWS-S3

Discussion (0)