DEV Community

Uploading Files to MongoDB with GridFS and Multer Using NodeJS

Shubham Battoo on November 23, 2019

Hello, in this tutorial we will learn how to upload files directly to MongoDB using GridFS specification. If you think TLDR; just check finish cod...

Read full post

DavidPalomeque • Dec 1 '19

The only thing that i can do is thank you for such amazing post . Today I was learning how to manage the images files with nodejs and mongodb , I saw some tutorials but some of them didn´t work and there were others that I couldn´t understand , so I was a little stuck . But your post solved me a lot of doubts and luckily I figure out and could do it .

The only thing that i didn´t get was this part : function readSingleFile(e) {const name = e[0].name;document.getElementById("file-label").textContent = name;}

I´ve not get why is this for . Anyway , the post is amazing , thank you .

Shubham Battoo • Dec 2 '19

Thanks glad this could help.

Basically what that part is doing is that i am getting the file name and displaying it on the input element, its not required just some code to complete the tutorial, can be skipped.

DavidPalomeque • Dec 2 '19

Thanks for the answer !

Krishnaraj Yadav • Oct 5 '20

Error: Cannot find module 'ejs '
Thanks alot. This is very helpful tutorial for me.I am new in this field. I am facing above error and have installed ejs by npm and following line by line your code.
I can upload files but can't retrieve it . After writing ejs code of retrieving files i am getting this error.I need your help.

Shubham Battoo • Oct 5 '20

did you try installing ejs? it can be done using npm, you can run npm install ejs to install ejs

Krishnaraj Yadav • Oct 5 '20

Yes I have done that already and even tried to install it globally.Actually, it just works fine if I pass only json data to index.ejs but error occurs only after typing the below code in index.ejs file:

Code link : dev-to-uploads.s3.amazonaws.com/i/...
Error link : dev-to-uploads.s3.amazonaws.com/i/...

Shubham Battoo • Oct 7 '20

Can you share your GitHub repo, if you have one where you where trying it, i can maybe take a look for you 😀

Krishnaraj Yadav • Oct 8 '20 • Edited

Sure . Here it is
github.com/krishna-y2000/Uploading...
Server file of above code is in /routes/uploadFiles.js and /routes/upload.js.
Client file is in /views/HomePage.ejs and /views/upload.ejs

Krishnaraj Yadav • Oct 9 '20

Update in error :
Please refer this image because the comment box is not responding properly .
dev-to-uploads.s3.amazonaws.com/i/...
You can mail me if you want to text me in detail.
Email : krishnay.75676@gmail.com
Thanks alot for your effort.

Saksham Pardesi • Feb 16 '23

THANK YOU BUDDY FOR MAKING THIS VALUEABLE DOCUMENT

Shubham Battoo • Feb 16 '23

Glad it helps

Syed Zubair(zubi-droid) • May 27 '20

Thank you for this post sir.. Now I can upload my files to my mongo-atlas cluster successfully. Its very kind of you on my reach to this. I can now see the uploaded collections binary data on the collections (upl.files , upl.chunks) in mongo-atlas.

But I can't find a way to download a pdf file that i uploaded sir...Can u please help me on how to download a uploaded pdf file to my database sir. :{ Help sir.

Shubham Battoo • May 28 '20

Hi Syed, you can do something like this to download a pdf or any other file for this matter

<a href="image/<%= file.filename %>" download><%= file.filename %></a>

I have created a route "image/:filename" which then downloads the image which is not limited to a image and can handle any other file types also, but for this demo I was only downloading images.

app.get("/image/:filename", (req, res) => {
  // console.log('id', req.params.id)
  const file = gfs
    .find({
      filename: req.params.filename
    })
    .toArray((err, files) => {
      if (!files || files.length === 0) {
        return res.status(404).json({
          err: "no files exist"
        });
      }
      gfs.openDownloadStreamByName(req.params.filename).pipe(res);
    });
});

Syed Zubair(zubi-droid) • May 31 '20 • Edited

Thank you so much for your help sir. What I have to do if i need to download the file that the user uploads from his end, and I have to download that from the database collection but not from the user side itself. if i'm not wrong, Like in google classroom app., we upload our assignments as .pdf to our professor and those attachments reaches him/her in their database(provided by that app.) and they download them on their end..... Is there any way to implement like this, sir. Any help would be appreciated.

Vasanti Suthar • Feb 5 '22 • Edited

Hey, thanks for this blog. I have implemented this but the file is not being able to download it says couldn't download.

staszeksoup • Jan 14 '20

Absolutely Marvelous. Thank you so much sir, I've been really scratching my head with this. My question is, how do you add other parameters? So the image has a name?

Shubham Battoo • Jan 14 '20 • Edited

Image does have a name, when we where setting up the GridFS as per Multer I used crypto and randomBytes to create a filename for the file, you can have your own strategy for naming.

following code was written if you missed it:

// Storage
const storage = new GridFsStorage({
  url: mongoURI,
  file: (req, file) => {
    return new Promise((resolve, reject) => {
      crypto.randomBytes(16, (err, buf) => {
        if (err) {
          return reject(err);
        }
        const filename = buf.toString("hex") + path.extname(file.originalname);
        const fileInfo = {
          filename: filename,
          bucketName: "uploads"
        };
        resolve(fileInfo);
      });
    });
  }
});

const upload = multer({
  storage
});

staszeksoup • Jan 14 '20

Yes, thanks but I mean when the user uploads the image, how can they add other fields? The image will need a unique filename, but can there be a title and description?

Shubham Battoo • Jan 15 '20

Yes sure why not, but then that needs to be in other collection and you can reference the image objectid in the collection where you are storing the meta information for the images

staszeksoup • Jan 16 '20

Yes I see, so you have to save the file location and file name and add that as a string to the other collection. Thanks for your help

Shubham Battoo • Jan 16 '20

File is being saved in its own collection as per gridfs, you can get an objectid for the image uploaded you can reference this objectid

Phillip DaCosta • Dec 11 '20 • Edited

Hello! Thank you for this helpful post! I have an issue I need some help with though.
I was able to successfully upload the images into MongoDB. However, I am not having much success retrieving the images and sending it to the frontend. The error I am currently getting is TypeError: Cannot read property 'find' of undefined ..this is happening at my ' gfs.find().toArray((err, files) => ' line under my 'app.get('/profilePicture' )' request. ..

`
const express = require('express')
const path = require('path')
const crypto = require('crypto')//to generate file name
const mongoose = require('mongoose')
const multer = require('multer')
const GridFsStorage = require('multer-gridfs-storage')
const Grid = require('gridfs-stream')
const app = express()

//CORS Middleware
app.use(function (req, res, next) {
//Enabling CORS
res.header('Access-Control-Allow-Origin', '*');
res.header('Access-Control-Allow-Methods', 'GET,HEAD,OPTIONS,POST,PUT');
res.header('Access-Control-Allow-Headers', 'Origin, X-Requested-With, Content-Type, Accept, x-client-key, x-client-token, x-client-secret, Authorization');
next();
});

const uri = "mongodb+srv://fakeaccount:password@xxxxx.mongodb.net/x_app?retryWrites=lkngdndfg"

let conn = mongoose.connection
var gfs
conn.once('open', () => {
//initialize our stream
gfs = Grid(conn.db, mongoose.mongo)
gfs.collection('employee')
})

let storage = new GridFsStorage({
url: uri,
file: (req, file) => {
return new Promise(
(resolve, reject) => {
const fileInfo = {
filename: file.originalname,
bucketName: "imageUpload"
}
resolve(fileInfo)

        })  }

})

const upload = multer({ storage })

app.post("/uploadImg",upload.single("profilePic"),(req,res)=>{

console.log(req.body)

res.json({file:req.file})
})

app.get('/proficPicture', (req, res) => {

    gfs.find().toArray((err, files) => {
        //check if files exist
        if (!files || files.length == 0) {
            return res.status(404).json({
                err: "No files exist"
            }) }
        // files exist
        return res.json(files)
    })


})

const PORT =5000
app.listen(PORT,()=>console.log(`Server started on port ${PORT}`))

abskmj • Sep 27 '20

Amazing post! I referred to this to implement one of my projects.

Eventually, I put together a reusable Mongoose Schema which further simplifies Mongoose and MongoDB GridFS setup. It is available as an NPM module called GridFile. It also supports the Mongoose schema association and query population.

Here is an example of how to use GridFile to upload/download files to/from MongoDB GirdFS.

Steve Groom • Jul 12 '20 • Edited

Hi,
thanks' for the blog post - its good to have a working example, however I want to take it a little further as I need to add file uploads to my own project and so must refactor your code to fit my app structure. Somewhere along the way I am missing some synchronisation. Upload triggers the file create and immediately shows the home page. If I press refresh the missing image is shown.

I think the error will be in my files.js /upload route - probably the next() is in the wrong place or the fact that I have an app.use nested inside my app.post...

I opened a Stack Overflow question and then saw I could perhaps ask here too :-)
stackoverflow.com/questions/628608...

My ‘fork’ of your repo is here: github.com/stevegroom/redogridfsst...

regards
Steve

Shaheryar Shaikh • Jan 6 '21

why cant we add data more than 16 MB

Shubham Battoo • Jan 9 '21

It is limited from MongoDb end. There are some cases listed down by mongodb where you can use this.

Timothy Dillman • Sep 30 '21

Although this is true for MongoDB itself, GridFS was designed to solve this problem. From Mongo's documentation directly it states: "In MongoDB, use GridFS for storing files larger than 16 MB." Reference: docs.mongodb.com/manual/core/gridfs/

Sumukhakb • Nov 8 '20

I am using react as a front end and I want to upload profile image with other information such as name, email etc. How to do this stuff, Please help me