Upload files to MongoDB GridFS with Express

abskmj — Thu, 01 Oct 2020 08:01:54 +0000

Create express based APIs to upload and download files to and from MongoDB GridFS.

GridFS is a specification for storing and retrieving files that exceed the BSON-document size limit of 16 MB.

Install Dependencies

express to create the APIs
multer to handle multi-part file uploads
mongoose to manage connections to MongoDB
gridfile to manage interactions with GridFS

GridFile Mongoose Model

GridFile is a reusable Mongoose Schema for MongoDB GridFS.

// gridfile.model.js

const mongoose = require('mongoose')
const schema = require('gridfile')

module.exports = mongoose.model('GridFile', schema)

Multer Middleware

Multer will parse multipart/form-data request and the uploaded files will be accessible as req.files

const multer = require('multer')
const upload = multer({ dest: path.join(__dirname, '.') })

Upload File API

API uses the multer middleware and GridFile model to upload files to GridFS.

app.post('/v1/files', upload.any(), async (req, res, nxt) => {
  try {
    // uploaded file are accessible as req.files
    if (req.files) {
      const promises = req.files.map(async (file) => {
        const fileStream = fs.createReadStream(file.path)

        // upload file to gridfs
        const gridFile = new GridFile({ filename: file.originalname })
        await gridFile.upload(fileStream)

        // delete the file from local folder
        fs.unlinkSync(file.path)
      })

      await Promise.all(promises)
    }

    res.sendStatus(201)
  } catch (err) {
    nxt(err)
  }
})

List Files API

API returns information about uploaded files.

app.get('/v1/files', async (req, res, nxt) => {
  try {
    const files = await GridFile.find({})

    res.json(files)
  } catch (err) {
    nxt(err)
  }
})

Sample response

[
  {
    "aliases": [],
    "_id": "5f6850023516552ad21d0007",
    "length": 7945,
    "chunkSize": 261120,
    "uploadDate": "2020-09-21T07:02:26.389Z",
    "filename": "attachment.pdf",
    "md5": "fa7d7e650b2cec68f302b31ba28235d8"
  }
]

Download File API

API returns the file from the GridFS using its id.

app.get('/v1/files/:id/:name', async (req, res, nxt) => {
  try {
    const { id, name } = req.params

    const gridFile = await GridFile.findById(id)

    if (gridFile) {
      res.attachment(name)
      gridFile.downloadStream(res)
    } else {
      // file not found
      res.status(404).json({ error: 'file not found' })
    }
  } catch (err) {
    nxt(err)
  }
})

Sample Request URL

/v1/files/5f6850023516552ad21d0007/attachment.pdf

A working codebase for reference is available at gist.github.com

Publish to Snapcraft Store

abskmj — Wed, 16 Sep 2020 07:41:43 +0000

Publish Kustomize automatically to Snapcraft Store

I usually work with Kubernetes clusters for deployment and a tool called kustomize to manage the manifests.

I recently started using Snapcraft apps on Ubuntu and found that Kustomize is not available in their store. So, I decided to put together a Github action that will publish automatically the latest version of Kustomize to the Snapcraft store.

The Github Action is scheduled to run daily. It checks if there is a new version of Kustomize available on their Github Repository. If so, bundle and publish it as a snap to the Snapcraft Store. Kustomize is popular on Snapcraft Store. https://snapcraft.io/kustomize

Submission Category: DIY Deployments

Yaml File or Link to Code: https://github.com/abskmj/snapcraft-kustomize

Additional Resources / Info

The core implementation is a NodeJS script.

DEV Community: abskmj