DEV Community

loading...

FileUpload with NestJS using MinIO

Edmar Diaz
Fullstack JavaScript Developer working with React, ReactNative, Ionic(3,4,5) Angular <3, NestJS <3, NodeJS, TS <3 and #geekineers
・6 min read

Introduction

Hello guys, since we are in a pandemic crisis due to coronavirus. Most of us are working from home and today I've decided to make a simple guide about FileUpload with NestJs and MinIO. This is going to be my first post on Dev.to.

Before I start the guide let me first give you a background about the technology that we are going to use in this post.

What is MinIO?

Minio is an open source object storage server released under Apache License V2. It is compatible with Amazon S3 cloud storage service. Minio follows a minimalist design philosophy.

Minio is light enough to be bundled with the application stack. It sits on the side of NodeJS, Redis, MySQL and the likes. Unlike databases, Minio stores objects such as photos, videos, log files, backups, container / VM images and so on. Minio is best suited for storing blobs of information ranging from KBs to TBs each. In a simplistic sense, it is like a FTP server with a simple get / put API over HTTP.

What is NestJS?

Nest (NestJS) is a server-side (backend) application framework beautifully crafted to support developers productivity and make their lives happier.

It uses progressive JavaScript, is built with and fully supports TypeScript (yet still enables developers to code in pure JavaScript) and combines elements of OOP (Object Oriented Programming), FP (Functional Programming), and FRP (Functional Reactive Programming).

It is heavily inspired by common libraries and frameworks such as Angular and Java Spring Boot which improve developer productivity and experience.

Let's Get Started

Before starting make sure you have the following installed and configured:

Create a new project on nest-cli:

 nest new minio-fileupload
Enter fullscreen mode Exit fullscreen mode

You will be asked to choose between yarn or npm as package manager. Feel free to choose what suits you.

We will be using nestjs-minio-client as our nestjs minio client:

via NPM:
npm install nestjs-minio-client --save
Enter fullscreen mode Exit fullscreen mode
via YARN:
yarn add nestjs-minio-client
Enter fullscreen mode Exit fullscreen mode

After installing the nestjs-minio-client, lets generate our module, service on nest-cli:

nest g module minio-client
nest g service minio-client
Enter fullscreen mode Exit fullscreen mode

After generating module and service let's create a config file:

touch src/minio-client/config.ts
Enter fullscreen mode Exit fullscreen mode

Inside the config.ts (this is my MinIO configuration. You must use yours):

export const config = {
  MINIO_ENDPOINT: 'localhost',
  MINIO_PORT: 9001,
  MINIO_ACCESSKEY: 'AKIAIOSFODNN7EXAMPLE',
  MINIO_SECRETKEY: 'wJalrXUtnFEMIK7MDENGbPxRfiCYEXAMPLEKEY',
  MINIO_BUCKET: 'test'
}
Enter fullscreen mode Exit fullscreen mode

Let us now register and configure our MinioModule and export our MinioService on our minio-client.module.ts file:

import { Module } from '@nestjs/common';
import { MinioClientService } from './minio-client.service';
import { MinioModule } from 'nestjs-minio-client';
import { config } from './config'
@Module({
  imports: [
    MinioModule.register({
      endPoint: config.MINIO_ENDPOINT,
      port: config.MINIO_PORT,
      useSSL: false,
      accessKey: config.MINIO_ACCESSKEY,
      secretKey: config.MINIO_SECRETKEY,
    })
  ],
  providers: [MinioClientService],
  exports: [MinioClientService]
})
export class MinioClientModule {}

Enter fullscreen mode Exit fullscreen mode

Now that we register our MinioModule let's start creating our file models:

touch src/minio-client/file.model.ts
Enter fullscreen mode Exit fullscreen mode

Here is the code for our file.model.ts:

export interface BufferedFile {
  fieldname: string;
  originalname: string;
  encoding: string;
  mimetype: AppMimeType;
  size: number;
  buffer: Buffer | string;
}

export interface StoredFile extends HasFile, StoredFileMetadata {}

export interface HasFile {
  file: Buffer | string;
}
export interface StoredFileMetadata {
  id: string;
  name: string;
  encoding: string;
  mimetype: AppMimeType;
  size: number;
  updatedAt: Date;
  fileSrc?: string;
}

export type AppMimeType =
  | 'image/png'
  | 'image/jpeg';
Enter fullscreen mode Exit fullscreen mode

Nest step, inside our minio-client.service.ts:

import { Injectable, Logger, HttpException, HttpStatus } from '@nestjs/common';
import { MinioService } from 'nestjs-minio-client';
import { Stream } from 'stream';
import { config } from './config'
import { BufferedFile } from './file.model';
import * as crypto from 'crypto'

@Injectable()
export class MinioClientService {
    private readonly logger: Logger;
    private readonly baseBucket = config.MINIO_BUCKET

  public get client() {
    return this.minio.client;
  }

  constructor(
    private readonly minio: MinioService,
  ) {
    this.logger = new Logger('MinioStorageService');
  }

  public async upload(file: BufferedFile, baseBucket: string = this.baseBucket) {
    if(!(file.mimetype.includes('jpeg') || file.mimetype.includes('png'))) {
      throw new HttpException('Error uploading file', HttpStatus.BAD_REQUEST)
    }
    let temp_filename = Date.now().toString()
    let hashedFileName = crypto.createHash('md5').update(temp_filename).digest("hex");
    let ext = file.originalname.substring(file.originalname.lastIndexOf('.'), file.originalname.length);
    const metaData = {
      'Content-Type': file.mimetype,
      'X-Amz-Meta-Testing': 1234,
    };
    let filename = hashedFileName + ext
    const fileName: string = `${filename}`;
    const fileBuffer = file.buffer;
    this.client.putObject(baseBucket,fileName,fileBuffer,metaData, function(err, res) {
      if(err) throw new HttpException('Error uploading file', HttpStatus.BAD_REQUEST)
    })

    return {
      url: `${config.MINIO_ENDPOINT}:${config.MINIO_PORT}/${config.MINIO_BUCKET}/${filename}` 
    }
  }

  async delete(objetName: string, baseBucket: string = this.baseBucket) {
    this.client.removeObject(baseBucket, objetName, function(err, res) {
      if(err) throw new HttpException("Oops Something wrong happend", HttpStatus.BAD_REQUEST)
    })
  }
}

Enter fullscreen mode Exit fullscreen mode

If you notice on my metaData object i have X-Amz-Meta-Testing this is just a sample if you want to add your custom metadata.

At this point, our minio-client module is now ready, to test it lets make a new module with service and controller:

nest g module file-upload
nest g service file-upload
nest g controller file-upload
Enter fullscreen mode Exit fullscreen mode

Now let's open our src/file-upload/file-upload.module.ts and import our MinioClientModule:

import { Module } from '@nestjs/common';
import { FileUploadService } from './file-upload.service';
import { FileUploadController } from './file-upload.controller';
import { MinioClientModule } from 'src/minio-client/minio-client.module';

@Module({
  imports: [
    MinioClientModule
  ],
  providers: [FileUploadService],
  controllers: [FileUploadController]
})
export class FileUploadModule {}
Enter fullscreen mode Exit fullscreen mode

Now we will create our first route on our file-upload.controller.ts:

import { Controller, Post, UseInterceptors, UploadedFile } from '@nestjs/common';
import { FileInterceptor } from '@nestjs/platform-express'
import { FileUploadService } from './file-upload.service';
import { BufferedFile } from 'src/minio-client/file.model';

@Controller('file-upload')
export class FileUploadController {
  constructor(
    private fileUploadService: FileUploadService
  ) {}

  @Post('single')
  @UseInterceptors(FileInterceptor('image'))
  async uploadSingle(
    @UploadedFile() image: BufferedFile
  ) {
    console.log(image)
  }
}
Enter fullscreen mode Exit fullscreen mode

In this code, we created the HTTP POST Method with and endpoint /file-upload/single first lets try to console.log the image to see if we get the BufferFile. Here is the screenshot:

Via Insomnia / Postman (Multipart Form):

Alt Text

Buffered Image File

Alt Text

Our next step is to use our MinioClientService to upload it on our MinIO S3 Storage, let's open our file-upload.service.ts:

import { Injectable } from '@nestjs/common';
import { MinioClientService } from 'src/minio-client/minio-client.service';
import { BufferedFile } from 'src/minio-client/file.model';

@Injectable()
export class FileUploadService {
  constructor(
    private minioClientService: MinioClientService
  ) {}

  async uploadSingle(image: BufferedFile) {

    let uploaded_image = await this.minioClientService.upload(image)

    return {
      image_url: uploaded_image.url,
      message: "Successfully uploaded to MinIO S3"
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

The next step is to use our FileUploadService on FileUploadController, let's go back to our file-upload.controller.ts:

import { Controller, Post, UseInterceptors, UploadedFile } from '@nestjs/common';
import { FileInterceptor } from '@nestjs/platform-express'
import { FileUploadService } from './file-upload.service';
import { BufferedFile } from 'src/minio-client/file.model';

@Controller('file-upload')
export class FileUploadController {
  constructor(
    private fileUploadService: FileUploadService
  ) {}

  @Post('single')
  @UseInterceptors(FileInterceptor('image'))
  async uploadSingle(
    @UploadedFile() image: BufferedFile
  ) {
    return await this.fileUploadService.uploadSingle(image)
  }
}
Enter fullscreen mode Exit fullscreen mode

Now try sending another POST request on Postman or Insomnia. And you should see the follwing:

Alt Text

Alt Text

Congrats, at this point we successfully uploaded our image on our MinIO storage. But wait!!! There's more.. How about multiple image field??? Is it possible on nestjs?? Well, my answer is YES!!! xD and heres how

First we need to create a new endpoint on our file-upload.controller.ts:

import { Controller, Post, UseInterceptors, UploadedFile, UploadedFiles } from '@nestjs/common';
import { FileInterceptor, FileFieldsInterceptor } from '@nestjs/platform-express'
import { FileUploadService } from './file-upload.service';
import { BufferedFile } from 'src/minio-client/file.model';

@Controller('file-upload')
export class FileUploadController {
  constructor(
    private fileUploadService: FileUploadService
  ) {}

  @Post('single')
  @UseInterceptors(FileInterceptor('image'))
  async uploadSingle(
    @UploadedFile() image: BufferedFile
  ) {
    return await this.fileUploadService.uploadSingle(image)
  }

  @Post('many')
  @UseInterceptors(FileFieldsInterceptor([
    { name: 'image1', maxCount: 1 },
    { name: 'image2', maxCount: 1 },
  ]))
  async uploadMany(
    @UploadedFiles() files: BufferedFile,
  ) {
    return this.fileUploadService.uploadMany(files)
  }
}

Enter fullscreen mode Exit fullscreen mode

Then we add the uploadMany method on our FileUploadService:

import { Injectable } from '@nestjs/common';
import { MinioClientService } from 'src/minio-client/minio-client.service';
import { BufferedFile } from 'src/minio-client/file.model';

@Injectable()
export class FileUploadService {
  constructor(
    private minioClientService: MinioClientService
  ) {}

  async uploadSingle(image: BufferedFile) {

    let uploaded_image = await this.minioClientService.upload(image)

    return {
      image_url: uploaded_image.url,
      message: "Successfully uploaded to MinIO S3"
    }
  }

  async uploadMany(files: BufferedFile) {

    let image1 = files['image1'][0]
    let uploaded_image1 = await this.minioClientService.upload(image1)

    let image2 = files['image2'][0]
    let uploaded_image2 = await this.minioClientService.upload(image2)

    return {
      image1_url: uploaded_image1.url,
      image2_url: uploaded_image2.url,
      message: 'Successfully uploaded mutiple image on MinioS3'
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Then try sending another post request on Postman or Insomnia. And you should see the follwing:
Alt Text

Now we are all done. Congratulations and Keep Safe Everyone.

I will leave the repo here:
https://github.com/efd1006/nestjs-file-upload-minio.git

Discussion (2)

Collapse
honeststudio profile image
honest-studio

Great tutorial m8. Thanks!

Collapse
efd1006 profile image
Edmar Diaz Author

Thank you. <3