Implementing Multer Storage Engine in TypeScript

Multer is pretty popular ExpressJS middleware, primarily used to handle file uploads from client to server. However, a lot of other options are available too depending on what fits your use case.

Multer plays along really well with ExpressJS and have out-of-the-box support for storing file in-disk and in-memory, on the server side. Interestingly, Multer can also be used for file streaming applications where you don't need to store the incoming file on server side and directly redirect the incoming file stream to some cloud storage service instead (say, AWS S3). When you need to support the uploads of bulky/large files (say, tens or hundreds of MBs or anything beyond that), streaming can come across as the most practical solution.

The file streaming logic can vary depending on your cloud storage service provide but in this post I'll simply present the storage engine implementation logic in generalized way which you can expand upon based on your use cases. This logic is already presented by the maintainers of Multer here but it uses object prototypes and vanilla JS. For those of us who have gotten the taste of syntactic sugar by using JS classes might find it hard to comprehend that logic right away. Moreover, owing to the popularity of TypeScript, a TS implementation of that storage engine logic may also be handy for the web dev community.

So, here's TS Classes based approach for implementing Multer's Storage Engine:

	import { createWriteStream, unlink } from "fs";
	import type { PathLike } from "fs";
	import type { Request } from "express";
	type Maybe<T> = T | null | undefined;
	type ResponseCallback<T> = (err: Error | null, payload?: Maybe<T>) => void;
	type RequestHandler<T> = (req: Request, file: Express.Multer.File, responseCb: ResponseCallback<T>) => void;
	/**
	* Custom Multer Storage Engine for uploading file to AWS S3
	* Ref: https://github.com/expressjs/multer/blob/master/StorageEngine.md
	* Example Usage:
	* ```ts
	* const storageEngine = new MulterS3StorageEngine((_req, file, responseCb) => {
	* responseCb(null, `/var/www/uploads/${file.originalname}`);
	* });
	* ```
	*/
	export class MulterS3StorageEngine {
	/**
	* Tells the engine where to save the file.
	* @param _req
	* @param _file
	* @param responseCb
	*/
	getDestination: RequestHandler<PathLike> = (_req, _file, responseCb) => {
	responseCb(null, "/dev/null");
	};
	/**
	* Stores the file and returns information on how to access the file in the future.
	* The information you provide in the callback will be merged with multer's file object, and then presented to the user via `req.files`.
	* @param req
	* @param file
	* @param responseCb
	*/
	_handleFile: RequestHandler<unknown> = (
	req: Request,
	file: Express.Multer.File,
	responseCb: ResponseCallback<unknown>
	) => {
	this.getDestination(req, file, (err, path) => {
	if (err !== null) {
	responseCb(err);
	}
	if (path !== null && path !== undefined) {
	const outStream = createWriteStream(path);
	file.stream.pipe(outStream);
	outStream.on("error", responseCb);
	outStream.on("finish", () => {
	responseCb(null, { path, size: outStream.bytesWritten });
	});
	}
	});
	};
	/**
	* Removes the file if an error is encounter during file stream handling.
	* Multer will decide which files to delete and when.
	* @param _req
	* @param file
	* @param responseCb
	*/
	_removeFile: RequestHandler<unknown> = (
	_req: Request,
	file: Express.Multer.File,
	responseCb: ResponseCallback<unknown>
	) => {
	unlink(file.path, responseCb);
	};
	constructor(destination?: RequestHandler<string>) {
	this.getDestination = destination ?? this.getDestination;
	}
	}

view raw multer-storage-engine.ts hosted with ❤ by GitHub