In-Memory File Generation For Node Server

Angie Sasmita — Sat, 30 Jan 2021 08:42:00 +0000

Use Case

You need to write an end-point that return a file (e.g .zip/.pdf)
You can't save the file in your server for some reasons*

Solution

General idea

Generate file as binary stream
Hold it in memory
Pipe the stream through API

Examples

Generate PDF containing QR-code

Install PDF and QR code library


  npm i qrcode && npm i --save-dev @types/qrcode
  npm i pdf-lib

Generate QR code as a data uri


  export const generateDataUri = (content: string): Promise<string> => {
    return QRCode.toDataURL(content);
  };

Create and embed QR code as PNG


  const pdfDoc = await PDFDocument.create();
  const page = pdfDoc.addPage();

  const qrCodeDataUri = await generateDataUri(name);
  const pngImage = await pdfDoc.embedPng(qrCodeDataUri);
  page.drawImage(pngImage, {
    x: page.getWidth() / 2 - pngImage.width / 2,
    y: (page.getHeight() - pngImage.height) / 2,
    width: pngImage.width,
    height: pngImage.height,
  });

Return PDF as a buffer


  const pdfBytes = await pdfDoc.save();

  return {
    contentBytes: Buffer.from(pdfBytes),
    filename
  };

Configure response as a file


  const respondWithAttachingFile = (
    contentBytes: Buffer,
    res: Response,
    filename: string,
    filetype: string
  ): void => {
    res.setHeader("Content-Type", `application/${filetype}`);
    res.setHeader("Content-Disposition", `attachment; filename=${filename}`);

    res.status(200).end(contentBytes);
  };

  respondWithAttachingFile(contentBytes, res, filename, "pdf");

Generate archive of multiple PDFs

Install archive and stream buffer libraries


  npm i archiver && npm i --save-dev @types/archiver
  npm i stream-buffer && npm i --save-dev @types/stream-buffer

Follow PDF instruction to generate files

Pipe file buffers into archive file


  export const archiveFiles =
    async (fileBuffers: FileBuffer[], outputFilename: string): Promise<FileBuffer> => {
      const archive = archiver("zip", {
        zlib: { level: 9 },
      });
      const filename =
        `${outputFilename}.zip`
          .replace(/ /g, "");

      const outputStreamBuffer = new streamBuffers.WritableStreamBuffer({
        initialSize: (1000 * 1024),
        incrementAmount: (1000 * 1024)
      });

      archive.pipe(outputStreamBuffer);

      fileBuffers.forEach(fileBuffer =>
        archive.append(Buffer.from(fileBuffer.contentBytes), { name: fileBuffer.filename }));

      await archive.finalize();
      outputStreamBuffer.end();

      return new Promise((resolve, reject) => {
        const contentBytes = outputStreamBuffer.getContents();
        if (contentBytes !== false) {
          resolve({ filename, contentBytes });
        }
        reject(new Error("Buffering failed."));
      });
    };