Why We Built MailPort: A Modern Streaming Email SDK for Node.js

#javascript #typescript #node #webdev

How I Streamed a 100 MB Email Attachment in Node.js Without Buffering the Complete File

Target Publication: Dev.to / Hashnode / Medium
Tags: #nodejs #typescript #webdev #performance #email

The Problem with Naive Email Attachment Buffering

In standard Node.js email implementations, sending a multi-megabyte email attachment often loads the entire binary file into a Node.js Buffer in memory.

When encoding a 100 MB attachment to Base64:

The 100 MB raw buffer is loaded into heap memory.
Base64 encoding creates a ~133 MB string.
MIME line wrapping creates another string object.
Total transient memory footprint exceeds 300+ MB for a single attachment!

If multiple concurrent requests stream large attachments, Node.js applications frequently experience V8 heap inflation, garbage collection pauses, or Out-Of-Memory (OOM) crashes.

The AsyncIterable Streaming Solution

To eliminate buffer inflation, MailPort's MIME engine (@mailport/mime) processes attachment data incrementally using AsyncIterable<Uint8Array> streams.

1. Base64 Chunk Boundary Handling

Base64 encoding transforms groups of 3 binary bytes into 4 ASCII characters. When streaming arbitrary file chunks (e.g. 64 KB or 10 MB chunks), a chunk boundary may split across a 3-byte tuple.

Base64StreamEncoder buffers only the leftover 1 or 2 bytes across chunk boundaries:

export class Base64StreamEncoder {
  #remainder: Uint8Array = new Uint8Array(0);

  encodeChunk(chunk: Uint8Array): Uint8Array {
    // Combine 1-2 remainder bytes with incoming chunk
    // Process in multiples of 3 bytes
    // Preserve new remainder for next chunk
  }
}

2. Direct Socket Pipeline

The generated AsyncIterable<Uint8Array> stream feeds directly into the TCP/TLS socket write queue with backpressure handling (socket.write() + drain event listener).

Empirical Benchmark Results

We benchmarked streaming a 100 MB attachment on Node.js v24 (macOS ARM64):

Metric	Result
Emitted Transmitted Payload	136.84 MB
Throughput	387.57 MB/s
Duration	353 ms
Peak Heap Memory Delta	8.98 MB

Peak heap memory usage remained under 9 MB while processing a 136 MB emitted payload!

Code Example

import { createReadStream } from 'node:fs';
import { createMailer, smtp } from 'mailport';

const mailer = createMailer({
  transport: smtp({
    host: process.env.SMTP_HOST!,
    port: 587,
    auth: { user: process.env.SMTP_USER!, pass: process.env.SMTP_PASSWORD! },
  }),
});

await mailer.send({
  from: 'exports@example.com',
  to: 'user@example.com',
  subject: 'Monthly Data Backup',
  attachments: [
    {
      type: 'stream',
      filename: 'backup.sql.gz',
      stream: createReadStream('./backup.sql.gz'),
    },
  ],
});