DEV Community: Ashish Barmaiya

Why I Rewrote My Reverse Proxy from Node.js to Go

Ashish Barmaiya — Thu, 23 Jul 2026 12:38:54 +0000

If you’ve read my previous posts, you know I spent months forcing Node.js to act like a production-grade API gateway.

First, I tried building an optimized streaming pipeline, only to discover that unhandled stream errors were quietly destroying my V8 processes. Then, I tried to scale horizontally using Node’s cluster module, which resulted into a nightmare of handling cross-platform IPC signals, graceful teardowns, and fighting Windows socket leaks.

I fixed the bugs and stabilized the prototype. But when I started running high-throughput load generators against it, I hit a wall.

It wasn't a functional bug, rather an architectural wall. Proxies live and die by memory allocations, zero-copy socket streams, raw concurrency overhead, and low-level kernel visibility. The V8 engine and the libuv event loop were no longer assets; they were opaque black boxes standing between my application code and the network stack.

So I scrapped the Node.js prototype and re-engineered Torus from scratch as a native Go infrastructure tool.

This blog documents why Node.js failed at the runtime level, how Go solved it, and the hard benchmark numbers behind the decision.

Part 1: Why Node.js Hits an Architectural Wall

1. The Heavy Compute Starvation Problem (The Event Loop Trap)

Node.js is famous for handling non-blocking I/O efficiently via libuv. If a proxy is just blindly shuffling raw bytes from Client A to Backend B, Node is fast.

But an API gateway doesn't just shuffle bytes, it has to think.

Every time Torus intercepts a request, it must parse inbound HTTP headers, match strings against a multi-tenant routing table, evaluate rate-limiting arrays, and verify cryptographic signatures or JWT tokens.

In Node.js, all user-space compute logic executes sequentially on the single main thread of the event loop. The moment a heavy routing regex calculation or crypto signature check takes 2 to 3 milliseconds to process, the entire network ingestion pipeline for that worker halts. Sockets wait and backpressure builds. Tail latency spikes because the main thread is starved of CPU cycles.

In Go, this constraint vanishes. Go replaces the event loop with the GMP scheduler-a cooperative, multi-threaded work-stealing engine baked directly into the compiled binary.

Instead of a single thread managing callbacks, Go multiplexes thousands of lightweight green threads (goroutines) across a pool of actual OS threads allocated to physical CPU cores. When a request hits Torus, it gets its own goroutine. If a specific tenant's routing logic requires heavy CPU computation, the GMP scheduler keeps that specific goroutine running on one core while seamlessly routing new inbound network connections across the remaining cores. Compute overhead is natively distributed without choking the gateway.

2. Memory Baggage and the Invisible Hand of the GC

When we scale a Node.js proxy via the cluster module, every worker we spawn is an independent instance of the V8 engine.

An idle Node.js worker consumes roughly 10 MB to 30 MB of RAM just to keep its internal context, heap layouts, and Just-In-Time (JIT) compiler metadata alive. Multiply that across a cluster matching your CPU core count, and your baseline memory footprint is hundreds of megabytes before you've routed a single packet.

Worse than the idle footprint are the Garbage Collection (GC) jitters.

In Node, every parsed HTTP header string, transient configuration, and intermediate token object is allocated dynamically on the V8 heap as a pointer-heavy structure. Under traffic surges, the heap fragments, eventually triggering V8’s generational garbage collector. The resulting Mark-Sweep phase causes tiny, unpredictable "Stop-the-World" pauses.

In a standard web app, we won't notice a 15ms GC pause. In an API gateway handling tens of thousands of requests per second, a 15ms pause causes a catastrophic backup on the TCP backlog.

Go treats memory like a systems language. There is no virtual machine wrapper. An idle Torus instance in Go runs at a lightweight ~3 MB of RAM.

More importantly, Go provides explicit control over memory layout via value types and structs. Instead of allocating everything on the heap, transient request metadata stays on the stack, and byte slices can be reused across requests using sync.Pool. By minimizing heap allocations, the Go garbage collector has almost nothing to track, flattening the p99 latency curve under heavy load.

3. Direct Mapping to OS Primitives

Writing abstraction layers in Node.js to handle graceful shutdowns across Windows and Linux (SIGINT vs CTRL_BREAK_EVENT) is painful because Node sits behind multiple layers of runtime abstraction.

Go compiles directly to a naked machine binary targeted at the host operating system. Its internal netpoller integrates directly with low-level OS primitives-using Linux epoll or macOS kqueue under the hood without intermediate runtime marshalling or C++ context-switching boundaries.

When Torus reads from a socket file descriptor in Go, it reads directly into contiguous physical memory byte slices. We don't have to wrap application code in external process managers or build IPC cluster boundaries to utilize multi-core hardware. The language is the runtime.

Part 2: Architecture Comparison Matrix

Characteristic	Node.js Architecture	Go Architecture
Concurrency Model	Multi-process (`node:cluster` + single-threaded V8 event loop)	Single-process (goroutines + M:N scheduler)
Memory Footprint	V8 Garbage Collection (~140–200 MB baseline heap)	Go Runtime GC (~2.8 MB idle, ~20 MB under load)
I/O Engine	libuv non-blocking event loop	Go `netpoll` (epoll/kqueue abstraction over blocking sockets)
Memory Allocation	Dynamic heap objects for parsed headers/tokens	Stack allocations + `sync.Pool` byte-slice recycling
OS Interface	High-level JavaScript abstractions over C++ wrappers	Direct system call interaction via compiled machine code

Part 3: The Benchmark Evolution

An architectural migration requires empirical proof. To measure the impact of the rewrite, I set up black-box stress-testing suites on a dual-core test environment. You can read the comprehensive Benchmark Report here.

The Node.js Baseline

Using a node:cluster topology across 4 worker processes with Autocannon pushing 100 concurrent connections, the native Node.js implementation hit its limits early:

Baseline Reverse Proxy: 1,647 req/sec (~200 MB memory footprint)
Production Edge (TLS + Redis Rate Limiting): 968 req/sec (Average latency: 103 ms, P99: 488 ms)

                          [ Autocannon ]
                        (100 Connections)
                                │
                                ▼
                      [ Master Process ]
                                │
              ┌─────────────────┴─────────────────┐
              ▼                                   ▼
      [ Worker 1 (V8) ]                   [ Worker 2 (V8) ]
              │                                   │
              ▼                                   ▼
      (stream.pipeline)                   (stream.pipeline)
              │                                   │
              ▼                                   ▼
     [ Redis Container ]                [ Upstream Node App ]

Under production-style loads, adding cryptographic operations and a Redis round-trip dropped throughput by 41%, pushing P99 tail latency near half a second.

Benchmark Harness Refinement (Isolating Runtime Drag)

Benchmarking a proxy can be tricky because the test client and mock backends can easily distort results. To capture Torus’s true performance, I evolved the benchmark across three distinct stages:

STAGE 1: Node.js Baseline

Environment: Windows
Harness: [Autocannon (JS)] ──► [Node.js Torus] ──► [Node.js Mock Backends]
Concurrency: node:cluster
Metrics: 1,647 req/sec Throughput | 103 ms Avg Latency | 488 ms p99 | ~200 MB Memory

STAGE 2: Initial Go Rewrite

Environment: Windows
Harness: [Autocannon (JS)] ──► [Go Torus] ───────► [Node.js Mock Backends]
Concurrency: Goroutines
Metrics: 6,044 req/sec Throughput | 16.06 ms Avg Latency | 294 ms p99 | ~20 MB Memory

STAGE 3: Refined Benchmark

Environment: Ubuntu (WSL)
Harness: [wrk (C-native)] ──► [Go Torus] ───────► [Go Native Backends]
Concurrency: Goroutines
Metrics: 17,865 req/sec Throughput | 6.12 ms Avg Latency | 45.32 ms p99 | ~20 MB Memory

The jump from Stage 1 to Stage 2 confirmed that rewriting Torus in Go produced an immediate 3.7× throughput improvement. Simply swapping the V8 engine for Go's runtime and replacing node:cluster with Goroutines slashed average latency from 103 ms down to 16.06 ms, and crushed the memory footprint from 200 MB to a stable 20 MB.

However, profiling revealed that the proxy itself was no longer the limiting factor. Torus spent much of its time waiting for the upstream Node.js mock servers to respond, while Autocannon's JavaScript runtime was saturating CPU cores under high concurrency.

Furthermore, I was still running on Windows, forcing Go to interact with the Windows networking stack instead of utilizing native Linux epoll system calls, which is where Go's netpoll truly shines. The benchmark harness and the host OS had become the bottlenecks.

To eliminate these external constraints, I refined the test environment by:

Migrated to Linux: I moved the entire testing suite into Ubuntu via WSL (Windows Subsystem for Linux) to allow Go to leverage Linux-native networking primitives.
Swapped the Backends: I replaced the Node.js mock applications with lightweight native Go HTTP servers.
Upgraded the Load Generator: I switched from Autocannon to wrk, a multithreaded C-based HTTP benchmarking tool with significantly lower client-side overhead.
Isolated the Processes: I ran the benchmark client, Torus, and the upstream servers in separate terminal sessions to minimize OS CPU-scheduling contention.

These changes stripped away the JS runtime drag and the Windows networking limitations, producing the final benchmark result of 17,865 req/sec and dropping latency even further to 6.12 ms.

To put that in perspective: when I isolated the internal router and completely removed downstream network socket delays (returning an in-memory response), Torus's routing throughput hit 98,565 req/sec. The code wasn't slow; the environment was. Isolating the proxy proved what the Go engine was capable of when it wasn't being choked by JavaScript testing tools and Windows sockets.

Part 4: The Hard Numbers

Experiment A: Full Production Proxying Pipeline

This test exercises the complete reverse proxy pipeline: HTTP parsing, longest-prefix route matching, round-robin load balancing, TCP connection pooling, header injection, and zero-copy stream forwarding.

Test Setup: wrk -t2 -c100 -d10s against native Go mock backends.

Metric	Node.js Original	Go Rewrite (Final)	Improvement
Throughput	1,647.00 req/sec	17,865.81 req/sec	+984.7% (10.8×)
Average Latency	103.00 ms	6.12 ms	−94.0%
Max Tail Latency (P99)	488.00 ms	45.32 ms	−90.7%
Memory Footprint	~200 MB	~20 MB	10× reduction
Data Transfer Rate	—	2.01 MB/sec (178,892 total requests)	—

Production Proxy Throughput (req/sec)

Node.js      [████                                           1,647]
Initial Go   [███████████████                                6,044]
Final Go     [███████████████████████████████████           17,865]

Under full proxy execution, throughput increased by 10.8× while average latency dropped from 103ms down to 6.12ms. Memory consumption dropped from 200MB down to a stable 20MB under load.

Experiment B: Short-Circuit Routing Path

To measure the maximum throughput of Torus's internal router (header parsing, route matching, load balancing logic) independent of downstream network socket delays, I tested a scenario where downstream backends were marked dead. Torus short-circuited and immediately returned an in-memory error response.

Test Setup: wrk -t2 -c100 -d10s (No Live Backends).

Metric	Initial Go (Autocannon)	Final Go (wrk)
Routing Throughput	15,644 req/sec	98,565.91 req/sec
Average Latency	5.94 ms	10.21 ms
Max Tail Latency (P99)	78.00 ms	86.63 ms
Data Transfer Rate	—	15.70 MB/sec

Removing downstream network I/O allowed the Go routing engine to approach 100,000 req/sec on a basic local development machine. This confirms that Torus’s internal execution pipeline adds minimal overhead-downstream network delays account for virtually the entire latency profile of the proxy.

Lessons Learned

Building Torus in Node.js first was a valuable exercise. Node.js remains an exceptional environment for rapid prototyping, forcing developers to handle edge cases, connection pooling, and stream mechanics explicitly. However, selecting an infrastructure runtime requires aligning tools with system requirements:

JavaScript is for Application State; Go is for Networking Primitives. If an application manages business logic, database mutations, and complex JSON schemas, Node.js performs well. But when a system primarily shuttles bytes between network cards without mutating payloads, Go's pointer controls, fixed-size types, and native network stack offer a fundamental advantage.
Account for Load Client Overhead. If a benchmarking tool shares a runtime environment or CPU thread pool with the system under test, metrics will be distorted. Switching to wrk uncovered significant performance reserves masked by client-side event loop contention.
Memory Control Directly Impacts Latency Stability. Dropping baseline memory usage from 200 MB to 20 MB does more than save RAM, it eliminates GC-induced tail latency, stabilizing p99 metrics under sustained traffic spikes.

The full benchmark specifications, ADR records, and Go implementation are available in the repository.

Surviving Node.js Clusters: Graceful Teardowns, Windows Quirks, and Black-Box Testing

Ashish Barmaiya — Tue, 07 Apr 2026 01:40:34 +0000

When I set out to build Torus, a reverse proxy in Node.js, the node:cluster module looked like the perfect solution to my biggest architectural hurdle. Because Node.js is natively single-threaded, this core module is the standard way to achieve multi-core scaling - it lets a single application spawn multiple worker processes that all share the same server port.

It solved the problem immediately. But it quietly introduced a serious vulnerability I didn't catch until it was almost too late.

The Trap: Blind Resurrection

After scaffolding the cluster, I wrote the standard if (cluster.isPrimary) block, looped through CPU cores, called cluster.fork(), and attached an exit listener to respawn workers on crash:

cluster.on('exit', (worker, code, signal) => {
  logger.warn(`Worker ${worker.process.pid} died. Booting a replacement...`);
  cluster.fork();
});

On the surface, this is exactly right. If a worker blows up with an Out-Of-Memory exception or a rogue regex tears down the V8 engine, the Master catches the exit event and spawns a fresh replacement. Capacity heals itself.

The problem is that this code is completely blind. It doesn't know why a worker died. It only knows that a process stopped, and its only response is to immediately restart it.

This breaks the moment you attempt a routine deployment.

The Kubernetes Tug-of-War

Imagine this code running in a Docker container orchestrated by Kubernetes. You push version 1.1 of your proxy. Kubernetes initiates a rolling update and sends a SIGTERM to your pod: "Finish your active requests and shut down cleanly."

Your workers receive the signal, drain their active TCP sockets, and exit with code 0.

But the millisecond the first worker exits, your Master's .on('exit') listener fires. It doesn't see a coordinated graceful shutdown - it sees a dead process, and it immediately forks a replacement.

While Kubernetes is trying to peacefully drain your pod, your Master is frantically spawning new workers to replace the ones shutting down. You're now locked in a fight with your own infrastructure.

Eventually Kubernetes runs out of patience, fires a SIGKILL, and brutally terminates the Master along with every zombie worker it just spawned. Any clients that were mid-stream have their TCP connections severed instantly.

The "zero-downtime deployment" was a lie.

The Fix: A State-Gated Lifecycle

To fix this, the Master process needs to distinguish between two very different events:

An unexpected crash (a V8 segfault, an OOM exception) → resurrect immediately
An intentional shutdown (a Kubernetes eviction, a SIGTERM) → stand down and let workers die cleanly

That requires three things: a global state lock, an IPC broadcast, and a lifecycle manager with a hard timeout.

1. The Global State Lock

In the Master process, a single boolean flag acts as the guillotine switch:

let isShuttingDown = false;

The .on('exit') listener is rewritten to check this flag before doing anything:

cluster.on("exit", (worker, code, signal) => {
  if (isShuttingDown) {
    logger.info(`Worker ${worker.process.pid} exited cleanly during shutdown.`);

    if (Object.keys(cluster.workers || {}).length === 0) {
      logger.info("All workers stopped. Master exiting with code 0.");
      process.exit(0);
    }
  } else {
    logger.warn(`Worker ${worker.process.pid} crashed. Forking a replacement...`);
    cluster.fork();
  }
});

If the cluster is shutting down, the Master lets workers die in peace and waits until the pool is empty before exiting cleanly. If the flag is false, a worker crashed unexpectedly and gets resurrected immediately.

2. The Teardown Broadcast via IPC

When the OS sends SIGTERM, the Master intercepts it, flips the lock, and broadcasts a SHUTDOWN command to every worker over Node's native IPC channel - rather than killing them instantly and dropping active payloads:

const initiateClusterTeardown = (signal: string) => {
  if (isShuttingDown) return;
  isShuttingDown = true;

  logger.info({ signal }, "Master received termination signal. Broadcasting shutdown to workers...");

  for (const id in cluster.workers) {
    cluster.workers[id]?.send({ type: "SHUTDOWN" });
  }
};

process.on("SIGTERM", () => initiateClusterTeardown("SIGTERM"));
process.on("SIGINT",  () => initiateClusterTeardown("SIGINT"));

3. The LifecycleManager and the 10-Second Guillotine

Inside each worker, the SHUTDOWN message is handled by a LifecycleManager singleton - a single object that owns the worker's entire teardown sequence and prevents the race conditions that come from ad-hoc async cleanup code.

When it receives the command, it executes a strict sequence:

Stop the bleeding - immediately reject new incoming TCP connections
Destroy idle sockets - aggressively close idle Keep-Alive connections to free OS file descriptors
Drain active payloads - wait for in-progress streams to finish naturally

Step 3 has a fatal edge case: a slow or malicious client trickling 1 byte per second will keep server.close() waiting indefinitely, hanging the deployment pipeline forever.

The LifecycleManager solves this with a hard 10-second timeout:

// Inside LifecycleManager.executeTeardown()
const drainPromise = Promise.all(
  this.tasks.map(async (task) => {
    try {
      await task;
    } catch (err) {
      logger.error({ err }, "A teardown task failed during shutdown.");
    }
  }),
);

const timeoutPromise = new Promise((_, reject) => {
  setTimeout(() => {
    reject(new Error(`Shutdown timed out after ${this.SHUTDOWN_TIMEOUT_MS}ms`));
  }, this.SHUTDOWN_TIMEOUT_MS).unref(); // unref() prevents the timer from keeping the event loop alive
});

try {
  await Promise.race([drainPromise, timeoutPromise]);
  logger.info("All systems cleanly drained. Exiting process gracefully.");
  process.exit(0);
} catch (error: any) {
  logger.error( err: error.message },
  "Graceful shutdown aborted. Forcing exit.");
  process.exit(1);
}

The worker handles the IPC message like this:

process.on("message", (msg: any) => {
  if (msg && msg.type === "SHUTDOWN") {
    Lifecycle.executeTeardown("IPC_SHUTDOWN");
  }
});

With this in place: if a worker segfaults, it heals in milliseconds. If Kubernetes sends SIGTERM, the proxy drains active connections cleanly, brutally severs any hanging connections after 10 seconds, and exits with code 0.

The theoretical architecture was solid. Proving it worked was a different problem entirely.

The Testing Nightmare: Why Jest Can't Test This

Standard Jest unit tests are fundamentally incapable of testing a multi-process cluster. Jest runs inside a single Node.js process. Calling cluster.fork() or process.exit() from inside a test will either crash the test runner or leave orphan processes silently eating RAM in the background.

To prove Torus could survive a crash and execute a graceful teardown, I abandoned unit testing entirely and built a black-box integration test. The test treats the compiled proxy as a hostile external artifact: it boots it, wiretaps its output, attacks it, and evaluates how it responds.

Step 1: Boot the Cluster in Isolation

Using child_process.spawn, the test starts the compiled binary exactly as production would:

const entryPoint = path.resolve(__dirname, "../../dist/index.js");

masterProcess = spawn("node", ["--env-file=.env", entryPoint], {
  stdio: ["pipe", "pipe", "pipe", "ipc"],
});

The ipc channel in stdio is important - it's how the test sends commands to the Master later.

Step 2: Wiretap stdout and Assassinate a Worker

Because the proxy runs in an isolated background process, internal variables aren't accessible. Instead, the test intercepts raw stdout. When a worker logs that it's ready, the test extracts its PID and sends it a SIGKILL - bypassing any graceful shutdown handler entirely, simulating a fatal OOM crash:

masterProcess.stdout!.on("data", (data) => {
  const output = data.toString();

  if (!targetWorkerPid && output.includes("listening for Secure HTTPS traffic")) {
    const match = output.match(/Worker\s+(\d+)\s+listening/);
    if (match) {
      targetWorkerPid = parseInt(match[1], 10);
      process.kill(targetWorkerPid, "SIGKILL"); // Instant, uninterceptable death
    }
  }
  // ...
});

SIGKILL cannot be caught by any signal handler. This is the correct way to simulate catastrophic process death.

Step 3: Verify Resurrection and Trigger Teardown

The test continues monitoring stdout. Once it sees the Master log a resurrection, it locks a boolean and sends a shutdown command via IPC:

  if (targetWorkerPid && output.includes("crashed. Forking a replacement") && !workerResurrected) {
    workerResurrected = true;

    setTimeout(() => {
      masterProcess.send({ type: "TEST_SHUTDOWN" });
    }, 500); // Give the cluster 500ms to stabilize before triggering teardown
  }

Step 4: The Final Verdict

When the last worker exits, the Master closes. The test uses the close event as its final assertion gate:

masterProcess.on("close", (code) => {
  try {
    expect(workerResurrected).toBe(true); // Cluster self-healed after crash
    expect(shutdownInitiated).toBe(true); // IPC teardown broadcast worked
    expect(code).toBe(0);                 // Master exited cleanly
    resolve();
  } catch (error) {
    reject(error);
  }
});

The test was flawless. It spawned the cluster, assassinated the worker, verified the resurrection, and cleanly exited.

But notice that TEST_SHUTDOWN command in Step 3? I didn't write it that way originally. Originally, I just told the test to send a standard SIGINT to the Master. But when I ran it, my Windows machine silently choked to death.

The Windows Problem: POSIX Signals Are a Lie

The black-box test spawned the cluster, killed a worker, and watched the Master resurrect it. The final step was to prove the LifecycleManager could execute a zero-downtime graceful shutdown.

To simulate a Kubernetes pod eviction or a developer hitting Ctrl+C, the integration test needed to send a termination signal to the Master process.

masterProcess.kill("SIGINT");

I ran the test. It failed instantly.

The Master didn't gracefully drain the TCP sockets. It didn't trigger the 10-second timeout. It just died on the spot. The logs went completely silent.

On Linux and macOS, sending a termination signal to the Master process works perfectly. SIGINT is a native POSIX signal that politely knocks on the process's door and allows the Node.js event loop to execute its .on('SIGINT') handler.

Windows doesn't have native POSIX signals.

When a Node.js process programmatically calls .kill() with SIGINT on Windows, the OS can't route it through the event loop. Instead, it bypasses the signal handler entirely and terminates the target process immediately. My Master process wasn't failing to drain its sockets - Windows was killing it before the LifecycleManager could even start.

The fix was to bypass the OS signal layer entirely. I added a dedicated IPC backdoor directly in the Master's message handler:

// Testing Backdoor for Windows OS limitations
process.on("message", (msg: any) => {
  if (msg && msg.type === "TEST_SHUTDOWN") {
    initiateClusterTeardown("TEST_SHUTDOWN");
  }
});

And in the test, .kill("SIGINT") became:

setTimeout(() => {
  masterProcess.send({ type: "TEST_SHUTDOWN" });
}, 500);

This sends a plain JSON payload over the IPC channel - no OS signal routing required. The Master receives it, flips the isShuttingDown flag, broadcasts teardown to the workers, and the LifecycleManager executes the full drain sequence.

The CI/CD Trap: Environment Drift

With the Windows fix in place, the local test suite passed cleanly. I pushed to GitHub and waited for the green checkmark.

The pipeline spun for 15 seconds and failed silently.

Diagnosing the Silence

When the black-box test's wiretap never hears the expected log line, it just sits there until Jest's timeout guillotine drops. To find out what was actually happening in the CI runner, I temporarily dumped raw stdout:

masterProcess.stdout!.on("data", (data) => {
  console.log(data.toString()); // Raw output — for CI diagnostics only
  // ...
});

On the next run, the pipeline printed the truth:

ENOENT: no such file or directory, open '/home/runner/work/torus-proxy/certs/key.pem'

Missing Certificates

TLS certificates belong in .gitignore. On my local machine they existed. On GitHub Actions' naked Ubuntu runner, they didn't. Workers hit the TLS configuration block, threw an ENOENT, and died before ever binding to a port.

Because the proxy runs as a real detached process, jest.mock('fs') isn't an option - the isolated binary needs real files on disk. The fix was to generate dummy self-signed certificates in the CI pipeline before the test runs:

- name: Generate Dummy TLS Certificates
  run: |
    mkdir -p certs
    openssl req -nodes -new -x509 \
      -keyout certs/key.pem \
      -out certs/cert.pem \
      -days 365 \
      -subj "/CN=localhost"

Missing Environment Variables

The pipeline failed again. This time:

Error: JWT Secret is required to boot JWT Authenticator.

The .env file was also gitignored. Rather than writing a fake .env to the runner disk, I injected the dummy secret directly into the spawn environment:

const envPath = path.resolve(process.cwd(), ".env");

const nodeArgs = fs.existsSync(envPath)
  ? ["--env-file=.env", entryPoint]
  : [entryPoint];

masterProcess = spawn("node", nodeArgs, {
  stdio: ["pipe", "pipe", "pipe", "ipc"],
  env: {
    ...process.env,
    JWT_SECRET: "dummy_test_secret_for_ci_pipeline_only",
  },
});

I pushed the final commit. The pipeline generated the certificates, injected the secret, booted the Master, assassinated a worker, watched it resurrect, triggered teardown, drained sockets, and exited with code 0.

Green checkmark.

Conclusion: Trust Nothing

The standard Node.js cluster tutorials are dangerously incomplete. Blindly resurrecting workers on every exit event creates a system that will fight your deployment pipelines and drop active TCP connections mid-stream.

Building something production-grade requires:

Deterministic state machines - not reflexive code that acts without knowing why things happened
Structured teardown sequences - not ad-hoc async cleanup that races itself
Skepticism about the OS - signals behave differently across platforms in ways the documentation doesn't always make clear
Black-box integration tests - not unit tests that mock away the exact failure modes you're trying to protect against
CI environment parity - assume the runner has nothing your .gitignore excluded

The cluster module is powerful. But its standard usage pattern gives you the illusion of resilience, not the real thing.

If you'd like to see the full implementation, the source code for Torus Proxy is on GitHub.

Why I Ripped stream.pipe() Out of My Node.js API Gateway

Ashish Barmaiya — Fri, 27 Mar 2026 16:09:13 +0000

When I started building Torus, a multi-core Layer 7 Edge API Gateway from scratch in Node.js, I handled incoming network requests the way I had always seen it done in standard web applications:

TypeScript

let body = '';
req.on('data', (chunk: Buffer) => {
  body += chunk.toString();
});
req.on('end', () => {
  forwardToBackend(body);
});

It worked perfectly for lightweight tests. But as I started pushing concurrent loads and larger payloads through the proxy, my server began to choke. CPU usage spiked to 100%, the event loop lagged, and memory consumption grew uncontrollably until the process crashed.

I had fallen into a classic architectural trap: I was dragging raw TCP payload bytes directly into the V8 JavaScript engine's memory heap.

Because the V8 heap has a strict memory limit, pulling massive payloads into user-space memory forces the Node.js Garbage Collector (GC) to work overtime. The GC halts the single-threaded event loop to clean up the allocated memory, effectively stalling every other active network connection in the proxy.

I learned a fundamental rule of proxy engineering the hard way:

Proxies shouldn't read data; they should just move it.

To build a production-grade gateway, I realized I had to bypass the V8 heap entirely. I needed to keep the data in raw C++ memory blocks and move it to the Operating System level. But as I refactored my routing logic to achieve this, I stumbled into a silent, catastrophic flaw in the standard Node.js stream API that brought my entire test suite to a halt.

The First Evolution: Bypassing V8 with `.pipe()`

The architectural fix required a fundamental shift in how I viewed the data. I had to stop treating payloads as static variables and start treating them as flowing water.

I didn't need to load an entire 50MB file into memory before forwarding it. I only needed to hold a few kilobytes in a temporary buffer, flush it to the destination, and reuse that memory space.

In Node.js, this is exactly what the node:stream module and Buffer objects are designed for.

A Buffer in Node.js allocates memory outside the V8 JavaScript engine. It utilizes raw C++ memory blocks mapped directly to the OS. By keeping the network chunks as raw Buffers, the payload never enters the JavaScript heap. Because it never enters the heap, the V8 Garbage Collector completely ignores it, leaving the event loop free to handle other connections.

To wire this up, I utilized the native .pipe() method to connect the readable client stream to the writable backend stream:

TypeScript

// Connects the incoming ReadableStream directly to the outgoing WritableStream
clientReq.pipe(proxyReq);

This single line of code acts as an OS-level plumbing system. It takes the incoming TCP stream, reads the raw C++ buffers, automatically manages the backpressure (ensuring a fast client doesn't overwhelm a slow backend connection), and pushes the bytes directly out to the routing pool.

My CPU usage plummeted. The memory footprint stayed flat, regardless of how large the incoming payloads were. It felt like I had solved the scaling problem entirely.

But .pipe() was hiding a massive, silent vulnerability.

The Plot Twist: The Silent Socket Leak

I thought I had engineered the perfect solution. The proxy was fast, the CPU was idle, and the memory footprint stayed completely flat, regardless of payload size.

Then I ran my integration test suite.

All the assertions passed. I got the green checkmarks. But the terminal just froze. Jest refused to exit, eventually spitting out that infuriating warning: "Jest did not exit one second after the test run has completed."

My initial reaction was to treat it like a standard web app bug. I meticulously checked my teardown logic, making sure proxyServer.close() was being called and my Redis clients were fully disconnected. I ran the tests again. It still hung.

I had to drop down to the OS level to understand what was actually happening. The Node.js event loop is mathematically programmed to never exit as long as there is an active I/O handle (like a net.Socket) in its queue. Something was keeping a socket alive.

The culprit was .pipe().

When my Jest test fired a dummy request through the proxy and then disconnected, the client-side socket closed gracefully. But I learned a lesson about Node.js streams: .pipe() blindly pushes data; it does not propagate lifecycle events.

When the client dropped, .pipe() did not send an error or close event to the destination stream. It left the backend connection completely open. The proxy was sitting there holding a dead connection to the backend, waiting for network bytes that would never arrive.

I had built a machine that generated half-open sockets. In a production environment, this would silently exhaust the Operating System's File Descriptors (FDs). Every dropped client connection would permanently lock up an FD until the OS hit its limit and violently crash the Node process with an EMFILE error.

The Fix: stream.pipeline()

The Node.js core maintainers knew .pipe() was dangerously naive for production infrastructure. That is exactly why they introduced stream.pipeline().

Instead of blindly shoving data from one socket to another, .pipeline() acts as a unified state machine that monitors the entire stream chain. It pushes the responsibility of socket teardown back to the Node.js core networking stack where it belongs.

If any stream in the pipeline fails, throws an error, or abruptly closes (like a client dropping off with an ECONNRESET), .pipeline() automatically intercepts it. It destroys all other connected streams in that specific chain and bubbles up a single error for you to catch.

I removed every instance of .pipe() and the dozens of lines of manual .on('error') spaghetti I had written. Because raw TCP proxying requires bidirectional data flow, I replaced it with two parallel, sequential pipelines:

TypeScript

try {
  await Promise.all([
    pipeline(clientSocket, backendSocket),
    pipeline(backendSocket, clientSocket)
  ]);
} catch (err: any) {
  // If either side drops, the pipeline throws, and we clean up natively.
  clientSocket.destroy();
  backendSocket.destroy();
}

The moment I swapped to this architecture and ran my integration suite, the terminal didn't hang. Jest executed all 18 network tests and exited flawlessly in 2.7 seconds. The event loop was instantly cleared. The silent socket leak was completely eradicated.

Conclusion: Trust Nothing, Understand Everything

Building a multi-core Edge Gateway from scratch taught me that I cannot blindly trust high-level abstractions.

stream.pipe() looks elegant in a standard web tutorial, but in the trenches of raw TCP networking, it is a massive liability. If you are building infrastructure that handles thousands of concurrent connections, you must understand the Operating System-level lifecycle of your File Descriptors and your sockets. If you don't, your system will slowly bleed to death under load, and logs won't even tell you why.

If you want to see the exact implementation of this bidirectional TCP routing, you can check out the raw source code for Torus Proxy on my GitHub.