Why I Rewrote My Reverse Proxy from Node.js to Go

If you’ve read my previous posts, you know I spent months forcing Node.js to act like a production-grade API gateway.

First, I tried building an optimized streaming pipeline, only to discover that unhandled stream errors were quietly destroying my V8 processes. Then, I tried to scale horizontally using Node’s cluster module, which resulted into a nightmare of handling cross-platform IPC signals, graceful teardowns, and fighting Windows socket leaks.

I fixed the bugs and stabilized the prototype. But when I started running high-throughput load generators against it, I hit a wall.

It wasn't a functional bug, rather an architectural wall. Proxies live and die by memory allocations, zero-copy socket streams, raw concurrency overhead, and low-level kernel visibility. The V8 engine and the libuv event loop were no longer assets; they were opaque black boxes standing between my application code and the network stack.

So I scrapped the Node.js prototype and re-engineered Torus from scratch as a native Go infrastructure tool.

This blog documents why Node.js failed at the runtime level, how Go solved it, and the hard benchmark numbers behind the decision.

Part 1: Why Node.js Hits an Architectural Wall

1. The Heavy Compute Starvation Problem (The Event Loop Trap)

Node.js is famous for handling non-blocking I/O efficiently via libuv. If a proxy is just blindly shuffling raw bytes from Client A to Backend B, Node is fast.

But an API gateway doesn't just shuffle bytes, it has to think.

Every time Torus intercepts a request, it must parse inbound HTTP headers, match strings against a multi-tenant routing table, evaluate rate-limiting arrays, and verify cryptographic signatures or JWT tokens.

In Node.js, all user-space compute logic executes sequentially on the single main thread of the event loop. The moment a heavy routing regex calculation or crypto signature check takes 2 to 3 milliseconds to process, the entire network ingestion pipeline for that worker halts. Sockets wait and backpressure builds. The p99 tail latency spikes because the main thread is starved of CPU cycles.

In Go, this constraint vanishes. Go replaces the event loop with the GMP scheduler-a cooperative, multi-threaded work-stealing engine baked directly into the compiled binary.

Instead of a single thread managing callbacks, Go multiplexes thousands of lightweight green threads (goroutines) across a pool of actual OS threads allocated to physical CPU cores. When a request hits Torus, it gets its own goroutine. If a specific tenant's routing logic requires heavy CPU computation, the GMP scheduler keeps that specific goroutine running on one core while seamlessly routing new inbound network connections across the remaining cores. Compute overhead is natively distributed without choking the gateway.

2. Memory Baggage and the Invisible Hand of the GC

When we scale a Node.js proxy via the cluster module, every worker we spawn is an independent instance of the V8 engine.

An idle Node.js worker consumes roughly 10 MB to 30 MB of RAM just to keep its internal context, heap layouts, and Just-In-Time (JIT) compiler metadata alive. Multiply that across a cluster matching your CPU core count, and your baseline memory footprint is hundreds of megabytes before you've routed a single packet.

Worse than the idle footprint are the Garbage Collection (GC) jitters.

In Node, every parsed HTTP header string, transient configuration, and intermediate token object is allocated dynamically on the V8 heap as a pointer-heavy structure. Under traffic surges, the heap fragments, eventually triggering V8’s generational garbage collector. The resulting Mark-Sweep phase causes tiny, unpredictable "Stop-the-World" pauses.

In a standard web app, we won't notice a 15ms GC pause. In an API gateway handling tens of thousands of requests per second, a 15ms pause causes a catastrophic backup on the TCP backlog.

Go treats memory like a systems language. There is no virtual machine wrapper. An idle Torus instance in Go runs at a lightweight ~3 MB of RAM.

More importantly, Go provides explicit control over memory layout via value types and structs. Instead of allocating everything on the heap, transient request metadata stays on the stack, and byte slices can be reused across requests using sync.Pool. By minimizing heap allocations, the Go garbage collector has almost nothing to track, flattening the p99 latency curve under heavy load.

3. Direct Mapping to OS Primitives

Writing abstraction layers in Node.js to handle graceful shutdowns across Windows and Linux (SIGINT vs CTRL_BREAK_EVENT) is painful because Node sits behind multiple layers of runtime abstraction.

Go compiles directly to a naked machine binary targeted at the host operating system. Its internal netpoller integrates directly with low-level OS primitives-using Linux epoll or macOS kqueue under the hood without intermediate runtime marshalling or C++ context-switching boundaries.

When Torus reads from a socket file descriptor in Go, it reads directly into contiguous physical memory byte slices. We don't have to wrap application code in external process managers or build IPC cluster boundaries to utilize multi-core hardware. The language is the runtime.

Part 2: Architecture Comparison Matrix

Characteristic	Node.js Architecture	Go Architecture
Concurrency Model	Multi-process (node:cluster + single-threaded V8 event loop)	Single-process (goroutines + M:N scheduler)
Memory Footprint	V8 Garbage Collection (~140–200 MB baseline heap)	Go Runtime GC (~2.8 MB idle, ~20 MB under load)
I/O Engine	libuv non-blocking event loop	Go netpoll (epoll/kqueue abstraction over blocking sockets)
Memory Allocation	Dynamic heap objects for parsed headers/tokens	Stack allocations + sync.Pool byte-slice recycling
OS Interface	High-level JavaScript abstractions over C++ wrappers	Direct system call interaction via compiled machine code

---

Part 3: The Benchmark Evolution

An architectural migration requires empirical proof. To measure the impact of the rewrite, I set up black-box stress-testing suites on a dual-core test environment. You can read the comprehensive Benchmark Report here.

The Node.js Baseline

Using a node:cluster topology across 4 worker processes with Autocannon pushing 100 concurrent connections, the native Node.js implementation hit its limits early:

• Baseline Reverse Proxy: 1,647 req/sec (~200 MB memory footprint)

• Production Edge (TLS + Redis Rate Limiting): 968 req/sec (Average latency: 103 ms, P99: 488 ms)
  

                          [ Autocannon ]
                        (100 Connections)
                                │
                                ▼
                      [ Master Process ]
                                │
              ┌──────────┴────────────┐
              ▼                                   ▼
      [ Worker 1 (V8) ]                   [ Worker 2 (V8) ]
              │                                   │
              ▼                                  ▼
        (stream.pipe)                       (stream.pipe)
              │                                   │
              ▼                                  ▼
     [ Redis Container ]                [ Upstream Node App ]

Under production-style loads, adding cryptographic operations and a Redis round-trip dropped throughput by 41%, pushing P99 tail latency near half a second.

Benchmark Harness Refinement (Isolating Runtime Drag)

Benchmarking a proxy can be tricky because the test client and mock backends can easily distort results. To capture Torus’s true performance, I evolved the benchmark across three distinct stages:

STAGE 1: Node.js Baseline

• Environment: Windows

• Harness: [Autocannon (JS)] ──► [Node.js Torus] ──► [Node.js Mock Backends]

• Concurrency: node:cluster

• Metrics: 1,647 req/sec Throughput | 103 ms Avg Latency | 488 ms p99 | ~200 MB Memory

STAGE 2: Initial Go Rewrite

• Environment: Windows

• Harness: [Autocannon (JS)] ──► [Go Torus] ───────► [Node.js Mock Backends]

• Concurrency: Goroutines

• Metrics: 6,044 req/sec Throughput | 16.06 ms Avg Latency | 294 ms p99 | ~20 MB Memory

STAGE 3: Refined Benchmark

• Environment: Ubuntu (WSL)

• Harness: [wrk (C-native)] ──► [Go Torus] ───────► [Go Native Backends]

• Concurrency: Goroutines

• Metrics: 17,865 req/sec Throughput | 6.12 ms Avg Latency | 45.32 ms p99 | ~20 MB Memory

The jump from Stage 1 to Stage 2 confirmed that rewriting Torus in Go produced an immediate 3.7× throughput improvement. Simply swapping the V8 engine for Go's runtime and replacing node:cluster with Goroutines slashed average latency from 103 ms down to 16.06 ms, and crushed the memory footprint from 200 MB to a stable 20 MB.

However, profiling revealed that the proxy itself was no longer the limiting factor. Torus spent much of its time waiting for the upstream Node.js mock servers to respond, while Autocannon's JavaScript runtime was saturating CPU cores under high concurrency.

Furthermore, I was still running on Windows, forcing Go to interact with the Windows networking stack instead of utilizing native Linux epoll system calls, which is where Go's netpoll truly shines. The benchmark harness and the host OS had become the bottlenecks.

To eliminate these external constraints, I refined the test environment by:

• Migrated to Linux: I moved the entire testing suite into Ubuntu via WSL (Windows Subsystem for Linux) to allow Go to leverage Linux-native networking primitives.

• Swapped the Backends: I replaced the Node.js mock applications with lightweight native Go HTTP servers.

• Upgraded the Load Generator: I switched from Autocannon to wrk, a multithreaded C-based HTTP benchmarking tool with significantly lower client-side overhead.

• Isolated the Processes: I ran the benchmark client, Torus, and the upstream servers in separate terminal sessions to minimize OS CPU-scheduling contention.

These changes stripped away the JS runtime drag and the Windows networking limitations, producing the final benchmark result of 17,865 req/sec and dropping latency even further to 6.12 ms.

To put that in perspective: when I isolated the internal router and completely removed downstream network socket delays (returning an in-memory response), Torus's routing throughput hit 98,565 req/sec. The code wasn't slow; the environment was. Isolating the proxy proved what the Go engine was capable of when it wasn't being choked by JavaScript testing tools and Windows sockets.

Part 4: The Hard Numbers

Experiment A: Full Production Proxying Pipeline

This test exercises the complete reverse proxy pipeline: HTTP parsing, longest-prefix route matching, round-robin load balancing, TCP connection pooling, header injection, and zero-copy stream forwarding.

Test Setup: wrk -t2 -c100 -d10s against native Go mock backends.

Metric	Node.js Original	Go Rewrite (Final)	Improvement
Throughput	1,647.00 req/sec	17,865.81 req/sec	+984.7% (10.8×)
Average Latency	103.00 ms	6.12 ms	−94.0%
Max Tail Latency (P99)	488.00 ms	45.32 ms	−90.7%
Memory Footprint	~200 MB	~20 MB	10× reduction
Data Transfer Rate	—	2.01 MB/sec (178,892 total requests)	—

---

Production Proxy Throughput (req/sec)

Node.js      [████                                           1,647]
Initial Go   [███████████████                                6,044]
Final Go     [███████████████████████████████████           17,865]

Under full proxy execution, throughput increased by 10.8× while average latency dropped from 103ms down to 6.12ms. Memory consumption dropped from 200MB down to a stable 20MB under load.

Experiment B: Short-Circuit Routing Path

To measure the maximum throughput of Torus's internal router (header parsing, route matching, load balancing logic) independent of downstream network socket delays, I tested a scenario where downstream backends were marked dead. Torus short-circuited and immediately returned an in-memory error response.

Test Setup: wrk -t2 -c100 -d10s (No Live Backends).

Metric	Initial Go (Autocannon)	Final Go (wrk)
Routing Throughput	15,644 req/sec	98,565.91 req/sec
Average Latency	5.94 ms	10.21 ms
Max Tail Latency (P99)	78.00 ms	86.63 ms
Data Transfer Rate	—	15.70 MB/sec

Removing downstream network I/O allowed the Go routing engine to approach 100,000 req/sec on a basic local development machine. This confirms that Torus’s internal execution pipeline adds minimal overhead-downstream network delays account for virtually the entire latency profile of the proxy.

Lessons Learned

Building Torus in Node.js first was a valuable exercise. Node.js remains an exceptional environment for rapid prototyping, forcing developers to handle edge cases, connection pooling, and stream mechanics explicitly. However, selecting an infrastructure runtime requires aligning tools with system requirements:

1. JavaScript is for Application State; Go is for Networking Primitives. If an application manages business logic, database mutations, and complex JSON schemas, Node.js performs well. But when a system primarily shuttles bytes between network cards without mutating payloads, Go's pointer controls, fixed-size types, and native network stack offer a fundamental advantage.

2. Account for Load Client Overhead. If a benchmarking tool shares a runtime environment or CPU thread pool with the system under test, metrics will be distorted. Switching to wrk uncovered significant performance reserves masked by client-side event loop contention.

3. Memory Control Directly Impacts Latency Stability. Dropping baseline memory usage from 200 MB to 20 MB does more than save RAM, it eliminates GC-induced tail latency, stabilizing p99 metrics under sustained traffic spikes.

The full benchmark specifications, ADR records, and Go implementation are available in the repository.