Why 1 Gbps Ethernet Doesn't Bottleneck Distributed Local LLMs (Weights vs. Activations)

RAMDeck eliminates the network bandwidth bottleneck in distributed local AI by keeping model weights parked permanently in local memory while passing only micro-activations between machines.

Whenever developers discuss pooling RAM across multiple computers to run 30B–120B local models without buying an expensive workstation, the first networking objection is always the same:

"Ethernet is too slow. DDR5 runs at 60–90 GB/s and modern GPUs push 500+ GB/s, while 1 Gbps Ethernet caps at 0.125 GB/s. Won't the cluster choke on the network?"

If you are mapping an operating system pagefile, gaming asset stream, or database swap file across the network, that objection is 100% right. Traditional virtual memory freezes because the OS constantly fetches gigabytes of random memory pages across the wire.

Large Language Models do not work like OS swap drives.

1. Layer-Pipeline Parallelism: Weights vs. Activations

During distributed token generation with RAMDeck:

1. Zero Weights Cross the Wire During Generation: When an 18 GB 30B model or a 42 GB 70B model loads across two or three computers, each machine holds its assigned transformer layer blocks permanently in its own local VRAM or system RAM.
2. Full Local Silicon Bandwidth: Node A evaluates its assigned layers by reading weights from its own local memory bus at full hardware speed (500+ GB/s on discrete GPUs, or 100–400 GB/s on unified-memory Apple Silicon).
3. Only Micro-Activations Travel: When Node A completes its layer slice, it transmits only the compact hidden state vector (activation tensor) to Node B. For 30B–70B models, this tensor is roughly 3 KB to 8 KB per step.
4. The Latency Math: Sending a 4 KB activation tensor over standard 1 Gbps Ethernet takes 0.03 milliseconds. Meanwhile, physical GPU layer computation takes ~25–35 milliseconds per token. Even over a 6 ms regional Tailscale connection, network transit stays tiny relative to compute—and in practice we measured 28 to 39 tokens/sec on 30B MoE locally and 31 tokens/sec across remote nodes over Tailscale.

2. Single-Machine Native Parity

A common hesitation with distributed inference runtimes is whether using them on a single computer sacrifices native speed.

On one machine, RAMDeck operates in pure standalone mode with full GPU offload and hardware KV cache reuse. In back-to-back single-machine benchmarks on Apple Silicon M4 running Qwen2.5-Coder-7B, RAMDeck delivered ~20 tokens/sec generation parity and sub-50 ms prompt verification latency matching Ollama—so there is zero penalty while running solo, and zero migration pain when bridging a second machine later.

3. Why a Dedicated Under-5W Hardware Hub?

Instead of leaving a 500W desktop running 24/7 as a cluster master or hand-maintaining fragile CLI scripts, RAMDeck pairs desktop/mobile Contributor Node Agents (https://ramdeck.com/downloads/) with a dedicated under-5W control-plane Hardware Hub (https://ramdeck.com/, $199 retail · 134% funded on Indiegogo · ships in January with 1 Free Official RAMDeck Node USB):

• Zero-Config Discovery: Discovers Windows, macOS, Linux, iOS, and Android nodes automatically over LAN or 1-click Tailscale.
• Dynamic Primary Selection: Automatically assigns the fastest compute device on your network to host the active inference context.
• Self-Healing Watchdog: Cleans up disconnected sockets when laptops sleep or Wi-Fi roams, recovering cleanly without crashing active clients.
• OpenAI-Compatible Gateway: Exposes a single /v1/chat/completions endpoint for Cursor, Continue, Cline, Open WebUI, and standard local AI workflows.

Explore the full architectural breakdown and interactive memory calculator at ramdeck.com and check the open-source public components at github.com/trademav/ramdeck-core-public.

— The TradeMAV Team