I built an offline P2P system to run 70B LLMs by pooling mismatched consumer laptops

Mohit — Tue, 20 Oct 2026 03:40:09 +0000

I built an offline P2P system to run 70B LLMs by pooling mismatched consumer laptops

Running massive, smart open-source AI models natively on a standard laptop is a painful experience. If you try to load a heavy coding model on a base 16GB MacBook Air or a standard gaming laptop, the system immediately runs out of RAM, stutters, or throws a brutal Out of Memory crash.

The tech giants want us to think the only solution is handing over a credit card to pay for expensive cloud APIs or renting virtual cloud servers.

I got sick of that limitation, so I built an open-source system utility to bypass it entirely: AirDrop-LLM.

It is a zero-dependency, offline, multi-node pipeline-parallel inference broker written in Java. It allows resource-constrained edge devices to pool their hardware resources (RAM, VRAM, and GPU compute) with other heterogeneous consumer machines over a local smartphone hotspot or raw Wi-Fi Direct network—100% offline, with no cloud dependencies.

How it works under the hood

Instead of trying to force one laptop to carry an entire massive model, AirDrop-LLM splits the model layers across a local device cluster:

Automated P2P Discovery: When the background daemon starts up on a hotspot LAN, it uses UDP Multicast Broadcasting to instantly detect other peers, exchange local hardware specs, and map the cluster topology dynamically.
Low-Overhead Binary Streaming: Communication is handled via raw TCP sockets optimized for low-latency binary streams. Node 0 loads and computes the first set of layers, serializes the intermediate activation tensors, and blasts them directly over the wireless pipe to Node 1 to finish the generation loop.
Asymmetric Layer Allocation: The scheduling engine automatically runs a network performance profile loop (blasting a 10MB test payload) before sessions begin. It balances the workload based on hardware capacity—routing lighter execution chunks to standard system RAM and heavier allocations to dedicated high-throughput VRAM (like an Nvidia CUDA card) to prevent pipeline stalls.
Fault-Tolerant Local Fallback: If a friend's laptop drops off the hotspot network or a battery dies mid-sentence, the daemon catches the socket drop instantly. It hot-swaps the execution target down to a highly optimized local standalone model (like Qwen 2.5 Coder 1.5B) executing entirely within the host machine's internal memory, keeping your IDE interface alive without crashing.

Try it out!

The framework is fully open source under the MIT license and works entirely in your bedroom without passing a single line of your private code to a third-party server.

If you have Node.js and standard compilation tools installed, you can spin up a local cluster right now using npm:

npm install -g airdrop-llm
airdrop start

Check out the full codebase, system layer specifications, and architecture layout here:
👉 GitHub Repository: https://github.com/Mohit-jpg-dot/AirDrop-LLM

I am a Computer Science Engineering student working on low-level distributed infrastructure and AI tooling. I would love to get your feedback on the architecture, tensor serialization optimization, or network overhead! If this helps you break past your local memory limits, please drop a star on the repository—it keeps the project moving forward!

DEV Community: Mohit

I built an offline P2P system to run 70B LLMs by pooling mismatched consumer laptops

I built an offline P2P system to run 70B LLMs by pooling mismatched consumer laptops

How it works under the hood

Try it out!