p2p-ai — Distributed Peer-to-Peer AI Inference Network

AVINASH S KARANTH — Fri, 22 May 2026 08:16:33 +0000

p2p-ai — Distributed Peer-to-Peer AI Inference Network

A submission for the Gemma 4 Challenge: Build with Gemma 4

What I Built

p2p-ai is a distributed peer-to-peer AI inference network. Instead of relying on a centralized cloud API, every client that has the model can act as an inference provider for others. When your device doesn't have the model (or is already busy), it can offload generation tasks to available peers in the network — with zero task content ever touching the server.

The architecture is fully end-to-end encrypted: a Node.js server acts only as a signaling layer and peer registry, while all prompts, tokens, and outputs flow directly between peers over WebRTC data channels. The server never sees the content — only metadata like token counts and job duration for performance analytics.

Key features:

On-device inference via MediaPipe LLM Inference API in the browser
Real-time token streaming from provider to requester over PeerJS DataConnections
Hybrid RSA-OAEP + AES-256-GCM encryption per task — a fresh AES key is generated for every request
Fair provider cycling so load is distributed evenly across the pool
Web and mobile clients (web is fully functional; React Native client is scaffolded for LiteRT-LM)

Demo

Live deployment: https://p2p-ai.nash.dpdns.org/

The project is self-hosted: the server runs Node.js + MariaDB, and the web client is served statically. You can spin it up locally with Docker or pnpm start + pnpm web. A short video walkthrough is coming soon.

Code

Repositories:

Web: https://github.com/AvinashSKaranth/p2p-ai
Mobile (WIP): https://github.com/AvinashSKaranth/p2p-ai-react-native

Stack:

Layer	Technology
Backend	Node.js, Express, MariaDB, PeerServer
Web client	Vanilla JS ES modules, PeerJS, MediaPipe Tasks GenAI
Mobile client	Expo + React Native (LiteRT-LM bridge planned)

How I Used Gemma 4

I chose Gemma 4 E4B IT (gemma-4-E4B-it) as the inference engine.

Why E4B was the right fit:

It runs in the browser. The E4B variant is small enough to download, cache in the Origin Private File System, and execute with the MediaPipe LLM Inference API via WebGPU/WASM — no server-side GPU farm needed.
Quality at the edge. The instruction-tuned version gives good chat completions for summarization, translation, and creative writing directly on user hardware, which is exactly what a P2P network needs: capable providers without cloud costs.
Mobile symmetry. The same E4B checkpoint is available for LiteRT-LM, so the upcoming React Native client can share the same model weights and behavior as the web client. A single model family keeps the network homogeneous — any provider can serve any requester.
IT (instruction-tuned) for generic tasks. The network is task-agnostic at the transport layer; requesters construct their own prompts. Using an instruction-tuned model means the provider can handle arbitrary prompts without custom fine-tuning per task type.

The web client downloads the model from HuggingFace on first use, stores it locally via OPFS, and then registers itself as a provider. If the model isn't present, the client operates purely as a requester and offloads to available peers. This "compute when you can, borrow when you can't" design is what makes Gemma 4 the perfect backbone for a privacy-first, decentralized inference mesh.

Evolution: Scaling Toward a Large-Scale Distributed Intelligence

The current architecture proves the concept of privacy-preserving peer inference. The following outlines how this foundation can evolve into a large-scale operation capable of distributed summarization, translation, index building, and beyond.

Large-Scale Task Pipelines

The transport layer is already task-agnostic — providers execute whatever prompt they receive. Scaling to fleet-level workloads means introducing a task queue and coordinator layer on top of the existing peer registry:

Summarization at scale: Long documents can be chunked client-side, dispatched to multiple peers in parallel (map), and reassembled by the requesting node (reduce). The server coordinates chunk routing without ever seeing content.
Translation pipelines: Language pairs can be broadcast as metadata so providers with multilingual model variants (or fine-tuned checkpoints) self-select for relevant tasks, improving quality without centralized routing logic.
Index building: Each peer can emit structured summaries or entity extractions from processed documents. These fragments can be aggregated incrementally into a distributed inverted index, with the server maintaining only routing tables and shard assignments.

Embedded Browser Agent

A natural evolution is building a browser agent directly into the p2p-ai client. Rather than just processing prompts, the agent would:

Open a URL and extract the full readable content from the page (article text, headings, metadata).
Run a local summarization or entity-extraction pass using the on-device Gemma model.
Package the result (title, URL, summary, key terms) into a structured document ready for indexing.

This transforms every participating device from a passive inference provider into an active knowledge harvester — contributing structured intelligence back to the network as a byproduct of normal browsing.

Federated Open Search Index

The output of the browser agent feeds directly into a federated open search layer. Rather than a single centralized index, results are pushed to one or more open-source search backends:

Mwmbl — a community-maintained, open-source search engine explicitly designed for crowdsourced crawling. p2p-ai peers become crawlers contributing to the public index.

The architecture is pluggable: the peer emits a structured indexing event (title, URL, excerpt, embeddings) and a thin adapter routes it to whichever backend is configured. Privacy is preserved because the content processing happens on-device; only the structured summary is transmitted to the index.

The Full Vision: Device-Native Intelligence Mesh

Combining all three layers — P2P inference → browser agent → open search index — produces a system where:

Every device in the network contributes both compute (running inference for peers) and knowledge (harvesting and indexing web content).
No single entity controls the index or the inference capacity.
Users retain full ownership of their browsing data; only opt-in summaries are shared.
The collective result is a continuously growing, crowd-sourced, AI-enriched knowledge base — built from the aggregate idle capacity of thousands of devices, without any centralized cloud infrastructure.

This is the long-term direction for p2p-ai: not just a demo of edge inference, but a blueprint for decentralized, device-native AI at internet scale.

DEV Community: AVINASH S KARANTH

p2p-ai — Distributed Peer-to-Peer AI Inference Network

p2p-ai — Distributed Peer-to-Peer AI Inference Network

What I Built

Demo

Code

How I Used Gemma 4

Evolution: Scaling Toward a Large-Scale Distributed Intelligence

Large-Scale Task Pipelines

Embedded Browser Agent

Federated Open Search Index

The Full Vision: Device-Native Intelligence Mesh