NodeShare AI: Building a Distributed AI Compute Network from Existing PCs

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

NodeShare AI is an experimental distributed AI inference platform designed to help developers explore running open-weight AI models using compute resources across their own computers.

The problem I'm exploring is simple: developers often depend on paid hosted AI APIs or dedicated GPU servers to experiment with AI. But what if they could coordinate existing computers and use them as part of their AI infrastructure?

NodeShare AI consists of three main components:

• Cluster Controller: A Python/FastAPI service that manages node information, exposes inference APIs, and coordinates job dispatch.
• Go Node Agent: An agent designed to run on participating computers and report their hardware and status to the controller.
• Local Inference: An architecture built around llama.cpp and GGUF-compatible open-weight models.

The project is intended for independent developers, students, freelancers, and small teams who want to experiment with self-hosted AI.

A potential business model is a free, self-hosted community edition with paid setup, private deployments, support, and managed orchestration services. This is a proposed business model, not existing revenue.

The project is currently experimental. Live node enrollment, end-to-end inference, and multi-node execution are verification milestones rather than capabilities I'm claiming are fully production-ready.

Demo

🌐 Project website: https://nodeagent-five.vercel.app/

☁️ Cluster controller: https://llm-cluster.onrender.com

The frontend is hosted on Vercel, while the Python cluster controller is hosted on Render. Participating computers are intended to provide the local inference compute.

The complete frontend-to-inference workflow still needs further end-to-end verification.

Code

💻 GitHub repository: https://github.com/Mahesh00234h/nodeshare-ai

The repository contains the project implementation, including the Python controller and API layer.

I'm building the system around a separation between coordination and computation, making it possible to investigate different node configurations, local models, and scheduling strategies.

How I Built It

I approached NodeShare AI as a distributed systems project rather than just another AI chat interface.

Technology stack:

• Python and FastAPI for the cluster controller and HTTP APIs.
• Go for the node-agent component.
• llama.cpp and GGUF models for local inference.
• REST APIs and an HTTP/gRPC-oriented communication architecture.
• Render for hosting the controller.
• Vercel for hosting the web interface.

The intended execution flow is:

1. A client sends an inference request to the controller.
2. The controller evaluates available node information and attempts to select a suitable compute node.
3. The controller creates and dispatches a job.
4. The node agent and local inference runtime are intended to execute the workload.
5. The result is returned through the API.

One important distinction is that selecting a node for a request is not the same as splitting a single model across multiple computers. I'm treating distributed execution, reliability, and resource-aware scheduling as areas to develop and validate further.

Why Does Open Innovation Matter?

Open innovation gives developers the freedom to inspect, adapt, and improve the infrastructure around AI instead of depending entirely on a single provider.

Open-weight models and local inference make it possible to experiment with model choices, deployment environments, and data-processing boundaries. Existing computers can become useful experimental resources without immediately purchasing dedicated GPU servers.

For privacy-conscious users, local inference can also provide greater control over where data is processed, provided the deployment is configured securely.

Open-source tooling makes this experimentation possible. Developers can inspect the code, change components, contribute improvements, and adapt the system to their own hardware.

There are real trade-offs: electricity, hardware limitations, networking, security, and reliability all matter. My goal is to investigate when coordinating existing compute resources can make AI infrastructure more accessible, flexible, and practical.

Prize Categories

I'm considering the Best Use of Render category because the cluster controller is hosted on Render.

I'll confirm eligibility against the official partner-category rules before finalizing this entry.