This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady
What I Built
Most computers spend their day the way a sloth does: switched on, barely moving, conserving energy. Meanwhile, developers who want to run an AI model have to buy a GPU, rent a cloud instance, or babysit a server.
AgentMesh connects those two groups so that both can do less.
It is an open-source platform that runs open-weight AI models on computers contributed by independent providers and exposes each deployment through an OpenAI-compatible API.
- Developers get to be lazy. Pick a model, click deploy, and call it with the OpenAI client you already use. AgentMesh finds a machine with enough capacity, downloads and verifies the model, runs a test generation, and keeps it alive. If a node goes offline, the deployment moves to another one with the same ID, endpoint and keys. No servers to manage, no 3 a.m. restarts.
- Providers get to be lazier. Install a worker, set CPU, memory and disk limits and availability hours once, and walk away. The worker makes outbound connections only, so there are no ports to open. The machine serves models while it would otherwise sit idle, and the provider earns a share of every charge.
Demo
Calling a deployed model takes only your existing OpenAI client:
import os
from openai import OpenAI
client = OpenAI(
base_url="https://agentmesh-riuz.onrender.com/api/v1/deployments/<id>/v1",
api_key=os.environ["AGENTMESH_API_KEY"],
)
reply = client.chat.completions.create(
model="qwen2.5:0.5b",
messages=[{"role": "user", "content": "Summarise this ticket"}],
stream=True,
)
Code
AgentMesh
AgentMesh lets developers deploy open-weight AI models on computers contributed by independent providers and call them through an OpenAI-compatible API. Workloads run inside resource limits on the provider's machine; the provider can see the data a workload handles Billing is prepaid, with Paystack top-ups and provider payouts. These run in Paystack test mode and have not yet been exercised against Paystack's live API.
Documentation
The full documentation is in the dashboard at agent-mesh-ten.vercel.app/docs:
- Getting started: roles, accounts, and what works today
- Customer guide: deploy, call, and pay for models
- Provider guide: install the worker and manage machines
- API reference: authentication, deployments, inference, errors
- Operators & self-hosting and Security & limitations
The Markdown files in docs/ hold the same material in more depth.
Prerequisites
- Python 3.12+
- Node.js 20+
- Docker & Docker Compose
- PostgreSQL 16
- Redis 7
Quick Start (Development)
- Clone the repository
- Copy environment variables
…
How I Built It
AgentMesh has three parts:
- Control plane (API deployed on Render, dashboard on Vercel): authentication, scheduling, capacity reservation, health checks, scoped API keys, per-request usage metering, prepaid billing and an audit log. Provider workers connect outbound to the Render service, so no provider ever has to open a port.
- Provider worker: a Docker worker or an installable agent for Linux and macOS. It enforces the provider's resource limits and supports pause, drain and availability schedules.
- Model runtimes: Ollama and llama.cpp serve the models. vLLM support for GPU nodes is built but not yet verified on real GPU hardware.
When a developer deploys a model, the scheduler reserves CPU, memory and, where available, GPU capacity on a node that fits. If no node fits, it explains why. The worker pulls the model, checks the weights against a pinned digest, and runs a real test generation. Only then does the endpoint go live with chat, completions and streaming.
Small models such as Qwen2.5 0.5B and SmolLM2 run comfortably on ordinary CPU machines. That is exactly the kind of hardware sitting idle on desks and in dorm rooms.
Coming next: a deployment agent for maximum laziness
Developers will describe what they want in plain language, such as "a small chat model for 24 hours within my budget," and set limits on spend, duration, hardware and recovery attempts. Gemma 4, running locally through Ollama on an internal inference machine, will turn that request into a structured deployment plan and help diagnose failures.
The model only proposes. Every action with consequences, including pricing, wallet checks, node selection, capacity reservation and stopping a deployment, runs in the control plane's own code, and every limit is enforced in code rather than in the prompt. If the inference machine goes offline, requests wait in a queue, and running deployments, billing and spend limits keep working without the model.
The adapter, configuration and tests are in the repository, but the agent is not live for users yet. Its inference machine is still being configured, and the end-to-end flow has so far been tested with scripted model answers.
Why Does Open Innovation Matter?
AgentMesh is impossible with closed models.
A closed model lives behind its vendor's API. You cannot ship its weights to a volunteer's laptop, verify them, or run them on hardware the vendor doesn't control. Open-weight models make every part of AgentMesh possible:
- Any machine can serve. Because the weights are public, a community laptop in Abuja can run the same model as a data-centre GPU, at a smaller scale.
- Verification is real. Weights are pinned to exact versions and checked by digest, so you know exactly which model answered your request.
- Nothing is locked in. Developers can swap models freely, and anyone can self-host the entire network using the operator guide.
- Open runtimes do the heavy lifting. Ollama and llama.cpp make CPU inference practical, which is what lets ordinary idle computers join the network at all.
- Even the agent stays open. The planned deployment agent runs Gemma 4 locally, so planning doesn't depend on a paid closed API, and the model can be swapped with a single configuration change.
An honest trade-off. Distributed compute is not private compute. A provider can see the model and every prompt and response their node handles. Container isolation protects the provider's machine, not your data. AgentMesh is built for experimentation, prototyping and non-sensitive workloads. Confidential computing is on the roadmap but not available today.
Current status: AgentMesh is in open beta. Deployment, failover, API keys, metering, dashboards and provider workers all work today. Billing runs in Paystack test mode, and no price list has been published yet, so usage is measured but not charged.
Prize Categories
Render. The AgentMesh control plane runs on Render at agentmesh-riuz.onrender.com. It handles authentication, scheduling, capacity reservation, health checks, API keys, usage metering, billing and the audit log. Every OpenAI-compatible request passes through this Render-hosted gateway before it reaches a provider node.
Gemma. Gemma 4 is the reasoning model behind the AgentMesh deployment agent. It runs locally through Ollama on an internal inference machine and turns plain-language deployment requests into structured plans. The integration includes a dedicated adapter, a hardware doctor check, configuration for gemma4:e2b-it-qat and gemma4:e4b-it-qat, and tests. The agent is not live for users yet: its inference machine is still being configured, and the end-to-end flow has been tested with scripted model answers.
Thanks for reading! I'd love to hear from developers working on open-source AI and distributed systems: how can we make compute more accessible without compromising reliability, security or performance?
Top comments (0)