This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady
What I Built
Hive is an open-source tool for training a machine learning model on data that never leaves the devices it lives on.
A host starts a training session with one command. Volunteers join, point the client at a folder of their own recordings, photos, or logs, and their machine trains the host's model locally. Only a small mathematical update goes back to the host, whose server combines the updates from everyone into one improved model.
Take Elena, a researcher with a bird-call classifier and no dataset. A hiker, a backyard birder, and someone who walks a forest trail each have days of recordings. None of them wants to upload gigabytes of audio, and Elena doesn't want to store or secure it. With Hive, each of them installs a client, picks a folder, and sets some limits. Their laptops train the model while idle, and Elena's model improves without a single recording changing hands.
Both sides get to do less. Contributors set it up once: idle time only, a CPU cap, pause on battery, a fixed number of rounds. Then they walk away. Hosts skip collecting, cleaning, storing, and legally protecting a pile of other people's data, and they skip renting a GPU cluster to train on it.
The same setup works for anything where data is scattered, sensitive, or legally stuck: hospitals that can't pool patient records, private document collections, community science projects.
Demo
Host Dashboard
Contributor Dashboard
Code
Hive
Train smarter AI models together — without sharing your data.
Hive is an open-source tool that lets people collaborate to train machine learning models, even when their data is private and cannot be shared. Think of it like a community science project: many volunteers contribute computing time and local expertise, but their raw recordings, photos, or health logs never leave their own devices.
What problem does it solve?
Building a good AI model usually requires a large, diverse dataset. Gathering that data in one place is expensive, often legally problematic, and always a privacy risk. Hive removes that bottleneck:
-
For model builders (Host): Start a training session with one command. Volunteers join from anywhere, train the model on their own data, and send back only a tiny mathematical summary of what they learned — not the data itself. Host authors the model, signs the
.hivepkgpackage with an Ed25519…
MIT licensed. The README.md has a detailed overview, and user-guide.md documents on real usages and deployments.
How I Built It
Hive is a Python package with one hive CLI (hive host, hive join, hive keygen, hive package, hive doctor, hive demo). The host and contributor sides are separate halves of the same codebase.
The host runs an async round orchestrator that handles deadlines, quorum, claims, aggregation, and checkpoints, with state kept in SQLite. Updates are combined with FedAvg by default, and Trimmed Mean, Median, Krum, and Multi-Krum are there for the case where some contributors send bad updates. The host dashboard (127.0.0.1:8000) shows loss and accuracy curves, active nodes, each contributor's share, and a model weight verifier that flags a tampered checkpoint on disk. The host can also expose the session through LAN discovery, a Cloudflare Quick Tunnel, or a manual public URL.
The client is the part I spent the most time on, because the host's model code runs on a stranger's computer. That code is treated as untrusted. It runs in a separate process (or a Docker container) with no network access and a scrubbed environment. Before anything runs, the contributor can read the Python in a read-only viewer, see static lint warnings from an AST check, and compare the host's Ed25519 author fingerprint against one they received out of band. The client pins that fingerprint, so a swapped package gets rejected.
Packages ship as .hivepkg files, which are deterministic signed archives with a dependency allowlist and safe extraction against tar-slip, symlinks, and zip bombs. Tensors travel as safetensors, never pickle.
For privacy, the worker clips each update's L2 norm and adds Gaussian noise before it leaves, and there is a Renyi DP accountant on the host. A "what leaves my device" panel shows the contributor the exact payload before upload. The host side stores no IP addresses.
The client also watches the machine. It detects idle time on Windows, macOS, and Linux, respects schedule windows, pauses on battery, duty-cycles the CPU, and suspends the training process mid-round when you start using the computer again.
Why Does Open Innovation Matter?
The whole pitch of Hive is "trust us with your data's shadow, not your data." That only works if people can check the claim, and closed software can't offer that.
A contributor is being asked to run someone else's training code on a machine that holds their health logs or voice recordings. In a closed system they'd have to take the platform's word that nothing leaks. Here they can read the model code before agreeing to it, read the client that sandboxes it, and read the payload preview that shows what will be uploaded. The privacy guarantees are in code anyone can audit, rather than in a policy page.
It also decides who can host. Federated training through a hosted API means the host depends on a provider's pricing, quotas, and terms, and the sensitive-data use cases often can't be sent to a third party at all. With Hive, the host runs the server on their own machine, brings whatever model they like, and swaps it for another without asking anyone. The tool provides the machinery and the host provides the resources, so nobody owns the pipeline except the people using it.
And the compute comes from hardware people already own. A hobbyist with a bird model and a few volunteers can train something real without a cloud bill, which is the only way this kind of community project gets started in the first place.
The Lazy Payoff
For a contributor, the effort is a few minutes once: pick a folder, accept the sandbox notice, set the idle and CPU rules. After that, training happens when the machine would otherwise sit unused and stops when they come back.
Top comments (0)