RelayPilot: An AI Night-Shift Engineer for LLM Relay Stations

Haoyi Zhu — Tue, 13 Oct 2026 04:14:46 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle.

What I Built

RelayPilot is an AI ops copilot for people who run LLM relay stations — the small, often one-person services that sit between users and model providers, juggling API keys, channel quotas, and bills.

I know this pain firsthand: I run a free public relay station myself. RelayPilot is the night-shift engineer I always wished I had.

It does four things:

Channel health monitoring — probes every upstream channel on a schedule and tracks latency, error rates, and quota burn, so a dying channel gets caught in minutes, not hours.
Automatic quarantine — an open-weight model classifies key usage patterns locally; when a key suddenly starts behaving like a stolen one (traffic spikes at 3 AM from a new region), RelayPilot quarantines it automatically and pings the owner.
Natural-language ops queries — "which channel burned the most tokens yesterday?" or "is the DeepSeek channel healthy right now?" answered in plain language, no dashboard spelunking.
Telegram alerts — the things that actually need a human get pushed to your phone; everything else stays in the log.

Who would pay for it: anyone running a relay station or reselling API access — indie hackers, small dev-tool teams, campus AI clubs. The pitch is simple: one prevented quota-burn incident pays for a year. I'm launching with a free self-hosted tier (bring your own server) and a $9/month hosted tier that handles monitoring, alerting, and multi-tenant billing. My first user is me.

Demo

62-second terminal demo of the simulated key-leak scenario (SIMULATION_MODE=1 — no real keys were harmed): channel probes → normal traffic → sk-dave's key starts leaking at 03:00 from a new region → WARNING → auto-quarantine + CRITICAL Telegram alert → natural-language queries → daily digest.

Code

https://github.com/2951461586/relaypilot — MIT licensed, repo created Oct 13, 2026, inside the challenge window.
Stack: Python, FastAPI, Ollama (open-weight models, local inference), python-telegram-bot, SQLite. Channel adapters forked from the open-source Veloera/New API gateway.

How I Built It

Anomaly detection, local by design. A small open-weight model (Qwen3 via Ollama) classifies per-key usage windows — request rate, token profile, time-of-day, geography — into normal/suspicious. It runs entirely on the operator's own machine, with automatic fallback to built-in heuristic rules when Ollama is unreachable. No usage telemetry ever leaves the server, which matters because key logs are the crown jewels of a relay business.
Deterministic where it counts. The model only classifies; all money-adjacent decisions (quarantine thresholds, quota math, billing) are plain code. An LLM never touches arithmetic it can get wrong.
Veloera adapters. Instead of writing channel integrations from scratch, I built on the open-source Veloera gateway's channel abstraction, so RelayPilot speaks the same plugin interface as the relay itself — any Veloera-compatible station can drop it in.
Telegram as the UI. Operators don't want another dashboard; they want their phone to buzz only when something is actually wrong. Alert fatigue was the design enemy: three severity levels, quiet hours, and a daily digest.
A static landing page ships in the repo (site/) and is headed for relaypilot.tech — the .tech domain I claimed through the Hacktoberfest promo.

Why Does Open Innovation Matter?

Four reasons, all of which are load-bearing for this business — not decoration:

The data can't leave the building. Key usage logs contain everything an attacker needs. Sending them to a third-party API for "AI monitoring" would be malpractice. An open-weight model running locally means the security tool doesn't become the security hole.
The unit economics only work at zero marginal cost. Monitoring means classifying thousands of usage windows a day, forever. At paid-API prices, the monitoring bill would exceed the relay's own margin. Local inference makes the $9/month tier actual profit instead of a loss leader.
Operators need to swap models, not vendors. When a better small classifier drops, I swap a model file — no contract renegotiation, no API migration, no retraining pipeline owned by someone else. The whole product is model-agnostic by construction.
Trust is the product. Relay operators are trusting RelayPilot with their keys and their users. Every line of the detection logic is inspectable; nothing phones home. You can't sell "we watch your keys" as a closed black box.

My Agent Session

I built this with an AI agent under my direction: I specified the architecture and requirements, the agent wrote the code, and I verified it end-to-end with the simulated leak scenario above. (No DevRelay session link — the build happened in my own environment.)

Prize Categories

Overall — every valid submission is considered for the overall prize; entering here.
Best Use of a .tech Domain — relaypilot.tech, claimed through the Hacktoberfest promo, is this project's domain. The landing page goes live there next.

DEV Community: Haoyi Zhu