I’ve been working on AnchorShell Relay, and it’s finally live.
Relay is an AI gateway that sits between your apps or agents and the models they use. The goal is to make model traffic easier to route, control, and understand without adding another slow layer in front of every request.
At a high level, Relay can:
classify incoming requests and make smart model-routing decisions
route across multiple providers and local models
queue and pace requests against provider RPM limits
avoid unnecessary fallbacks when your preferred model is temporarily at capacity
apply limits and guardrails
track latency, tokens, usage, and cost
show requests and routing decisions in realtime
One of the main ideas behind Relay is queue-first routing.
If the model you actually want is available again in a second, immediately falling back to another model may not be the best decision. Relay can pace traffic and wait for available capacity instead of blindly moving down a fallback list.
The routing layer also has to stay fast. Relay ships with purpose-built local classification models for request intent and capability signals, and it can also use System One models such as Laya as an alternative classification engine.
The Community version is self-hostable and written in Go.
You can run it with:
npx @anchorshell/relay
Or build it directly from source.
GitHub:
https://github.com/anchorshell/relay
Website:
We also launched on Product Hunt today:
https://www.producthunt.com/products/anchorshell-relay
There’s a hosted version with a free tier as well, but the Community edition is designed to work as a standalone self-hosted gateway.
This is the first public product from AnchorShell, and I’d especially love feedback from developers working with agents, multiple model providers, or high-volume AI workloads.
I’m particularly interested in opinions on the smart-routing approach, queue-first behavior, and what you’d want to see before putting something like Relay in front of production traffic.
Top comments (0)