A lot of agent implementations right now have static permissions. If an agent's allowed to run migrations, it stays allowed to run migrations, until a human happens to notice something's wrong.
SRE teams don't run production like that. Burn through your error budget and deployments freeze automatically, nobody has to catch it manually. So for the Agents of SigNoz hackathon, I built LEASH to see what happens if an agent's permissions worked on a similar principle: retained through sustained healthy execution, and lost automatically the moment that breaks down.
This runs in a simulated staging environment with failure injection I control. The goal was proving permissions can change mid-run, without a human in the loop, driven entirely by the alert conditions SigNoz evaluates from real telemetry.
The idea
LEASH is a broker that sits between the agent and every tool it's allowed to call, enforcing decisions triggered by alert conditions evaluated from SigNoz telemetry. Static permissions assume authorization doesn't need to adapt at runtime. LEASH treats an agent's permission tier as runtime state that changes based on observed behavior.
The authorization policy itself is intentionally simple, a threshold and a window. The novelty isn't the policy, it's closing the feedback loop between observability and authorization.
When failures cross a threshold, SigNoz fires an alert, hits a webhook on the broker, and the broker downgrades the agent's tier before its next privileged calls are authorized. LEASH effectively turns observability into a feedback controller for agent permissions. Observability is no longer just something you check after the damage is done, it decides what the agent's allowed to do while it's still running.
I split this into four separate services on purpose. An agent shouldn't be able to touch the system deciding its own permissions, keeping them genuinely separate processes was the point, not an implementation detail. LEASH itself runs as a standalone HTTP service the agent calls through for every tool invocation, not a sidecar or proxy sitting transparently in the network path.
Tiers
T3 — read, write, destructive cleanup
T2 — read and write, destructive calls blocked
T1 — read-only
Agent runs at T3 normally. The broker tracks the current consecutive-failure streak per tool and exports it as a metric, SigNoz's alert rule watches that. After 3 consecutive migration failures inside a 5 minute window, SigNoz fires, and in testing, the whole pipeline, failure to alert to webhook to tier downgrade, lands in under 10 seconds. Try a destructive call after that and you get a hard 403, with the exact trace ID of the failures that caused it attached as evidence.
That trace ID isn't decoration. Paste it into SigNoz and you see the actual failed spans behind the decision. The agent isn't refused because a prompt told it to behave, it's refused because there's a real record sitting in SigNoz.
Recovery isn't built yet, downgrade is one-directional within a run right now. Automatic recovery after a clean streak is the obvious next piece, and it'll need real hysteresis, not just a raw success count, or a fast recovering agent could flap between tiers on noisy data.
Why OpenTelemetry specifically
The broker doesn't have its own monitoring protocol. It just emits standard OTel telemetry, and SigNoz already understands it, no custom integration on either end. That's what let SigNoz's alert engine drive the authorization decision, instead of remaining something you'd only inspect after an incident.
Getting there wasn't the policy logic, that part's genuinely simple. It was the webhook. Docs tell you SigNoz alerts can call one, they don't tell you the exact JSON shape you get for your specific alert type. Had to trigger a real failure, watch the alert actually fire, and read what landed on my endpoint before the handler worked right. Also learned a 60 second window just reacts to noise, 5 minutes with 3 consecutive failures is what turned it into a real signal.
Scope
LEASH isn't trying to figure out if an agent is aligned. It's only deciding whether the agent retains permission to keep doing higher-risk things.
That's a real limit, worth being honest about. It reasons through execution health, tool failure rate, nothing more. An agent that repeatedly fails migrations loses privileges. An agent that succeeds at every call while doing the wrong thing doesn't, deleting the right table for the wrong reason still looks like a healthy 200 to this system. Closing that needs richer signals than pass or fail, future versions could incorporate policy violations or anomalous tool sequences.
What I'd change next
Tool risk tiers are hardcoded right now, a real deployment needs that configurable without redeploying the broker. There's one alert rule for one failure mode. The natural next step, generalize this so any policy-relevant metric crossing a threshold can demote any agent, plus an actual path back up with proper hysteresis.
Takeaway
A prompt telling an agent not to do destructive things after errors is a suggestion, not enforcement, the model can ignore it or misread it. What actually stops something is a system outside the model, reading real telemetry, blocking the call before it executes. That's what SigNoz's alerts and OpenTelemetry's traces gave me here, and a prompt never could.
Code: github.com/Vaibhav13Shukla/LEASH
Built for the Agents of SigNoz hackathon, Track 1.
Top comments (0)