I’ve been building Halo, an autonomous pentesting agent powered by a local LLM (Gemma 4 12B, abliterated, running via LM Studio). Last week I found a bug in it that taught me more about state management in agentic systems than anything else I’ve hit so far.
The symptom
I was testing a tool — httpx, used for HTTP probing — standalone from the terminal. Worked fine. Ran it through the full agent loop against the same target. The agent refused to even try it. No error, no retry, just… skipped.
At first I assumed it was a prompt issue — maybe the LLM just wasn’t selecting the tool. But when I dug into the logs, I found something stranger: the agent’s reasoning explicitly referenced the tool as “previously failed” — except it had never run against this target before.
The investigation
Halo has a failure cache (agent_cache.py) that fingerprints failed tool runs using SHA-256 hashes, so the agent doesn’t waste cycles retrying things that already didn’t work. Reasonable design — except when I traced the fingerprint logic, I found the cache key had no concept of which engagement a failure happened in.
That meant if httpx failed once against Target A (say, due to a transient network blip or a misconfigured flag), it was blacklisted globally — not just for Target A, but for every future target, forever, across completely unrelated engagements.
The root cause
The cache was scoped at the tool level only: tool_name + target as the fingerprint. It should have been: engagement_id + tool_name + target. Without engagement scoping, one bad run anywhere poisoned the well everywhere. The agent wasn’t being cautious — it was permanently and silently giving up on tools that had simply had a bad day once.
This is a classic state-management trap in agentic systems: caching for efficiency is good, but if your cache key doesn’t match the actual scope of validity for that data, you get the appearance of stability while quietly accumulating false negatives. And the worst part is it fails silently — there’s no crash, no obvious symptom, just a system that gets less capable over time without telling you why.
The fix
Added engagement_id as a required field threading through agent_cache.py and every call site in agent_loop.py. Each engagement now gets its own failure namespace. A tool that fails against one target is still blacklisted for that engagement (so the agent doesn’t waste time retrying within a session) but starts fresh on the next one.
Before the fix:
cache_key = hashlib.sha256(f"{tool_name}:{target}".encode()).hexdigest()
After the fix:
cache_key = hashlib.sha256(f"{engagement_id}:{tool_name}:{target}".encode()).hexdigest()
(Simplified — the real implementation also classifies failure type: timeout, permission denied, tool missing, network error, etc., so the agent can make smarter decisions about when to retry, not just whether to retry.)
What it taught me
If you’re building any kind of stateful agent — caching, memory, learned preferences, whatever — ask explicitly: what is the actual scope of validity for this piece of state? It’s tempting to cache broadly because it feels more efficient, but a cache that outlives its true scope doesn’t just waste effort, it actively corrupts future decisions. The agent wasn’t broken. It was being “smart” with the wrong boundaries.
Halo is still very much a work in progress — open source, local-first, no cloud dependency for the reasoning loop. If you’re working on anything agentic with persistent state, I’d be curious how you’ve handled scoping problems like this.
XenoCoreGiger31
/
GEMMA-by-GOOGLE
A fully local, autonomous AI penetration-testing agent powered by Gemma 4-12B and a Flask MCP tool server. Runs recon, attack loops, and report generation on its own — no cloud, no API keys. For authorized security testing only.
🔐 GEMMA-by-GOOGLE — HALO
A fully local, autonomous AI penetration-testing agent — Gemma 4-12B driving a 29-tool arsenal through recon, attack, and reporting, exposed as a standard Model Context Protocol (MCP) server. No cloud, no API keys.
What It Does · Tools · Architecture · Stack · Quickstart · Contributing
HALO is an autonomous security agent that runs inside a Linux environment driven by a local LLM — Gemma 4-12B (uncensored / abliterated) served through LM Studio. It plans, runs reconnaissance, chains attacks based on what it finds and writes a professional pentest report on its own. Everything runs locally no cloud, no API keys, nothing leaves your machine.
One word starts an engagement: engage.
What It Does
- 🔍 Autonomous recon — masscan + nmap to discover open ports and services
- ⚔️ Autonomous attack loop — selects and chains tools based on what it finds
- 🧠 Persistent negative-experience cache…
Top comments (1)
If anyone’s interested in contributing fresh ideas or just want to dig into this with me, all contributions are welcome — I won’t take more than 24 hours to respond to comments, PRs, or commit requests. I use this agent every day, all day, in my own workflow, so I’m pretty invested in making it better. Let’s see where we can take it together.