Day four of a two-week external assessment, scope is a few hundred subdomains wide, and the workflow hasn't changed since day one: subfinder to enumerate, httpx to probe what's alive, nuclei for the known stuff, then a manual pass through whatever's left. Same five tools, same order, every single subdomain, because that's what the methodology says to do. It's 11pm. The httpx output is forty pages long. Somewhere in page thirty-one is a subdomain serving a verbose 500 error with a stack trace in it, and it gets the same three seconds of attention as page thirty of things that turned out to be nothing.
That's the actual failure mode, and it isn't a skill problem. A tired, disciplined pentester running a fixed sequence against every target in the order it happened to resolve is going to miss things that a less careful person running the tools in the right order would catch, purely because fatigue doesn't respect where the interesting finding is sitting in the queue. The tool chain didn't fail. The order it was worked in did.
This is the specific gap between a script and an agent, and it's worth being precise about, because "AI agent" gets used loosely enough that the actual mechanism gets lost. A script runs a fixed sequence regardless of what comes back: step one, step two, step three, no matter what step one found. An agent runs a loop: observe what the last tool actually returned, decide what that result implies about what to try next, act on that decision, then observe again. The difference isn't that one uses AI and the other doesn't. It's that one can reprioritize mid-run and the other can't.
Concretely: give a model tool-calling access to your recon tools as functions it can invoke, feed it the structured output each one returns, and let the loop's decision step reorder the target queue based on what's actually showing up, not the order subdomains happened to resolve in. A response with an exposed .git directory, a verbose stack trace, or a login page running a CMS version with a known CVE gets bumped ahead of the pile of subdomains that returned a clean 200 and nothing else. The model isn't finding the vulnerability. It's making the same triage call a sharp analyst makes at 9am, at 11pm, consistently, run after run.
None of this works safely without guardrails that live outside the model's judgment, and skipping them is the actual risk here, not a footnote. Rate limiting and a scope allowlist need to be enforced by code the model can't talk its way around, and anything that could actually break something, a fuzz job, an intrusive nuclei template, needs a human decision point before it fires. An agent that decides on its own initiative to hammer an endpoint without throttling isn't an efficiency win, it's a liability with your client's production system on the other end of it.
The version of you skimming past page thirty-one at 11pm on day four is exactly the failure an agent loop with real guardrails is built to catch, because it doesn't get tired and it doesn't work the queue in resolution order. Codelivly's AI Hacking with Python walks through building exactly this: tool-calling scaffolding, structured observe-decide-act loops, and the recon and fuzzing pipelines that make the reprioritization real instead of theoretical.
Top comments (1)
Your exploration of the difference between scripts and agents is spot on, especially in the context of managing fatigue during long assessment sessions. Implementing a decision-making loop that prioritizes findings based on their relevance could greatly enhance efficiency and effectiveness, particularly when dealing with large outputs like your forty-page httpx results. I'd love to hear more about the guardrails you mentioned—finding that balance between automation and safety is crucial. If you're looking for support in advancing this tool or integrating further functionalities, I’d be happy to discuss a paid collaboration.