Three days into a two-week external assessment, the recon phase has done exactly what it's supposed to and produced a problem of its own: a subdomain enumeration run plus a mass port scan came back with several thousand hostnames, a pile of open ports across a dozen ranges, and a handful of tech-stack fingerprints per host. The engagement clock doesn't stop for triage. Somebody has to turn that dump into a short list of things worth actually looking at, and doing it by hand, host by host, is how a two-week engagement burns four days before a single request gets sent.
The tempting shortcut is pasting the raw list into an LLM with something like "find the vulnerabilities in this." It doesn't work, and it fails in a specific way worth understanding rather than just avoiding. A model handed four thousand undifferentiated hostnames has no scope, no context on which of those are in-scope assets versus third-party CDN entries, and no way to tell a genuinely stale staging box from a routine subdomain that just happens to have an old server header. What comes back is confident-sounding and mostly generic: OWASP Top 10 boilerplate attached to hosts that were never actually inspected, because the model is pattern-matching on the shape of the question, not reasoning about the specific data.
What actually works is narrower and less impressive-sounding: split the triage into small, bounded questions instead of one broad one. "Which of these subdomain names follow a staging, dev, test, internal, or old- naming pattern, independent of anything else" is a question a model answers reliably on a few thousand strings, because it's pure pattern classification, not vulnerability judgment. "Group these open-port and service-banner combinations by which ones suggest an outdated or default-configured service, based on the version strings present" is another one it handles well, for the same reason: it's summarization and clustering over data actually in front of it, not inference about exploitability it wasn't given evidence for. Each answer is a shorter list to check by hand, not a verdict to trust and move on from.
The discipline that makes this safe is treating every output as a lead, never a finding. A model flagging staging-old.example.com as worth a look is doing exactly the job it's suited for, cutting four thousand items down to forty. Whether that host is actually running an abandoned application with a real vulnerability is still something a person has to go check by hand, the same way it always was. Skip that verification step because the summary looked authoritative and the fast recon phase turns into a false positive reported to a client, or worse, a real finding that got waved past because the model's confident tone read as verification it never actually did.
That's the actual skill this unlocks: knowing which triage questions are safe to hand to a model because they're bounded pattern-matching, and which ones still require a human doing the exploitation reasoning, every time, no exceptions. Codelivly's AI for Penetration Testing Book: Red Team Workflows works through exactly that line across recon, code review, and reporting, as ethics-first workflows rather than "let the model find your vulnerabilities for you." The free Recon & Enumeration Tools learning path on codelivly.com is worth working through first if the recon side itself, not just the triage on top of it, is still the part that needs reps.
Top comments (0)