Security automation has a credibility problem: finding a suspicious pattern is easy; proving that it creates real impact is much harder.
Traditional scanners are useful, but they often leave security teams with a queue of alerts that still need manual reproduction. LLM-driven agents can explore more context—authentication flows, business logic, IDOR/BOLA, and chained behavior—but an AI-generated explanation is not the same thing as evidence.
That distinction shaped how we built Xalgorix, an Apache-2.0 open-source AI pentester for authorized security testing.
Detection is a hypothesis, not a finding
A scanner might observe an unusual response, infer that an endpoint is vulnerable, and produce a convincing description. That is useful as a hypothesis. It should not automatically become a confirmed finding.
A report becomes substantially more useful when it answers questions such as:
- Can the behavior be reproduced?
- What exact request, payload, or sequence triggers it?
- Does it cross a security boundary?
- What evidence supports the claimed impact?
- What should an engineer change to remediate it?
This is especially important for AI agents. Models are good at connecting weak signals, but they can also overgeneralize from incomplete context. The safest design is to treat the first agent's conclusion as a candidate that must survive a separate verification step.
Detect, challenge, verify
Xalgorix uses an autonomous testing agent to work through the attack surface, then passes candidate findings to a separate verifier. The verifier independently attempts to reproduce the issue before it is reported as confirmed.
The workflow is deliberately asymmetric:
- The testing agent explores broadly and develops hypotheses.
- The verifier narrows the scope to a specific claim and tries to reproduce it.
- Confirmed findings include technical evidence, CVSS context, proof-of-concept details, and remediation guidance.
- Inconclusive results stay clearly labeled instead of being presented as proven vulnerabilities.
This does not make false positives impossible. Security testing is too contextual for that promise. It does, however, make verification a first-class part of the system rather than a manual task pushed onto the person reading the report.
A separate verifier also reduces the risk of an agent simply agreeing with its own earlier reasoning. The goal is not to generate a more persuasive paragraph; it is to test whether the claim survives another attempt.
The self-hosted findings view keeps severity, endpoint, target, and CVSS context visible in one place.
Coverage still needs structure
Autonomy without structure can turn into aimless browsing. Xalgorix organizes testing into a 22-phase methodology covering reconnaissance, attack-surface mapping, authentication and sessions, authorization, injection, SSRF, IDOR/BOLA, APIs, file handling, business logic, cloud exposure, WebSockets, exploit verification, novel-vulnerability discovery, and final reporting.
The phases are selectable. A tester can run the complete workflow or focus on the parts relevant to a particular engagement. That matters because scope, time, and allowed techniques vary between a local development environment, an internal assessment, and a bug-bounty target.
The agent can combine terminal tooling with browser-assisted DAST, which is useful for JavaScript-rendered applications, authenticated workflows, and stateful interactions that are difficult to evaluate from HTTP responses alone.
The scan detail view exposes phase progress, risk, findings, events, and configuration while a test is running.
Open source is part of the security model
For a security tool, “trust us” is not a satisfying architecture.
The Xalgorix engine is self-hosted, written in Go and TypeScript, and released under Apache 2.0. The local dashboard binds to loopback by default, and scan records remain under the operator's control. Users choose their own model provider—OpenAI, Anthropic, Gemini, DeepSeek, Groq, Ollama, MiniMax, or another OpenAI-compatible gateway—and can inspect or modify the orchestration themselves.
Open source makes several practical things possible:
- Teams can review how scope, tool execution, and verification are handled.
- Sensitive engagements can run in infrastructure controlled by the tester.
- Researchers can add tools, methodology phases, or provider integrations.
- Findings can be audited against the code that generated them.
- Local models can be used when an external model provider is not appropriate.
The repository includes a local web dashboard, CLI, Docker packaging, live WebSocket telemetry, finding management, CVSS scoring, proof-of-concept evidence, remediation guidance, and branded PDF reports.
The self-hosted overview runs locally and brings active scans, findings, system health, and live activity together.
Source code can be the starting point
A deployed target is not always available. Xalgorix therefore supports two code-oriented modes.
In source-review mode, it audits a Git repository, local directory, or archive and traces data from entry points to dangerous sinks. Results are labeled as source-verified, not runtime-exploited.
In provision mode, it inspects the repository, builds and starts the application on a loopback port, then runs white-box-guided DAST against that instance. Findings can then be reproduced against the running application. If the application cannot be provisioned, the system can fall back to review mode while preserving the distinction in the report.
That labeling is important. “Reachable in code” and “exploited at runtime” are different levels of evidence and should never be blurred.
Operational safety matters
An autonomous pentester has access to powerful tools, so guardrails are not optional.
Use it only on systems you own or have explicit permission to test. Review the engagement scope before launching, set rate limits that match the rules, isolate Docker-based scans appropriately, and never expose the local dashboard publicly without authentication. A security tool should not create a new security problem around itself.
What we are building toward
The larger idea behind Xalgorix is simple: AI security tooling should be judged by the quality of its evidence, not the confidence of its prose.
A useful agent should be able to explore creatively, admit when a claim is inconclusive, and produce a report that an engineer can reproduce and act on. Verification should not be an afterthought.
You can inspect, self-host, and contribute to Xalgorix on GitHub. The public repository is the primary product. There is also an optional hosted workflow for teams that prefer managed scans, schedules, workspaces, and CI integrations.
The optional hosted dashboard adds managed operations around the same security workflow.
Top comments (0)