I Built an Agentic Linux Security Tool. It Took Way More Iterations Than I Expected.

Liam Romanis — Sun, 17 May 2026 23:53:51 +0000

This started as a simple experiment: can you point an AI at a Linux system, have it collect forensic data, and get something more useful than a wall of text back?

The answer, it turns out, is yes — but not in the way I originally thought, and not without a lot of iteration to get there.

How It Started

The initial idea was straightforward. Run a bunch of forensic commands — process lists, open sockets, SUID binaries, kernel modules, log anomalies, the usual — pipe the output to Claude, and get a triage report back. Simple agentic loop. Collect, analyse, report.

And that bit worked fine. Claude is actually pretty good at reading ps auxf output and spotting things that look wrong. Better than I expected, honestly.

The problem was what happened next. You'd get a list of findings and then... nothing. The same problem every security tool has. Here are some things that look suspicious. Good luck. The AI had done the easy bit and left you to figure out the hard bit on your own.

That's not really agentic. That's just automation with a language model bolted on.

The Interesting Problem

What I actually wanted was an AI that could investigate alongside you. Not just flag things, but help you work through whether a finding is real, what to do about it, and whether the remediation you're considering is going to cause more problems than it solves.

The challenge is that investigation requires running commands on the live system. And if you're going to run commands on a live system based on AI suggestions, you absolutely cannot have those commands run automatically. The AI will get things wrong. The AI will suggest things that sound reasonable but aren't appropriate for your specific setup. The AI will, if you let it, suggest hardening measures that stop your system from booting.

That last one happened. Not in a catastrophic way, but enough to make the point very clearly: you need a human in the loop, and that human needs to actually understand what they're approving.

The Man-in-the-Loop Pattern

What emerged after a lot of iteration is a batch investigation loop that goes like this:

Claude analyses a finding and proposes a set of verification or remediation commands — typically three to six per round. These are displayed to you with a type badge (VERIFY, REMEDIATE, or INSTALL), a plain-English description of what the command does, the rationale for why it's useful, and — critically for anything that could affect system stability — a rollback command so you know how to undo it.

You review all of them. You can deselect any you don't want. You can ask Claude questions about any command before approving it — "what does this actually do", "is there a safer alternative", "why is this necessary" — and get a direct answer in context.

Then you click run. The approved commands execute sequentially, the output comes back, and Claude analyses everything together in one consolidated response rather than reacting to each command individually. If it needs more information, it proposes another batch. If it has enough to make a determination, it gives you a verdict and action buttons: confirm as false positive, keep as active finding, mark resolved.

It took a lot of iterations to get this feeling natural. The early versions had Claude proposing one command at a time, which created an exhausting back-and-forth. Batch proposals with a single analysis pass work much better. The thread also has a tendency to grow unwieldy, so completed investigation rounds collapse into summaries.

The False Positive Problem

Something that became obvious quickly: AI-generated findings are going to overlap with things you already know about and have decided to accept. Your custom SSH port. Your pentest tooling. The forensic agent's own token file sitting in /tmp.

The tool has a false positive management system that goes beyond a simple whitelist. It uses fuzzy matching — a combination of token-based Jaccard similarity and longest-common-subsequence ratio — so that when Claude words a finding slightly differently on the next scan, it still gets suppressed. There's also a session-dismiss for things you want to acknowledge without permanently suppressing, and an FP audit workflow where Claude reviews your saved false positives and flags any that probably shouldn't be permanently suppressed because they could indicate real malicious activity in a different context.

That last one is more useful than it sounds. "Orphaned PTY sessions" is a reasonable false positive if you left some terminals open. It's not a reasonable false positive to permanently ignore if it could also indicate someone else's session on your system.

Is It Any Good?

Honestly, it's useful. More useful than I expected. For investigating real findings on a system you're responsible for, the investigation loop pattern genuinely helps — it keeps you from either ignoring things you should look at or taking action you don't understand.

But let's be clear about what it isn't.

It is not production ready. It is not comprehensively tested. The AI analysis is non-deterministic and occasionally wrong. The agent runs as root with minimal authentication over localhost — fine for personal use, not something you'd put in front of customers.

Is it secure? No. It's a forensic tool that runs as root and executes commands you approve. Security is mostly your problem. Use it on systems you own, in environments you control, for purposes you understand.

Use it at your own risk. If you blindly approve every command Claude suggests without reading them, you will eventually do something you regret. The tool tries to help — boot-risk warnings, required rollback instructions for kernel changes, explicit confirmation checkboxes before anything that could affect system stability — but it cannot protect you from yourself.

What I Learned About Agentic Development

The most interesting thing about building this wasn't the security tooling. It was what the development process revealed about building agentic systems in general.

The gap between "AI that does a task" and "AI that works alongside a human on a task" is much larger than it looks. The first is just automation. The second requires thinking carefully about where the human needs to be in the loop, what information they need to make good decisions, and how to present AI suggestions in a way that encourages understanding rather than blind acceptance.

Getting that right took a lot more iteration than I expected. The first version had the AI running ahead too fast. Later versions were too cautious and required too many clicks for simple cases. The batch proposal pattern that ended up working is something I arrived at through trial and error, not design.

That feels like the honest state of agentic development right now. The patterns are still being worked out.

Where It Goes From Here

It's open source under MIT: github.com/liamromanis101/SysForensics

If you're interested in the agentic investigation pattern, want to add checks for other distributions, want to explore what commercial-grade security tooling built on this approach would look like, or just want to kick the tyres — get in touch. I'd be genuinely happy to make this a community project if there's interest.

There's a lot of room to go further with this. Better context-awareness between findings, CVE cross-referencing, fleet management, proper reporting for auditors. The foundation is there. Whether it goes anywhere depends on whether other people find the approach interesting.

If you do try it: read the commands before you approve them. That's the whole point.

DEV Community: Liam Romanis