DEV Community: Hammad Anjum

Your Firewall Already Saw the Attack: Bringing AI Detection to Network Logs

Hammad Anjum — Sun, 18 Oct 2026 19:28:38 +0000

Endpoint agents are good at seeing what happens on a machine. But a lot of an attack happens between machines: the scan that finds an exposed service, the exploit that lands on it, the malware download, the call home, the data leaving the building.

And plenty of what sits on a network will never run an agent: printers, cameras, appliances, unmanaged laptops, and the firewalls themselves.

The firewall, though, logs all of it. Every session, every denied connection, every IPS hit, every admin login. The problem has never been missing data. The problem is that the data is huge, vendor-specific, and noisy, so it mostly gets read after an incident, not during one.

At Logster we've spent the last few weeks building AI network detection: extending the AI-driven detection that today explains attacks on Windows and Linux endpoints to firewall and flow logs. It's not generally available yet, but it runs end to end, and we want to share how it works and how it performs.

What AI network detection does

In one sentence: Logster reads the logs your firewalls and routers already produce, finds the activity that looks like an attack, and tells you, in plain language, what happened and why it thinks so.

Concretely:

It ingests firewall syslog and NetFlow/IPFIX, detects the vendor automatically, and converts every format into one common network event. FortiGate and Cisco ASA are tested on real data; Palo Alto and Juniper parsers are in progress.
It looks for behaviour, not just signatures: scanning, exploitation, malware downloads, beaconing, unusual data volume, tunnelling apps, brute force, and tampering with the firewall itself.
It gives every verdict an explanation: attack or benign, a confidence, a numbered story of the attack phases, and MITRE ATT&CK techniques.
It shows the evidence as a graph: who talked to whom, on which ports and applications, how many bytes moved in each direction, how sessions ended (completed, reset, no reply, denied), and what the firewall's own IPS flagged.
It links network alerts to endpoint alerts when both describe the same machine. More on that below; it's our favourite part.

How it works

Logster turns the logs your firewalls and routers already write into something an AI can reason about, and then checks its own answer before anybody gets paged. Vendor formats are collapsed into one common network event, activity is grouped by host, and what the AI finally sees is a compact picture of that host's behaviour rather than a wall of raw log lines.

Reason with the AI

Each host's recent activity becomes a compact graph: the host in question, its peers, flow edges with states, applications and bytes, and IPS-alert edges. That graph goes to the AI with a network-specific prompt. Here is a real verdict for a web server under attack:

host:192.168.250.70: 1. Recon - External host 40.80.148.42 performed extensive vulnerability scanning (Acunetix.Web.Vulnerability.Scanner). 2. Exploitation - The scanner successfully delivered multiple high-severity exploits, including Bash RCE (Bash.Function.Definitions.Remote.Code.Execution), PHP CGI Argument Injection, and HTTP XXE, all of which were allowed by the firewall.

And one for an infected workstation a few minutes after a malware download:

host=192.168.250.100: 1. Recon - The host performed massive external scanning via UDP port 6892 and port 137 across thousands of external IPs. 2. Command and Control - The port 6892 traffic is identified as Cerber.Botnet, indicating active botnet communication. 3. Novelty - The host established connections to 6,421 new external peers.

This is roughly what the AI was looking at for that workstation: a completed download the firewall's IPS flagged as malware and let through, then thousands of unanswered connections labelled Cerber.Botnet.

Guard

The AI's "attack" is not the last word. An alert also needs corroborating evidence, from more than one kind of observation. One odd connection is never enough to wake someone up.

One incident, two views

This is the part we're most excited about.

When Logster's endpoint agent flags a machine and the firewall flags that machine's IP address, those are two views of the same incident. In most SOCs, connecting them means someone looking up which host had that IP at that time, by hand.

Logster does it automatically. It learns which machine owns which address from the endpoint's own network activity (an address that appears in most of a host's connections is that host's address) and from DHCP lease logs. When a network alert arrives for an IP, it's linked to the open endpoint alert for the machine behind it, in both directions.

In our test replay of a real-world style infection, the endpoint alert ("this desktop's processes look malicious") and the network alert ("this IP downloaded malware and is now spraying the internet") arrived 24 seconds apart, already linked as one incident. The analyst gets the process tree and the network picture side by side, without matching anything by hand.

How it performs

We built a replay harness that pushes labelled public datasets through the full AI network detection pipeline, the same services that run in production, and scores detection, time to first alert and false alarms. Two of the datasets were held out entirely: we never tuned on them.

Scenario	Dataset	Result
Web server: vulnerability scan, SQL injection, Shellshock, reverse shell	Splunk BOTS v1 (FortiGate)	detected 73 s after the attack began, at real-time speed
Workstation: malware download, then mass scanning	Splunk BOTS v1	detected 12 s after the download
A normal business day	Splunk BOTS v1	0 false alarms
Spam botnet with command-and-control (held out)	CTU-13 (NetFlow)	detected; 0 false alarms across 81 normal hosts over an hour
Firewall tampering: internet admin login, logging disabled (held out)	Splunk attack_data (Cisco ASA)	both episodes detected, in 42 s and 4 s

5 of 5 attacks detected, 0 false alarms.

To be clear about what this is: public datasets, replayed through the real pipeline — which is the kind of evidence we think every detection vendor should be asked for. Validation on live networks is the next step, and some detections (brute force, lateral movement, beaconing, exfiltration) are built and unit-tested but not yet proven on a real-world dataset, because none of the ones we've run contain those attacks.

What's next

The AI network detection pipeline runs end to end today. Next:

An Alerts view in the dashboard that shows linked endpoint and network incidents together
Whitelisting for network rules, so approved scanners, backup jobs and management hosts stay quiet
Collectors and deployment packaging, plus validation on live lab traffic
More coverage: more vendors, and real-world validation for brute force, lateral movement, beaconing and exfiltration
Sharper output: technique IDs checked against the ATT&CK catalogue, and richer graph edges (packet counts, inbound data-theft detection)

Closing

Logster started by explaining what happens on your machines. Soon it will explain what happens between them, and tell you when both views are describing the same attack.

If you run a SOC, manage firewalls, or just have opinions about AI network detection, we'd love to hear from you. Tell us which vendors and which attacks you'd want covered first.

Datasets: Splunk Boss of the SOC v1 (CC0), CTU-13 by the Stratosphere Lab at the Czech Technical University (CC BY), and Splunk attack_data (Apache 2.0). Thanks to their authors for making realistic security data public.

Logster v1.3.0: Whitelisting You Can Trust, Evidence You Can Click

Hammad Anjum — Mon, 28 Sep 2026 07:51:22 +0000

Every detection platform asks for your trust twice: once when it raises an alert, and once when you tell it to stay quiet. Logster v1.3.0 is built around both halves of that bargain. It rebuilds whitelisting so that approvals stay visible, measurable and impossible to hide behind, and it connects every explanation Logster gives you to the evidence that produced it.

Alongside that, this release adds storage that looks after itself and a delivery model for air-gapped clusters. Here is what changed.

Whitelisting you can trust

A whitelist is a statement of trust, and, as we have written before, whitelisting a graph is harder than whitelisting a filename. Logster v1.3.0 makes that statement easier to write precisely, easier to check, and much harder to abuse.

Attackers can't hide under an approved parent. This is the most important change in the release. An approved process that starts something unapproved is no longer removed. The whole branch is kept and scored, so running malware from inside a trusted backup job doesn't make it disappear.
Approve by description, not only by example. New attribute rules let you approve "this exact backup command" or "anything writing to this log path" by typing it, with no source graph needed. Rules can apply to every endpoint or just one.
Watch before you enforce. New rules start in observe-only mode. They report what they would have excluded ("would have excluded 214 nodes in 7 days") without changing any verdicts. Switch a rule to enforcing once the numbers match what you intended.
A breadth check that says no. Before a rule is saved, Logster tests it against recent activity. A rule that would silence a large share of everything is refused unless someone deliberately overrides it.
Nothing disappears silently. Whitelisted activity is excluded from scoring but still kept and visibly marked on the graph, and fully approved windows are still recorded. Every entry shows what it covers and how much it has excluded over the last seven days, so entries nobody needs any more are easy to find.
Backup and restore. Whitelist entries are the only thing in Logster your analysts create by hand. They can now be exported to a file and restored from the console or from a script.
Unknown parents handled safely. When a process's parent started before Logster began observing, the parent appears as an unknown placeholder. Placeholders can now be included in approvals, but Logster refuses any approval where an unknown parent is the only actor, because "unknown" would otherwise match any long-running program, malware included.

Why this matters beyond convenience: every suppression that nobody can explain later is tuning debt, a permanent blind spot. Approvals should stay visible, measurable and reversible, and Logster v1.3.0 is built that way.

Evidence you can click

When Logster flags an attack, it explains its reasoning. In Logster v1.3.0 that explanation is connected to the evidence.

Open an investigation and the events behind the verdict move to the top of the list. The graph highlights the handful of nodes that form the attack chain, out of what can be dozens of routine ones. Click a step in the explanation, for example "PowerShell wrote to the startup folder", and the graph spotlights exactly the nodes that step refers to.

This runs only when an analyst opens an investigation, so it never slows detection. It also annotates the original verdict rather than re-deciding it, so what you click through is the reasoning that produced the alert. Linux events now carry the same traceability back to their source, so this works across Windows and Linux.

It answers the question we think every buyer should ask a detection vendor: show me a finding exactly as an analyst receives it. (More on that in our guide to evaluating threat detection platforms.)

Storage that looks after itself

Security data grows quietly until the disk fills up, usually at the worst possible moment. Logster v1.3.0 adds storage-aware retention.

Sensible defaults out of the box: events for 30 days, detections for 90 days, activity graphs for 14 days.
If disk usage crosses its target anyway, the oldest data is removed first, before storage becomes an outage.
The System Health dashboard now shows storage use per data type with a forecast against the target, plus message-queue lag, so you can see a problem coming.

Built for air-gapped environments

For organisations that can't send security data anywhere, the Logster AI model can now be delivered separately from its runtime image and loaded from local storage inside an air-gapped OpenShift cluster. The runtime image is much smaller, the two downloads can be verified independently, and the model runs fully offline with no internet access at runtime.

Under the bonnet

A few smaller things that shipped with Logster v1.3.0:

The Splunk receiver has been rebuilt in Rust, and correctly handles large, multi-line PowerShell events.
Shipped container images passed our release security scan with zero known fixable CVEs, including a move to a hardened, rebuilt monitoring image.
Linux events collected through eBPF and auditd now keep a link back to the original event, so every node in a Linux graph can be traced to its source.
Many stability improvements across ingestion, inference and the dashboard.

Upgrading to Logster v1.3.0

Logster v1.3.0 is available now. Upgrade instructions from v1.2.1, including the new retention settings, are in the Logster documentation.

If you'd like to see Logster v1.3.0 on your own data, get in touch.

Originally published at logster.ai.

12 Questions That Actually Separate Threat Detection Vendors

Hammad Anjum — Thu, 10 Sep 2026 10:45:05 +0000

Most threat detection evaluations are decided by a demo, and demos are designed to succeed. The data is curated, the attack is one the product has seen, and the environment has none of the strange legacy software that makes your estate yours.

The second problem is that every vendor's marketing now says the same words. AI-powered. Behavioural. Zero-day. Reduces alert fatigue. The words no longer discriminate between products, which means the burden falls entirely on the questions you ask.

Below are twelve that do discriminate — in the sense that different products give genuinely different answers, and the answers are hard to fake. I've included what a good answer sounds like and what should worry you.

Full disclosure, since it's relevant: I work on a detection platform. I've tried to write these so they'd be useful against us too, and I've flagged the ones where our own answer is uncomfortable.

Detection quality

1. What is the false positive rate at my event volume — not on your benchmark?

This is the single most important question and the most commonly dodged.

Detection rates are quoted constantly; false positive rates rarely are. But at security's base rates the FP rate is what determines whether the product is usable. A system that catches 99% of attacks and produces 500 false positives a day is not a detection system, it's a queue that your team will learn to ignore.

Good answer: a number, with the event volume it was measured at, and an acknowledgement that it varies by environment.

Red flag: a detection rate offered in place of a false positive rate. Or a percentage with no denominator — "99.9% accurate" is meaningless when the positive class is one in ten million.

2. What was the model trained or tuned on, and how similar is that to my estate?

Real labelled attack data is scarce. Everyone improvises — lab-generated attacks, public datasets, red team exercises, customer telemetry. Each has different failure modes, and a model trained on lab data has learned the artefacts of the lab's generation process alongside the actual attack behaviour.

Good answer: specifics about the data, honest about the gap between it and production.

Red flag: "proprietary" as a complete answer, or "millions of samples" without saying what kind. Ask specifically whether the training data came from lab-executed MITRE ATT&CK techniques, public datasets, or production customer telemetry — the three have very different failure modes.

3. What is your unit of detection — an event, or a relationship?

This distinction determines more about a product's behaviour than any feature list.

If each event is scored independently, an attack made of individually-benign steps produces either nothing or a scatter of low-severity alerts that get closed one by one — which is a large part of why alert fatigue costs what it does. If the product reasons over relationships between events, that same attack can surface as a single correlated incident.

Good answer: a clear description of what gets correlated and how.

Red flag: confusion between correlation and aggregation. Grouping alerts by host and time window is not the same as understanding that one process created another.

4. What happens with activity you've never seen before?

Everyone claims zero-day detection. Push on the mechanism.

Good answer: a specific explanation — anomaly relative to a learned baseline, structural patterns that hold across technique variants, behavioural constraints that any implementation of a technique must satisfy — plus its limitations.

Red flag: "our AI detects unknown threats" with no mechanism. Also worth noting: if the answer is purely statistical anomaly detection, ask how they handle the fact that living-off-the-land attacks are specifically designed to sit inside the normal distribution.

What it's actually like to operate

5. Show me a finding exactly as an analyst receives it.

Ask for a screenshot of the real thing, not a slide. This one question tells you more about daily life with the product than an hour of feature discussion.

If a finding is a severity score and a rule name, your analysts will reconstruct the reasoning themselves from raw logs — which is the work you were trying to avoid. If it carries the evidence chain, they can verify or dismiss it in minutes.

Good answer: they show you one immediately, and it contains enough for you to form your own judgement about the activity.

Red flag: reluctance, or a mockup.

6. Can a mid-level analyst verify a finding without escalating?

Related but distinct. A finding can be detailed and still be unverifiable — buried in jargon, or requiring knowledge of the model's internals.

The practical test: could someone eighteen months into the job look at this and decide whether it's real? If every finding needs your best person, the product hasn't reduced your workload, it's concentrated it.

7. What's the median time from finding to decision?

Ask existing customers, not the vendor. This is the number that converts to money, and it's the one you should be measuring in a POC.

8. What happens when my environment changes?

New software, a migration, a reorganisation, a merger. Baseline-dependent systems degrade as the estate drifts.

Good answer: a clear account of retraining or re-baselining — cadence, who performs it, whether it's included, how long it takes.

Red flag: "it learns continuously" with no operational detail. Ask what happens on day one of a new deployment, and how long until it's useful.

Coverage

9. Which telemetry sources, on which platforms, and what does each give you?

Specificity matters here. "Full endpoint visibility" means nothing. Sysmon, auditd, eBPF, ETW, and vendor-proprietary collectors see genuinely different things — we wrote a detailed comparison of what each endpoint telemetry source actually captures if you want the specifics and have different overheads and kernel requirements.

Ask particularly about Linux, which is where coverage is usually thinnest — and about which kernel versions are supported, since eBPF-based collection has real floor requirements and older estates fall back to something else.

10. What are you not good at?

The most informative question in the list, and the one whose value comes entirely from how it's answered rather than what's said.

Every product has a shape. A vendor who can describe theirs has thought about it; one who claims comprehensive coverage of everything is either inexperienced or managing you.

Good answer: a real limitation, stated without being walked back in the next sentence.

Red flag: "that's a great question" followed by a strength.

Adversarial

11. Which of your detections still work if the attacker knows exactly how the system works?

Assume your vendor's documentation is public and read. Which detections survive?

Detections keyed on things cheap to change — filenames, hashes, specific strings, timing — don't survive. Detections keyed on what an attack must structurally do to succeed are much more durable, because changing them means changing the attack.

Good answer: an explicit distinction between the two, and an honest statement that some detections are in the first category.

Red flag: a claim that everything is durable. Nothing is entirely durable, and saying so signals they haven't modelled an adaptive adversary.

12. What happens when an attacker kills the agent?

They will try. Ask what the product does when telemetry stops — whether agent silence is itself an alert, how quickly it fires, and what tamper protections exist.

An agent going quiet on one host at 3am is a finding. Surprisingly many products treat it as a health metric on a dashboard nobody watches.

Every vendor answers these the same way: "Do you use AI?" · "Do you detect zero-days?" · "Do you reduce false positives?" · "Do you map to MITRE ATT&CK?"

These separate them: "What's your false positive rate at my event volume?" · "Show me a finding exactly as an analyst receives it."

How to run the evaluation itself

The questions matter less than the test. Four things worth insisting on:

Use your own data. A POC on vendor-supplied data tells you the product works on data the vendor chose. Run it against your estate, including the odd legacy application nobody wants to talk about.

Measure false positives over at least two weeks. Detection is easy to demonstrate and false positives take time to accumulate. A three-day POC systematically flatters every product in this category.

Run a real technique, not a sample file. Executing an actual ATT&CK technique — with Atomic Red Team, Caldera, or by hand — tells you far more than dropping EICAR. Have the vendor not know when you're doing it.

Have the analysts who'll use it do the triage. Not the security architect who ran the procurement. The people whose queue this becomes will tell you within a week whether findings are actionable.

The uncomfortable ones

In the interest of the disclosure at the top — the questions above that are hardest for us to answer:

Question 2, on training data. Like everyone in this space, we generate attacks in a lab — MITRE techniques executed on baremetal Windows and Linux VMs — and lab-generated attacks differ from real ones in ways that are genuinely hard to characterise. Anyone claiming otherwise is not describing a situation that exists.

Question 8, on environmental change, is a real cost for any behavioural system and it's worth pressing every vendor on, including us.

Where we do well is 3, 5, and 11 — the unit of detection is a behavioural graph rather than an individual event, findings carry the evidence chain that produced them, and structural features are considerably more expensive for an attacker to evade than name-based ones. Those are the questions we'd want asked. Which is exactly why you should ask all twelve, of everyone, including us.

Common questions

What should you ask a threat detection vendor before buying?

Start with the false positive rate at your own event volume, not their benchmark. Then ask to see a finding exactly as an analyst receives it, and ask what the product is not good at. Those three separate serious vendors from marketing faster than any feature comparison, because none of them can be answered with a slide.

How long should a threat detection POC run?

At least two weeks, on your own data. Detection is easy to demonstrate on day one; false positives take time to accumulate. Short proofs of concept systematically flatter every product in this category, which is why vendors like them short.

What's the difference between detection rate and false positive rate?

Detection rate is the proportion of attacks caught. False positive rate is how much benign activity is wrongly flagged. Vendors quote the first constantly and the second rarely — but at security's base rates, where genuine intrusions are extremely rare against millions of daily events, the false positive rate is what determines whether a SOC can actually use the product.

How do EDR, XDR, SIEM and behavioural detection differ?

EDR collects and analyses endpoint telemetry. XDR extends that across more signal sources. SIEM aggregates logs centrally and runs rules over them. Behavioural detection describes the method rather than the category — reasoning over what processes do and how they relate, rather than matching known-bad artefacts. The categories overlap heavily, which is why question 3 above — what is your unit of detection — discriminates better than asking which acronym a product belongs to.

Does behavioural detection replace signature-based detection?

No, and be sceptical of anyone saying it does. Signatures are fast, cheap, and produce near-zero false positives on known threats. They just have no coverage of anything novel. Production estates run both.