I have been building CNDS, a home-network intrusion detector, for a while. It captures packets with Scapy, scores each flow with five engines (a supervised FT-Transformer, an Isolation Forest, an LSTM autoencoder, a rules engine and a baseline that trains on my own traffic), and maps what it finds to MITRE ATT&CK. This month I put it on a Raspberry Pi 5 and let it run. This is what I learned, including the parts that are not flattering.
Week one: the alerts were all wrong
On day one the detector produced about 45 alerts an hour. On 30 and 31 October the database had 195 and 362 alerts, and I ended up labelling almost all of them as false positives (178 and 336).
The first cause was embarrassing and easy to find. The Isolation Forest and the LSTM that ship with the project are demo models trained on synthetic N(0, 1) data. On real traffic the LSTM scored 1.0 on 300 of 300 alerts I looked at. It was not detecting anything; it was saturating. I set its weight to zero.
The second cause was the rules. All 54 critical alerts came from payload regexes (XSS, SQL injection, command injection) matching TLS and binary traffic: CloudFront downloads and an S3 service on my own network. The supervised model had called those flows benign, but a critical rule overrides the blend and forces a score of 1.0. The fix was to skip payload regexes on encrypted or high-entropy payloads.
The third cause was a monitoring probe. Two flows (my Pi talking to a proxy, and a blackbox exporter probing its neighbours) made up 89 % of the remaining alerts. I replaced IP-wide suppression with narrow rules: source, destination and port.
There was also a logging trap. The detector wrote the [ALERT] line before suppression rules ran, so 243 [ALERT] lines in ten hours meant about three real alerts. I now log [SUPPRESSED] separately.
After all that, daily alerts fell to 63, 48 and 37, with none rated high or critical. That is a quieter system. It is not proof the system works.
The test: can it detect a SYN flood?
False positives are only half of the question. To measure the other half I built a small red-team tool, cnds-red-team. It has an allowlist, a hard-coded list of hosts it can never touch, a duration and packet budget for every run, and a kill switch. I pointed it at the Pi from another Pi.
The first scenarios, a port scan and a burst of DNS queries, detected nothing. The DNS result was expected: CNDS has no DNS rule, and my "expected signal" for that scenario had been a guess. The port scan result sent me to read the code.
CNDS has a syn_scan rule. It fires when a flow has more than 20 SYN flags and fewer than 5 forward packets. Read that again. Every SYN is itself a forward packet, so a flow with more than 20 SYNs has more than 20 forward packets. The two conditions can never be true together. Using git log -L, I confirmed the condition had been like that since the very first implementation, and a later refactor that renamed magic numbers into constants copied it forward unchanged.
The rule's own comment says "high SYN count with low data". The second clause was meant to look at payload, not packet count. The fix was one line (use the count of packets that carry data), plus a test.
The second bug was in my own configuration
With the rule fixed, the SYN flood still produced no alert. The reason: on my Pi the rules engine weighs 5 % of the ensemble. A perfect rule hit adds 0.05 to a score that has to cross 0.55. I made syn_scan a critical rule, which bypasses the blend, as unambiguous signatures like SQL injection already did.
The part I have not solved
I then confirmed that every packet leaves the sender's network card and reaches the Pi (a sniffer on the target matched the timestamps), and that feeding the same packets straight into the deployed detector code produces a score of 1.0 and syn_scan. A normal request containing a literal UNION SELECT generates an alert through the live detector. But the raw-socket SYN flood does not, and I do not yet know why. I ruled out the logic, the feature indices, the port state and a stuck process. What is left is somewhere between the capture queue and the flow dispatcher, and I have not instrumented it yet.
I am telling you this because the result of this project is not "it works". It is a list of things I now know do not work, found by testing instead of reading my own code with kind eyes.
What I'd do differently
Test with real traffic from day one. Two of the bugs above had unit tests that passed, and neither test could have caught them, because the tests built the same flows the code expected. If a model's score is always 1.0, it isn't a detector, whatever its F1 says on the test split. And log what you drop: my alert counter counted before suppression, so for a week it told me the system was noisier than it was.
I also wrote the limits into the README before cutting the release. CNDS 2.0.0-rc.1 says that recall hasn't been measured. A pre-release that lists what it can't do yet is easier to trust than one that doesn't.
The code is AGPL-3.0, release v2.0.0-rc.1: https://github.com/RobertoDeLaCamara/Cognitive-Intrusion-Detection-System
If you have an idea why a raw-socket SYN-only flow reaches the NIC but never reaches the flow dispatcher in a Scapy-based sniffer, I'm all ears.
Top comments (1)
Rules that can never fire are dangerous because they create the appearance of coverage. Adding a synthetic event that must trigger each detection rule, plus a negative control that must not, turns the rule set from static configuration into something continuously verified.