DEV Community

Rahad Bhuiya
Rahad Bhuiya

Posted on

Why Fail2ban Isn't Enough: Building CNSL, a Self-Hosted SIEM for Linux & Kubernetes

#python #linux #cybersecurity #devops

If you manage Linux servers, your security stack probably looks something like this:

  • Fail2ban scanning /var/log/auth.log for failed SSH passwords.
  • A reverse proxy or web server (NGINX / Caddy) writing HTTP access logs.
  • A database (PostgreSQL / MySQL) logging authentication errors.
  • A firewall (iptables / UFW) enforcing static port rules.

On paper, this sounds like a layered defense. In practice, it has a glaring, dangerous architectural flaw: these tools operate in complete isolation.

Each tool watches a single log file and counts failures independently. And modern attackers know this.

An attacker does not hammer your SSH port with 100 rapid-fire passwords anymore—that would trip Fail2ban within seconds. Instead, they scan your web endpoints for exposed .env files, probe your database port for default passwords, and attempt just two SSH logins with leaked administrator credentials.

To your web server, it was just a minor 404 error. To your database, it was a routine connection failure. To Fail2ban, two failed SSH logins are well below the threshold of 5.

Every single tool saw a harmless blip. In reality, your server was undergoing a coordinated, multi-stage breach.

To eliminate this architectural blind spot, I built CNSL (Correlated Network Security Layer)—a self-hosted, lightweight SIEM for Linux and Kubernetes that correlates attacks spanning web, SSH, database, and cloud logs simultaneously, stopping breaches before they complete.

Here is the story of why I built it, how its cross-vector correlation engine works, and what it takes to design a modern defensive security layer from scratch.

1. The Core Philosophy: Cross-Vector Correlation

The foundational principle behind CNSL is simple: No security event should ever be evaluated in a vacuum.

When an IP address or actor touches your infrastructure, CNSL tracks their activity across every listening service simultaneously. Instead of waiting for a single service threshold to fire, CNSL computes a dynamic threat score across all vectors:

Web scan (directory traversal)  from 45.33.32.1  --+
SSH brute attempt               from 45.33.32.1  --+---> HIGH Alert & Auto-Block
DB authentication failure       from 45.33.32.1  --+
Enter fullscreen mode Exit fullscreen mode

When CNSL observes an actor combining reconnaissance with authentication probing across disparate services, the threat trajectory escalates immediately.

The attacker is automatically banned at the firewall/iptables level or quarantined across the entire cluster—before they ever guess a password or exploit an unpatched service.

2. Kill-Chain Progression & Predictive Blocking

Traditional intrusion prevention systems (IPS) are reactive: they wait until a threshold is crossed before taking action.

CNSL integrates Cyber Kill-Chain Tracking modeled after the MITRE ATT&CK framework:

  1. Reconnaissance: Port probing, vulnerability scanning, spidering.
  2. Weaponization & Delivery: Testing malicious payloads, path traversal.
  3. Exploitation & Initial Access: Credential stuffing, brute-forcing, privilege escalation attempts.
  4. Lateral Movement: Probing internal service meshes, database exfiltration.

Predictive Trajectory Blocking

Instead of waiting for an attacker to complete the exploitation phase, CNSL features opt-in Predictive Blocking.

By analyzing the velocity and breadth of an attack vector, the correlation engine calculates the statistical probability of an impending breach. If an IP rapidly transitions from passive web reconnaissance to active database auth probing, CNSL severs the connection proactively—reacting to the attack's trajectory before any single rule's threshold trips.

3. Beyond IP Addresses: Attacker Fingerprinting & Campaign Graphs

Sophisticated threat actors do not use a single IP address. They utilize residential proxy networks, VPNs, and Tor exit nodes, rotating their source IP address between requests to evade IP-based rate limiters.

If your security system only tracks IP addresses, you are playing an endless game of whack-a-mole.

To counter IP rotation, CNSL incorporates Attacker Fingerprinting and Graph Correlation:

  • Behavioral Fingerprinting: Tracks TLS cipher suites, TCP window signatures, request ordering, and user-agent entropy.
  • Graph-Based Campaign Correlation: Connects seemingly disparate IP addresses attacking your servers into a unified attack campaign graph.
  • When Actor X rotates from IP_A to IP_B, CNSL recognizes the behavioral fingerprint, associates the new IP with the existing threat entity, and immediately inherits the accumulated threat score.

4. Threat Intelligence Federation: Built-in STIX 2.1 and TAXII 2.1

Security should not be isolated to a single machine. When one node in your cluster discovers an aggressive threat actor, every other node should immediately know about it.

Rather than relying on proprietary, vendor-locked sync protocols, CNSL natively implements global open threat intelligence standards:

  • STIX 2.1 Export: Converts detected threat actors, campaigns, and indicators of compromise (IOCs) into standardized STIX JSON objects.
  • Built-in TAXII 2.1 Server: Runs a native TAXII endpoint directly inside CNSL. Other servers in your infrastructure (or peer organizations) can subscribe to your TAXII collections to ingest real-time threat intelligence.
  • Multi-Node Federation: Allows multi-region VPS deployments or edge servers to pool threat intelligence without routing all traffic through a centralized, vulnerable bottleneck.

5. From Single VPS to Kubernetes DaemonSet

CNSL was designed to be lightweight enough to run on a $5/month VPS, yet cloud-native enough to protect modern containerized environments.

Running on a Single Linux Server

CNSL can be installed directly with Python and pip:

pip install cnsl[full]
sudo python -m cnsl --dashboard --no-tcpdump
Enter fullscreen mode Exit fullscreen mode

By default, CNSL boots into safe dry-run mode—it correlates events, populates the live dashboard, and logs detected campaigns without modifying firewall rules until you explicitly pass --execute.

The built-in web dashboard (running on port 8765) gives operators real-time visibility into active threats, geographic origins, attack kill-chain phases, and blocked IPs.

Running on Kubernetes

In modern Kubernetes clusters, attackers frequently compromise one container and attempt lateral movement toward neighbor pods or database secrets.

CNSL provides an official Helm chart designed to run as a DaemonSet:

helm install cnsl ./helm/cnsl --namespace cnsl --create-namespace
Enter fullscreen mode Exit fullscreen mode

One CNSL agent runs per Kubernetes node, monitoring container socket events, host logs, and ingress traffic, feeding telemetry into a unified multi-node Hub view.

6. What I Learned Building a SIEM from Scratch

Building a real-time correlation engine in Python and systems-level Linux tooling provided some invaluable engineering insights:

  1. Log parsing is a performance minefield: When a high-traffic server generates 10,000 log lines per second, regex matching can easily peg your CPU at 100%. Implementing ring buffers, compiled zero-allocation parsers, and asynchronous pipeline queues is mandatory to keep memory and CPU footprints negligible.
  2. False positives destroy credibility: If a security tool accidentally bans an administrator or a legitimate customer, operators will turn it off immediately. Multi-vector correlation actually reduces false positives because a user making a typo on a web form is never banned—only entities combining suspicious behavior across multiple services are flagged.
  3. Open standards win every time: Implementing STIX/TAXII and integrations with Wazuh/OSSEC meant CNSL could plug seamlessly into existing SOC workflows without requiring operators to reinvent their operational playbooks.

Conclusion: Take Back Control of Your Server Security

Enterprise SIEMs (like Splunk or Datadog) cost tens of thousands of dollars per year and consume massive amounts of memory. On the other end of the spectrum, single-log watchers like Fail2ban are too blind to stop multi-vector attacks.

CNSL was built to bridge this gap: an open-source, self-hosted, correlated security engine that gives developers, self-hosters, and DevOps engineers enterprise-grade situational awareness with zero vendor lock-in.

The entire project is open-source under the MIT license:

Clone it, test it in dry-run mode on your servers, and let me know your thoughts!

How do you currently monitor and correlate security events across your Linux servers? Have you ever caught an attacker probing across multiple ports? Let's discuss in the comments below!

Top comments (0)