Building a Self-Learning DDoS Guard

Real-Time Anomaly Detection with
Python
By Victor • HNG DevSecOps Project Case Study

In the modern web landscape, static rate limiting is often a blunt instrument. While it
can stop basic brute-force attacks, it struggles with sophisticated, low-and-slow DDoS
attacks or sudden legitimate traffic spikes. For my latest HNG DevSecOps project, I
built a dynamic Anomaly Detection & DDoS Engine that learns from your traffic
patterns and defends your AWS infrastructure in real-time.

The Problem: Why Static Limits Fail
Most developers set a hard limit: "Allow 100 requests per minute." But what happens at
2:00 AM when your server is usually empty? A sudden burst of 90 requests per
minute from a single IP might be an attack, yet it passes under the radar. Conversely,
during a Black Friday sale, 150 requests might be perfectly normal. I needed a system
that understood context.

The Solution: Statistical Learning
The heart of this engine is a Python-based daemon that "learns" what normal traffic
looks like for every hour of the day. It uses two key mathematical concepts:

1. The Rolling Baseline
   Instead of hardcoded numbers, the engine maintains a 30-minute rolling window
   of traffic metrics. It calculates the mean and standard deviation for every hour
   slot. This allows the system to distinguish between a busy Monday afternoon and
   a quiet Sunday night.

2. The Z-Score
   To identify an anomaly, we calculate the Z-Score of incoming traffic. The formula
   is:

z = (x - μ) / σ

Where x is the current traffic rate, μ is the learned mean, and σ is the standard
deviation. If the z exceeds 3.0, the system flags the IP as an anomaly.

The Architecture
The project is deployed on AWS EC2 using a Dockerized stack:
Nginx: Acts as the frontline, logging every request in a structured JSON format.
Nextcloud: Our sample application being protected.
Python Detector: The "Brain." It tails the Nginx logs, performs statistical
analysis, and makes decisions.

Active Defense with Iptables
Detection is useless without action. When an IP is flagged, the engine doesn't just
send an alert; it executes a system-level command using iptables to DROP all
traffic from that IP. To ensure we don't block legitimate users forever, I implemented
an Unbanner module. It follows an exponential backoff schedule: 10 minutes, then
30 minutes, then 2 hours, before finally issuing a permanent ban for repeat
offenders.

Real-Time Visibility
I integrated a Slack notification system to keep the DevOps team informed. Whether
it’s a specific IP being banned, a global traffic surge, or an automatic unban, the team
receives a formatted alert within seconds. Additionally, a Flask-based dashboard
provides a live look at current metrics and system health.
•
•
•

Conclusion
Securing infrastructure is not just about building walls; it's about building systems
that can think. By combining Python’s data processing power with Linux’s networking
tools, I've created a resilient, self-correcting defense mechanism that scales its
sensitivity based on actual usage patterns.
The code for this project is open-source and available on GitHub at https://github.com/Adewumicrown/hng-anomaly-detector for anyone looking to try it on their own