When I first started in DevSecOps, this felt overwhelming. How do you stop attacks when you are asleep?
The answer is Automated Anomaly Detection. I recently built a Python-based security engine that watches server traffic, learns what "normal" looks like, and automatically bans malicious bots.
If you have never worked on security tooling before, don't worry! I am going to break down exactly how this project works, piece by piece, without the dense jargon.
What This Project Does (And Why It Matters)
Imagine your web server is an exclusive nightclub, and the Nginx log file is the guest list. Every time someone visits your site, Nginx writes their IP address in the log.
This project acts as an invisible bouncer standing next to the list. It reads the logs in real-time. If someone is walking in and out a normal amount of times, the bouncer ignores them. But if one person tries to enter the club 500 times in ten seconds, the bouncer grabs them, throws them out, and locks the door.
This matters because manual security doesn't scale. You cannot sit at your computer 24/7 watching logs. By automating this, your server protects itself.
1. The Sliding Window: The Short-Term Memory
To know if someone is attacking us right now, the engine needs short-term memory. We use a concept called a Sliding Window.
Imagine a 60-second timer. Every time an IP address makes a request, we record the exact timestamp. As time ticks forward, we continuously "drop" any timestamps that are older than 60 seconds.
[ Time: 10:00:00 ] -> Request comes in. Add to window.
[ Time: 10:00:30 ] -> More requests come in.
[ Time: 10:01:05 ] -> The request from 10:00:00 is now too old. We drop it!
This ensures we are always looking at the current rate of traffic. In Python, we do this using a deque (a double-ended queue), which makes it incredibly fast to add new requests to the right side and pop old requests off the left side.
from collections import deque
import time
class RateTracker:
def init(self):
# Keeps memory of timestamps
self.requests = deque()
def add_request(self):
self.requests.append(time.time())
def get_rate(self):
current_time = time.time()
# Remove anything older than 60 seconds
while self.requests and self.requests[0] < current_time - 60:
self.requests.popleft()
# The remaining length is our current Requests-Per-Second!
return len(self.requests) / 60.0
2. The Baseline: How the Engine "Learns"
A static rule like "Ban anyone who makes 50 requests" is actually a terrible idea. What if your site goes viral? Suddenly, your engine will ban all your real customers!
Instead, the engine needs to learn what "normal" is. We call this the Baseline.
Every minute, our engine takes a snapshot of the global traffic rate and saves it. Over time, it calculates two things:
The Mean (Average): The normal amount of traffic we expect to see.
The Standard Deviation: How much traffic naturally "wiggles" up and down on a normal day.
If normal traffic is 10 requests per second (the Mean), and it usually fluctuates by about 2 requests (the Standard Deviation), the engine learns that anything between 8 and 12 is perfectly safe.
3. The Detection Logic: The Judge
Now we have our short-term memory (Sliding Window) and our long-term learning (Baseline). How do we make a decision to ban someone?
We use a mathematical concept called a Z-Score. Don't let the math term scare you! A Z-Score simply answers the question: "How weird is this right now compared to what we normally see?"
Here is the logic:
We look at a specific IP address's current request rate.
We subtract the "normal" Baseline Mean.
We divide by the Standard Deviation.
If the Z-Score is 1, it means traffic is slightly elevated, but fine. If the Z-Score hits 5, it means that IP is behaving so aggressively that there is practically a 0% chance it is a normal user.
When the logic sees a high Z-Score, it pulls the alarm.
4. iptables: The Muscle
Once the engine decides an IP is an attacker, it needs to physically block them from the server. It does this using iptables.
iptables is the built-in firewall for Linux systems. Think of it as a brick wall surrounding your server. When our Python script catches a bad IP, it executes this exact terminal command:
sudo iptables -A INPUT -s 192.168.1.100 -j DROP
Let's break down what this command actually means:
sudo: Run this with absolute administrator power.
iptables: Wake up the firewall program.
-A INPUT: Append (add) a new rule to the INPUT door (incoming traffic).
-s 192.168.1.100: Look for this specific source IP address.
-j DROP: Jump to the DROP action. Do not send an error message, do not say hello, just silently drop their connection into a black hole.
Once this command runs, that attacker can no longer touch our Nginx server, view our website, or waste our CPU resources. They are completely locked out!
Wrapping Up
By combining Nginx logs, a sliding window, statistical math, and Linux firewalls, we built an engine that thinks, learns, and defends entirely on its own. DevSecOps isn't just about reading logs—it is about building automated systems that do the hard work for you.
Have you ever built a security tool or worked with iptables? Let me know in the comments below!
Top comments (0)