A common penetration test malicious visitors like to deploy is running a script full of not-so-random URIs.
Depending on if they receive a 200, 4xx, or 5xx response, this will let them know certain things about your web app. Such as:
- What tech stack is being used, which can let them know how to exploit the site further
- If files that contain sensitive information are exposed
These types of attacks often result in the generation of a plethora of 404 errors.
"404 attack" isn't the best descriptor of what the attack is, but 404 errors are a byproduct that we can key off of to help stomp these types of attacks.
In addition to the security issues these attacks expose, another reason to consider stomping these attacks is the server load they expend. These attacks will often occur over a relatively short period, but will go through hundreds, or thousands of URLs, which can easily impact the performance of any app hosted on the server.
In this exercise, we'll provision a server using Cleavr and add a WordPress site. Cleavr installs and configures fail2ban, which we'll further configure to detect and squash these 404 attacks.
We'll create a filter rule for fail2ban to check the NGINX
access.log to detect if an IP generates too many 404 errors within a specified period of time. If an IP is caught breaking the rules, we'll put them in jail by temporarily banning them from accessing the server.
With a test server ready to go, let's put our detective cap on!
SSH into your server. View our guide on how to SSH into your server. Or, use your favorite SFTP client.
Add the following new file named
[Definition] failregex = ^<HOST>.*"(GET|POST).*" (404|444|403|400) .*$ ignoreregex =
The above is a filter definition that tells fail2ban to look for errors marked 400, 403, 404, and 444.
/etc/fail2ban/jail.conf and add the following block of code to the end of the file.
[nginx-4xx] enabled = true port = http,https filter = nginx-4xx logpath = /var/log/nginx/access.log bantime = 1800 findtime = 10 maxretry = 10
In the above, pay attention to
findtime. This is saying if a user hits the 4xx error 10 times over a 10 second period, then ban the user's IP for 1800 seconds, which is 30 minutes.
Modify the values according to your needs - but, I recommend something that will deter attackers, who tend to probe many pages in less than a minute, but that also will minimally impact legitimate visitors.
Now, restart process by running following command:
service fail2ban restart
To view status, including jailed IPs, you can run the following command:
fail2ban-client status nginx-4xx
Now, go to the website, type in a URL that doesn't exist and will generate a 404 error, then refresh quickly until you get blocked by the server.
You can then go back to the terminal and run
fail2ban-client status nginx-4xx to see that you have been jailed. 🚔
It is important to note that this is just one way to handle this type of issue and it may not be the best option for you. You should perform the due diligence and verify this method works for you, your system, and your users.