Why I Developed an AWS WAF Log Analysis Tool

#aws #waf #security

TL; DR

I've launched Accompanist, a tool for analyzing and visualizing AWS WAF logs.
You can use it to quickly evaluate accesses and fine-tune the rules of AWS WAF.
Analyzing WAF logs is an essential operation in preparation for cyber attacks and high-impact vulnerabilities, even if WAF rules haven't been used aggressively until now.

Introduction of this tool

The original intent behind the tool, Accompanist, is to serve as your "accompanist" in WAF operations, assisting you in identifying trends in malicious access and attack patterns.

Currently, this tool supports AWS WAFv2 and can handle logs of BLOCK or COUNT actions. It is written in Python and can be installed using the following command:

pip install accompanist

Here's how this tool operates:

Download WAF logs locally from CloudWatch Logs or S3 bucket.
Calculate the top 5 items in various categories such as rules, URL paths, IPs, etc.
And it generates a PDF report based on the calculated data.

In Accompanist, It executes the above with 2 or 3 CLI commands with ease.

I originally developed this analysis tool for our team's WAF with the goal of streamlining our analysis operations. Subsequently, I released it for startups or small organizations without a dedicated WAF operator.

However, while developing this tool, I have come to understand not just the trends in attacks but also the potential importance of analyzing WAF logs. In other words, I figured out why we should install WAFs in all workloads and, of course, needs to analyze the log.

Q : When and what should you use this tool?

In the context of enterprise companies, WAF operators actively monitor logs and block malicious or bot accesses with robust policies and carefully tailored customizations. This represents an ideal situation and a favorable environment.

On the other hand, it's a well-known issue that some WAF operators are troubled by numerous false positive accesses. This implies the necessity to fine-tune the WAF rule configuration to exclusively ban malicious tools and permit normal user access. However, this is no simple task and can't be resolved quickly. As a result, operators often analyze logs and use the COUNT action feature.

The COUNT action, a key concept in AWS WAF, merely counts up malicious accesses without blocking them. This is a standard and common feature among various WAFs. For instance, a similar feature named "Preview Mode" exists in Google Cloud's WAF, Cloud Armor. Thus, the COUNT action proves extremely useful for evaluating and testing rules to prevent blocking normal accesses, such as internal bots or batch accesses executed by the company itself.

In this blog post, I won't dig into methods for fine-tuning WAF rules, but you may add certain conditions to rules or limit the scope of rules' targets.

Here we assume the following scenario.

An early-career engineer working at a unicorn company often finds themselves deeply concentrating on developing new features while operating under substantial pressure from their superiors. Consequently, application security isn't always the top priority, and the implementation of a WAF may be overlooked. However, they think rapidly enabling WAF rules becomes an attractive solution when faced with a targeted DDoS attack or high-impact vulnerabilities such as Log4J.

Months later, the moment they had feared arrives. They rapidly deploy a WAF and apply the robust and latest rule set. Fortunately, the attack is prevented and the successful blocking by the WAF rules is confirmed. Just as they are beginning to breathe a sigh of relief, an engineer from the neighboring department calls, "Our scheduled Lambda function has been terminated. We must address this issue promptly, or we risk losing a significant sales opportunity!"

A : "At this moment" and "To understand it"

As you may have surmised from the story, the root cause of the problem is the lack of understanding of the current state of the web system, specifically, the normal access patterns. As, sometimes, distinguishing between normal and malicious access can be challenging, the key to overcoming this issue is routinely analyzing the WAF logs. Furthermore, fine-tuned rules are essential to WAF operations, even in startups.

I understand this might seem time-consuming, a bit bothering, and require a particular passion. I couldn't agree more.
However, I firmly believe that continuously steady work on cybersecurity gives us ultimate safeguards from devastating data breaches or terrific leakages in the future.