DEV Community: Log Audit

Apache vs Nginx Log Analysis: Security Patterns to Watch

Log Audit — Fri, 03 Apr 2026 08:15:47 +0000

Apache vs Nginx Log Analysis: Security Patterns to Watch

Most security log analysis guides focus on one or the other. But many production environments run both — Nginx as a reverse proxy in front of Apache, or Apache handling legacy apps while Nginx serves new services. Understanding the differences matters.

Log Format Differences

Nginx Default Combined Format

Top 5 Security Events Hidden in Your Server Logs

Log Audit — Fri, 03 Apr 2026 08:15:11 +0000

Top 5 Security Events Hidden in Your Server Logs

Your server logs are a security goldmine — and a graveyard. They contain evidence of every attack, every reconnaissance probe, every failed intrusion attempt. They also contain evidence of successful ones that you haven't noticed yet.

Here are the 5 most dangerous security events that hide in plain sight in your logs.

1. Slow Brute Force Attacks

What they look like: A single IP making 1–2 login attempts per hour. Slow enough to bypass rate limits. Patient enough to fly under the radar for weeks.

What Logs Do You Need for GDPR Compliance?

Log Audit — Fri, 03 Apr 2026 08:14:35 +0000

What Logs Do You Need for GDPR Compliance?

GDPR doesn't have a single "you must log X" requirement — but Articles 5, 25, 30, and 32 together create a clear picture of what logging you need to demonstrate compliance. Get it wrong and you're not just non-compliant, you may be logging PII you shouldn't be.

The Two Logging Challenges Under GDPR

GDPR creates a double-edged problem for logging:

You must log — to demonstrate accountability, detect breaches, and respond to access requests
You must not over-log — logging PII unnecessarily violates data minimisation principles

This tension is why GDPR logging is tricky.

What You're Required to Log

Article 30 — Records of Processing Activities

You need to maintain a record of all data processing activities. For logging purposes, this means documenting:

Who accesses personal data
When they access it
For what purpose
From which system

This isn't necessarily in your server logs — but your server logs are evidence that supports this record.

Article 32 — Security of Processing

You must implement "appropriate technical measures" including "a process for regularly testing, assessing and evaluating the effectiveness of technical and organisational measures."

In practice: you need logs that let you detect and investigate security incidents.

Article 33 — Breach Notification

You have 72 hours to notify your supervisory authority of a breach. Without logs, you can't:

Confirm a breach occurred
Determine what data was affected
Establish a timeline
Identify who was impacted

The Minimum Logging Requirements for GDPR

1. Authentication Events

Every login to systems that process personal data:
\

How to Detect Brute Force Attacks in Nginx Access Logs

Log Audit — Fri, 03 Apr 2026 08:13:43 +0000

How to Detect Brute Force Attacks in Nginx Access Logs

Brute force attacks are relentless. Automated bots hammer login endpoints thousands of times per minute hoping to hit a weak password. Your Nginx access logs record every attempt — here's how to find them before the attackers succeed.

What a Brute Force Attack Looks Like

A single IP hammering your login endpoint:

SOC2 Type II Audit Checklist for SaaS Teams (2026)

Log Audit — Fri, 03 Apr 2026 08:13:40 +0000

SOC2 Type II Audit Checklist for SaaS Teams (2026)

SOC2 Type II is no longer just for enterprise. More and more B2B SaaS buyers are asking for it before they sign. If you're a small team trying to get certified without a dedicated compliance team, this checklist covers what you actually need — with a focus on logging.

What Is SOC2 Type II?

SOC2 (System and Organization Controls 2) is a framework developed by the AICPA. Type I is a point-in-time snapshot. Type II covers a period of time (usually 6–12 months) and verifies that your controls are operating effectively throughout that period.

The five Trust Service Criteria are:

Security (required)
Availability
Processing Integrity
Confidentiality
Privacy

Most SaaS teams pursue Security + Availability as the minimum scope.

The Logging Requirements Auditors Care About

Logging is one of the most scrutinized areas of a SOC2 audit. Here's what auditors look for:

1. Access Logs

Every authentication event should be logged:

Successful logins (user, IP, timestamp, user-agent)
Failed login attempts
Password resets
MFA challenges

5 Nginx Log Patterns Every SaaS Developer Should Monitor

Log Audit — Wed, 01 Apr 2026 10:01:00 +0000

Most SaaS applications run behind Nginx, and most teams only look at their logs when something breaks. That is a mistake. Your access logs are a real-time feed of what is happening to your application — including attacks, abuse, and infrastructure problems — if you know what to look for.

Here are five patterns worth monitoring continuously.

1. Repeated 401/403 Responses to the Same Endpoint

A spike in 401 Unauthorized or 403 Forbidden responses targeting a single endpoint — especially /api/login, /admin, or /api/token — is a strong indicator of brute force or credential stuffing activity.

awk '$9 == "401" || $9 == "403" {print $7}' /var/log/nginx/access.log \
  | sort | uniq -c | sort -rn | head -20

If you see hundreds of hits on /api/login returning 401, an automated attack is almost certainly in progress. Combine with IP analysis:

awk '$9 == "401" {print $1}' /var/log/nginx/access.log \
  | sort | uniq -c | sort -rn | head -20

A single IP hammering your login endpoint warrants an immediate block via fail2ban or a firewall rule.

2. Abnormal 4xx/5xx Ratios

A healthy application has a low error rate — typically under 1-2%. A sudden spike in 500 errors often means a deploy went wrong, a dependency is down, or something is crashing under load. A spike in 400 errors can indicate a scanning tool probing malformed requests.

awk '{print $9}' /var/log/nginx/access.log \
  | grep -E '^[45]' \
  | sort | uniq -c | sort -rn

Track this over time with a rolling window:

tail -n 10000 /var/log/nginx/access.log \
  | awk '{print $9}' \
  | sort | uniq -c | sort -rn

If 500 errors appear after a deployment, you want to know within minutes — not hours.

3. High Request Volume from a Single IP

Legitimate users do not send 10,000 requests per hour. Bots, scrapers, and DDoS traffic do. Catching this early lets you rate-limit or block before it impacts paying users.

awk '{print $1}' /var/log/nginx/access.log \
  | sort | uniq -c | sort -rn | head -20

For a more useful view, filter to the last hour only:

awk -v date="$(date '+%d/%b/%Y:%H')" '$4 ~ date {print $1}' /var/log/nginx/access.log \
  | sort | uniq -c | sort -rn | head -10

Pair this with nginx rate limiting in your config:

limit_req_zone $binary_remote_addr zone=api:10m rate=30r/m;

server {
    location /api/ {
        limit_req zone=api burst=10 nodelay;
    }
}

4. Requests to Non-Existent Routes (404 Farming)

A wave of 404 responses across random paths — /wp-admin, /.env, /phpMyAdmin, /config.json — is an automated scanner probing for known vulnerabilities. These are typically harmless if your app does not have those files, but they indicate your server is being actively enumerated.

awk '$9 == "404" {print $7}' /var/log/nginx/access.log \
  | sort | uniq -c | sort -rn | head -30

Watch for patterns like:

/.env — environment file leakage probes
/wp-login.php — WordPress brute force
/actuator/health — Spring Boot endpoint scanning
/.git/config — source code exposure attempts

If you see these, the traffic is coming from automated tools. Blocking the source IP is reasonable.

5. Slow Response Times on Critical Paths

Nginx can log request processing time with $request_time. If your checkout, login, or payment endpoints are suddenly taking 5+ seconds, something is wrong — slow queries, N+1 problems, or resource exhaustion.

First, enable timing in your log format:

log_format timed '$remote_addr - $remote_user [$time_local] '
               '"$request" $status $body_bytes_sent '
               '$request_time $upstream_response_time';

access_log /var/log/nginx/access.log timed;

Then find your slowest endpoints:

awk '{print $NF, $7}' /var/log/nginx/access.log \
  | sort -rn | head -20

A sudden increase in $upstream_response_time (time your backend took) vs $request_time (total time) tells you whether the bottleneck is in your app or at the Nginx layer.

Putting It Together

Monitoring these five patterns manually is a start, but doing it continuously at scale requires automation. You need alerting when error rates spike, when a new IP crosses a request threshold, or when a scanner starts probing your endpoints.

If you want this without spinning up a full ELK stack, LogAudit runs these checks against your Nginx logs automatically and alerts you when something worth acting on shows up.

GDPR and Your Nginx Logs — What PII Are You Accidentally Logging?

Log Audit — Thu, 26 Mar 2026 11:46:25 +0000

GDPR and Your Nginx Logs — What PII Are You Accidentally Logging?

Most developers know GDPR applies to their databases and forms. Few realise their nginx access logs may already be GDPR non-compliant — logging personal data without consent, no retention policy, no protection.

Here is what your default nginx log line looks like:

185.22.30.11 - - [26/Mar/2026:08:20:00 +0000] "GET /reset-password?token=abc123&email=john.doe@example.com HTTP/1.1" 200 512

You just logged an email address. That is personal data under GDPR Article 4. And it is sitting in a plaintext file on your server.

What PII Ends Up in Nginx Logs?

1. Email addresses in query parameters

Password resets, magic links, email verification — all commonly pass email as a URL parameter:

GET /verify?email=user@example.com&token=xyz
GET /unsubscribe?email=user@example.com
GET /invite?ref=user@example.com

2. Names and usernames in URLs

GET /profile/john.doe
GET /u/jane_smith/settings
GET /invoices/acme-corp-march-2024

3. Session tokens and auth tokens

GET /api/data?session=eyJhbGciOiJSUzI1NiJ9...
GET /download?token=sk_live_abc123

4. IP addresses

Under GDPR, IP addresses are considered personal data in most EU member states (CJEU ruling, 2016). Your nginx access log logs every IP by default.

5. Health and sensitive data in URLs

GET /doctors/oncology/book?patient=john-doe
GET /therapy/session?user_id=4421

Health-related data is special category data under GDPR Article 9 — even stricter rules apply.

Why This Matters

Log retention: GDPR requires data minimisation. Keeping access logs for 90 days "just in case" may not have a lawful basis.
Log access: Who can read your access logs? If it is everyone on your team, you may be over-sharing personal data.
Log storage: Are logs encrypted at rest? GDPR Article 32 requires appropriate technical measures.
Breach notification: If your log files are exfiltrated, that is a personal data breach requiring notification within 72 hours.

What You Can Do

Option 1: Strip PII from log format

map $remote_addr $remote_addr_masked {
    ~^(\d+\.\d+\.\d+)\. $1.0;
    default               $remote_addr;
}

log_format gdpr_safe '$remote_addr_masked - $remote_user [$time_local] '
                     '"$request_method $uri $server_protocol" '
                     '$status $body_bytes_sent';

Option 2: Exclude sensitive URL patterns

map $request_uri $log_it {
    ~*email=  0;
    ~*token=  0;
    default   1;
}
access_log /var/log/nginx/access.log combined if=$log_it;

Option 3: Redact PII from existing logs

#!/bin/bash
# Run daily via cron
LOG=/var/log/nginx/access.log.1
sed -i -E 's/[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/[REDACTED]/g' $LOG

Option 4: Set a log retention policy

/var/log/nginx/*.log {
    daily
    rotate 30
    compress
    missingok
    notifempty
}

Building a GDPR-Ready Log Pipeline

The goal is privacy by design (GDPR Article 25):

Collect only what you need — strip PII at ingestion
Retain only as long as needed — 30-day rotation as default
Encrypt logs at rest
Restrict access to log files
Scan regularly for PII that slips through

Quick Checklist

[ ] Review nginx log format for PII fields
[ ] Check if query parameters contain emails or tokens
[ ] Set log rotation to 30 days max
[ ] Encrypt log storage
[ ] Document lawful basis for log retention in your DPIA
[ ] Set up automated PII scanning

GDPR compliance is not just about your database. Your logs are data too. Tools like LogAudit scan nginx logs automatically for PII exposure, API key leaks, and compliance issues — so you catch problems before your next audit.

5 Security Patterns You Should Be Scanning For in Your Apache/Nginx Logs

Log Audit — Tue, 24 Mar 2026 08:45:36 +0000

Every web server generates logs. Most teams ignore them until something breaks. But your access logs are a goldmine of security intelligence — if you know what to look for.

After analyzing millions of log lines across production systems, here are 5 attack patterns that show up consistently. Each one is easy to detect, and catching them early can save you from a breach.

1. Path Traversal Attempts

What it looks like:

192.168.1.105 - - [24/Mar/2026:03:14:22 +0000] "GET /images/../../etc/passwd HTTP/1.1" 400 0
10.0.0.33 - - [24/Mar/2026:03:14:45 +0000] "GET /static/%2e%2e/%2e%2e/etc/shadow HTTP/1.1" 403 0
203.0.113.50 - - [24/Mar/2026:03:15:01 +0000] "GET /download?file=../../../proc/self/environ HTTP/1.1" 200 1245

Why it matters: Attackers use ../ sequences (or URL-encoded variants like %2e%2e) to escape the web root and read system files. The third example is the scariest — it returned a 200, meaning the server actually served the file.

What to scan for:

Requests containing .. or %2e%2e in the URI
Any 200 responses to paths referencing /etc/, /proc/, or system directories

grep -iE "(\.\\./|%2e%2e|%252e)" /var/log/nginx/access.log

2. Brute Force Detection (Repeated 401s)

What it looks like:

198.51.100.14 - admin [24/Mar/2026:04:00:01 +0000] "POST /admin/login HTTP/1.1" 401 112
198.51.100.14 - admin [24/Mar/2026:04:00:01 +0000] "POST /admin/login HTTP/1.1" 401 112
198.51.100.14 - admin [24/Mar/2026:04:00:02 +0000] "POST /admin/login HTTP/1.1" 401 112
198.51.100.14 - admin [24/Mar/2026:04:00:03 +0000] "POST /admin/login HTTP/1.1" 200 3842

Why it matters: A burst of 401 responses from the same IP followed by a 200 is the textbook signature of a successful brute force attack.

What to scan for:

More than 5-10 failed auth attempts (401/403) from a single IP within a short window
A 200 response after a series of failures (indicates successful compromise)

awk \ == 401 {print \} /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20

3. XSS Attempts in Query Strings

What it looks like:

10.0.0.88 - - [24/Mar/2026:05:22:10 +0000] "GET /search?q=<script>alert(document.cookie)</script> HTTP/1.1" 200 4521
172.16.0.5 - - [24/Mar/2026:05:22:33 +0000] "GET /profile?name=%3Cimg%20src%3Dx%20onerror%3Dalert(1)%3E HTTP/1.1" 200 3200

Why it matters: XSS payloads in URLs mean someone is actively testing whether your application sanitizes input. If those requests return 200, your app may be vulnerable.

What to scan for:

<script>, <svg, <img tags in query parameters
Event handlers like onerror=, onload=
URL-encoded versions of the above

grep -iE "(<script|%3Cscript|onerror=|onload=|javascript:)" /var/log/nginx/access.log

4. Sensitive File Access (.env, .git, wp-admin probing)

What it looks like:

45.33.32.156 - - [24/Mar/2026:06:10:05 +0000] "GET /.env HTTP/1.1" 200 890
45.33.32.156 - - [24/Mar/2026:06:10:06 +0000] "GET /.git/config HTTP/1.1" 200 234
45.33.32.156 - - [24/Mar/2026:06:10:07 +0000] "GET /wp-admin/ HTTP/1.1" 302 0
45.33.32.156 - - [24/Mar/2026:06:10:08 +0000] "GET /phpinfo.php HTTP/1.1" 200 52314

Why it matters: This is an automated scanner probing for exposed config files. A .env file returning 200 is a critical incident — it likely contains database credentials, API keys, and secrets.

What to scan for:

Requests for dotfiles: .env, .git/, .htaccess, .aws/credentials
Admin panel probing: /wp-admin, /phpmyadmin, /adminer
Any of the above returning a 200 status code

grep -iE "(\.env|\.git|wp-admin|phpinfo|\.aws|server-status)" /var/log/nginx/access.log

5. API Keys and Tokens Leaked in URLs

What it looks like:

10.0.0.22 - - [24/Mar/2026:07:30:15 +0000] "GET /api/data?api_key=sk_live_4eC39HqLyjWDarjtT1zdp7dc HTTP/1.1" 200 1580
10.0.0.22 - - [24/Mar/2026:07:30:16 +0000] "GET /webhook?token=ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx HTTP/1.1" 200 40

Why it matters: This is not an external attack — it is your own application leaking secrets. API keys passed as query parameters get logged in plain text in access logs, browser history, and analytics tools.

What to scan for:

Query parameters named api_key, token, access_token, secret
Known key patterns: sk_live_, ghp_, AKIA (AWS), xoxb- (Slack)

grep -iE "(api_key=|token=|access_token=|secret=|sk_live_|ghp_|AKIA)" /var/log/nginx/access.log

Immediate action: If you find leaked keys, rotate them immediately.

Automate It

Scanning for these patterns manually works, but it does not scale. You have to check every log rotation, across every server, every day.

I built LogAudit to automate exactly this — it scans your nginx/Apache logs against these patterns (and more), flags issues by severity, and gives you a compliance score. Free tier available if you want to try it.

But regardless of what tools you use — start scanning. These patterns are hitting your servers right now.

How to Detect Credential Stuffing Attacks in Your Nginx Logs

Log Audit — Fri, 20 Mar 2026 11:33:45 +0000

How to Detect Credential Stuffing Attacks in Your Nginx Logs

Credential stuffing is when attackers take leaked username/password lists from data breaches and try them automatically against your login endpoint. Unlike brute force (guessing random passwords), credential stuffing uses real credentials — which makes it far more dangerous and harder to spot.

The good news: it leaves a very clear pattern in your nginx logs.

What Credential Stuffing Looks Like

A normal login attempt looks like this:

192.168.1.1 - - [20/Mar/2026:09:15:00 +0000] "POST /api/login HTTP/1.1" 200 156

A credential stuffing attack looks like this:

185.220.101.34 - - [20/Mar/2026:09:15:01 +0000] "POST /api/login HTTP/1.1" 401 53
185.220.101.34 - - [20/Mar/2026:09:15:02 +0000] "POST /api/login HTTP/1.1" 401 53
185.220.101.34 - - [20/Mar/2026:09:15:02 +0000] "POST /api/login HTTP/1.1" 401 53
185.220.101.34 - - [20/Mar/2026:09:15:03 +0000] "POST /api/login HTTP/1.1" 401 53
185.220.101.34 - - [20/Mar/2026:09:15:03 +0000] "POST /api/login HTTP/1.1" 200 156

Key signals:

Same IP, rapid succession — dozens of 401s in seconds
Single endpoint — always hitting /login, /api/auth, /signin
Eventually a 200 — when they find a valid credential
No other activity — not browsing the site, just hammering the endpoint

Detecting It with grep and awk

# Find IPs with many failed logins (401s) to login endpoint
grep 'POST /api/login' /var/log/nginx/access.log | \
  grep ' 401 ' | \
  awk '{print $1}' | \
  sort | uniq -c | sort -rn | head -20

# Find IPs with 10+ failed logins
grep 'POST /api/login' /var/log/nginx/access.log | \
  grep ' 401 ' | \
  awk '{print $1}' | \
  sort | uniq -c | \
  awk '$1 >= 10 {print $1, $2}' | sort -rn

# Check if any of those IPs then got a 200 (successful login)
SUSPECT_IP="185.220.101.34"
grep "$SUSPECT_IP.*POST /api/login" /var/log/nginx/access.log | \
  awk '{print $9}' | sort | uniq -c

Detecting Distributed Credential Stuffing

Sophisticated attackers rotate IPs to avoid per-IP rate limits. This is harder to catch but still has a pattern:

# High volume of 401s across many IPs in a short window
grep 'POST /api/login' /var/log/nginx/access.log | \
  grep ' 401 ' | \
  awk '{print $4}' | \
  cut -d: -f1-3 | \
  sort | uniq -c | sort -rn | head -10
# Shows 401 counts per hour — a spike = distributed attack

# Count unique IPs hitting login in last hour vs normal
grep 'POST /api/login' /var/log/nginx/access.log | \
  awk -v date="$(date -u +'%d/%b/%Y:%H')" '$4 ~ date {print $1}' | \
  sort -u | wc -l

Block It with fail2ban

Create /etc/fail2ban/filter.d/nginx-login.conf:

[Definition]
failregex = ^<HOST> .* "POST /api/login HTTP.+" 401
ignoreregex =

Add to /etc/fail2ban/jail.local:

[nginx-login]
enabled  = true
port     = http,https
filter   = nginx-login
logpath  = /var/log/nginx/access.log
maxretry = 10
bantime  = 3600
findtime = 60

This bans any IP that fails login 10 times within 60 seconds for 1 hour.

systemctl restart fail2ban
fail2ban-client status nginx-login

nginx Rate Limiting

For defense-in-depth, add rate limiting directly in nginx:

# In http block
limit_req_zone $binary_remote_addr zone=login:10m rate=5r/m;

# In server block
location /api/login {
    limit_req zone=login burst=10 nodelay;
    limit_req_status 429;
    # ... rest of config
}

This limits each IP to 5 login attempts per minute with a burst of 10.

Warning Signs You've Been Hit

Check these after finding a stuffing campaign:

# Any successful logins from attacking IPs?
ATTACKER_IPS=$(grep 'POST /api/login' /var/log/nginx/access.log | \
  grep ' 401 ' | awk '{print $1}' | sort | uniq -c | \
  awk '$1 >= 20 {print $2}')

for IP in $ATTACKER_IPS; do
  SUCCESS=$(grep "$IP.*POST /api/login" /var/log/nginx/access.log | grep ' 200 ')
  if [ -n "$SUCCESS" ]; then
    echo "COMPROMISED: $IP got a 200"
    echo "$SUCCESS"
  fi
done

If you find successful logins from attacking IPs, those accounts are compromised and need immediate password resets.

Beyond grep: Automated Detection

Manual log analysis works for incident response but you need continuous monitoring to catch attacks as they happen — not hours later.

LogAudit automatically detects credential stuffing patterns in your nginx logs in real time, alerting you the moment an attack campaign starts rather than after accounts are compromised. Free tier available.

Found this useful? The next post covers GDPR risks hiding in your nginx logs — specifically what PII you might be accidentally logging in query strings.

How to Detect SQL Injection Attempts in Your Nginx Logs

Log Audit — Thu, 19 Mar 2026 21:11:35 +0000

placeholder