From Access Log to Kernel Drop: Building a Single-Process WAF Ban Pipeline in C

In self-hosted Linux infrastructure, bridging L7 application security with L3/L4 network enforcement traditionally requires stitching together multiple independent daemons. A typical setup involves an inline proxy module for application-layer inspection, a log parser to tail access logs, and an external utility to push dynamic iptables rules.

While functional, this approach introduces architectural fragmentation, higher memory overhead, and multiple points of failure. To address this, I built Linux Log Guardian — a self-hosted, MIT-licensed origin WAF and dynamic IP ban pipeline implemented as a single C process.

Here is an architectural walkthrough of how it works, how it achieves median ~26 ms ban latency, and the honest tradeoffs made during design.

---

1. The Architecture: Pipeline Overview

The entire pipeline operates within a single binary with zero external daemon dependencies:

Nginx access.log -> Zero-Copy Log Parser -> OWASP CRS (PCRE2 JIT) -> Policy Engine (Named Skips) -> Kernel Drop (XDP / ipset)

• Ingestion & Parsing: Tails Nginx access logs using a zero-copy line parser (parse_log_line()) and io_uring disk I/O.
• L7 WAF Scoring: Passes parsed fields through 121 OWASP CRS patterns compiled with PCRE2 JIT.
• Policy Engine: Evaluates scores against named skips and purpose-bound policies.
• Kernel Enforcement: Pushes dynamic block rules directly to the NIC via XDP (eBPF), with an automated fallback to ipset if daemon boundaries fail.

---

2. High-Performance L7 Inspection in C

Rather than running inline inside Nginx (which risks worker crashes if a rule panics), the pipeline tails access logs asynchronously.

To keep CPU and memory overhead low:

• Zero-Copy Parsing: Log lines are parsed by pointing directly into buffers without memory allocation.
• PCRE2 JIT Execution: OWASP CRS rules are pre-compiled at startup. To prevent Regular Expression Denial of Service (ReDoS) attacks, match limits (pcre2_set_match_limit) are strictly enforced.

c
// Example snippet: Enforcing match execution limits for PCRE2 JIT
pcre2_match_context *mctx = pcre2_match_context_create(NULL);
pcre2_set_match_limit(mctx, 10000); // Guard against ReDoS chokepoints

Benchmarked engine throughput reaches 114,003 EPS vs 17,193 EPS for ModSecurity on the same 121 CRS pattern set (Note: Measured via regex pattern replay (@rx), not end-to-end HTTP processing).
3. Policy Controls: Named Skips & Purpose Binding

A common issue in automated blocking systems is "all-or-nothing" enforcement. If a rule produces a false positive, administrators are often forced to disable the entire WAF engine.

Linux Log Guardian introduces Named Skips:

    skip_consult_only: The WAF continues scoring and logging, but dynamic kernel banning is paused for that specific rule class.

    skip_fp_trust: Bypasses ban triggers for trusted subnets while maintaining detection metrics.

Additionally, Purpose Binding ensures that threat intelligence feeds can only contribute to scoring metrics and can never directly write kernel ban rules without explicit policy authorization.
4. Down to the Kernel: XDP and ipset Fallback

When a log line triggers a ban threshold, the IP must be dropped before the attacker can issue subsequent requests.

Log Event -> Threat Score -> IPC -> eBPF Map Update -> XDP_DROP

XDP Path: Drops packets directly at the network interface card (NIC) driver level before sk_buff allocation, completely bypassing the Linux network stack.

    ipset Fallback: If XDP driver support is missing or the daemon crashes, a fallback mechanism shifts enforcement to ipset rules.

Measured Latency: Median ban latency from log write to kernel enforcement is ~26 ms (n=5 lab sample; soft SLO targeted at p50 < 75 ms, p99 <= 150 ms).
5. Honest Limits & Operational Realities

Every architecture makes tradeoffs. Here is where this pipeline honestly ends:

    Reactivity: This is an origin-side, log-tailing layer. The first malicious request in an attack payload will hit the application before its log line is parsed.

    Not a CDN Replacement: It does not protect against volumetric L3/L4 DDoS attacks. The recommended stack is: CDN -> Nginx limit_req -> Linux Log Guardian.

    Scope: Designed specifically for single-host and edge-origin Linux servers needing lightweight, reproducible security controls.

Try It Yourself

The project is fully open-source, MIT-licensed, and backed by reproducible test suites.

    GitHub: https://github.com/kurtulusutkucenik/Linux-Log-Guardian

    Live Test Matrix & Demo: https://www.ceniklinuxlogguardian.org/