DEV Community

jaryn
jaryn

Posted on

Audit WAF Deny Logs Before a Model Explains the Block

#security #ai #python #devops

The failing request is one 403. Not a new CVE. Not a jailbreak.

192.0.2.10 - - [03/Sep/2026:09:14:22 +0000] "POST /admin/users HTTP/1.1" 403 182 "-" "Mozilla/5.0" "session=CANARY_SESS_7f3a; theme=dark"
Enter fullscreen mode Exit fullscreen mode

I keep that line in fixtures/negative.log. I used to treat lines like it as a puzzle. Paste into a chat. Ask what the probe wanted. Then I looked at the Cookie field again. The WAF already held a live session token. My terminal already held a second copy. Why hand a third copy to a model host whose disk I do not rotate?

That is the invariant. If a deny log still contains CANARY_SESS, it must never become a prompt.

The deny log is already a secret store

You already refuse to paste .env into a chat. Do you treat $http_cookie the same way?

Edge logs exist for incident response. They were not designed as model input. During a messy outage someone adds $http_authorization, $http_cookie, or $request_body to log_format. The next 403 looks rich. It is rich because it still holds a credential.

A model does not need the victim's cookie to classify a probe. It needs the method, the path shape, the status, and the rule that fired. The rest is just another reader of a secret you already failed to isolate.

Five readers of one 403

I draw the trust boundary on the log line, not on the model vendor.

[browser] --cookie--> [WAF/edge]
                         |  intended hop
                         v
                      [origin]
                         |
                         |  extra copy is born
                         v
                   [deny / access log]
                         |
                         v
                   [analyst laptop]
                         |
                         |  the copy people forget
                         v
              [model HTTP API / worker]
                         |
                         v
         [proxy access_log, container journal, backup]
Enter fullscreen mode Exit fullscreen mode

Client to WAF is normal. WAF to origin is normal. WAF to the log volume is where the extra copy appears. Laptop to model API is the hop this fixture gates. Model host to its own journal is the hop most “we self-host, so we are safe” stories skip.

The threat is disclosure. Not prompt injection. Not weight theft. A session token in a completion log is a credential in a file with the wrong retention.

Reader Needs the cookie? Typical retention
Origin app Yes, for the session Session lifetime
WAF decision engine Sometimes, for bot/session rules Minutes in memory
Deny log No Days to years
Analyst terminal No Shell history, forever
Model prompt store No Whatever the host logs

If the cookie is not required for classification, it does not cross hop four. Full stop.

What I refuse to send

I keep a deny-list next to the redactor. Not a vibe. A list the fixture can fail.

  1. Cookie / Set-Cookie values, including the boring theme= sibling that sits next to session=.
  2. Authorization bearer values and any eyJ JWT-shaped blob in query, header, or body.
  3. Password-reset tokens and magic-link query strings in the referrer.
  4. Raw request bodies. That is where mass-assignment payloads and pasted API keys hide.
  5. Internal source IPs when the reader is a model I do not operate. I hash them. I do not need RFC1918 coordinates to say “this looks like credential stuffing.”
  6. Emails, upload filenames, and UUIDs in the path if the ticket is “what rule fired,” not “who was the victim.”

Can the model still be useful? Yes. Method, path template, status, bytes, rule id, and a coarse user-agent family are enough for “this is a scanner” versus “this is a broken client.”

Reproduce the leak with two fixtures

I do not start from a production bucket. I start from two files I can check into git.

fixtures/positive.log — should survive redaction unchanged in the fields I care about:

203.0.113.40 - - [03/Sep/2026:09:14:22 +0000] "GET /healthz HTTP/1.1" 403 18 "-" "curl/8.5.0" "-"
Enter fullscreen mode Exit fullscreen mode

fixtures/negative.log — must not survive:

192.0.2.10 - - [03/Sep/2026:09:14:22 +0000] "POST /admin/users?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.CANARY_JWT.sig HTTP/1.1" 403 182 "https://app.example/reset?token=CANARY_RESET" "Mozilla/5.0" "session=CANARY_SESS_7f3a; theme=dark"
Enter fullscreen mode Exit fullscreen mode

The canaries are the point. If CANARY_SESS_7f3a, CANARY_JWT, or CANARY_RESET appear downstream of the redactor, the gate failed. I do not need a real user session to prove that.

Expected evidence when the gate is absent:

grep -E 'CANARY_(SESS|JWT|RESET)' fixtures/negative.log
# expected: matches. this is the failing request.
Enter fullscreen mode Exit fullscreen mode

Expected evidence when the gate is present:

python3 redact_waf_log.py --in fixtures/negative.log --out /tmp/redacted.log --canary CANARY_
grep -E 'CANARY_' /tmp/redacted.log && echo 'FAIL' || echo 'PASS'
# expected: PASS, exit 0 from the redactor's canary check
Enter fullscreen mode Exit fullscreen mode

Treat the snippets below as a runnable fixture against these two files. I am not claiming a production WAF finding.

A redactor you can fail in CI

Pin the runtime: Python 3.12. No extra packages. The script exits 2 if a canary survives, 1 on IO errors, 0 when the negative fixture is clean and the positive fixture still has method, path, and status.

#!/usr/bin/env python3
"""Redact WAF/nginx-style access lines before they become a model prompt.

Fixture-only. Not a production SIEM pipeline.
"""
from __future__ import annotations

import argparse
import hashlib
import ipaddress
import re
import sys
from pathlib import Path

JWT = re.compile(r"eyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+")
UUID = re.compile(
    r"[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}"
)
EMAIL = re.compile(r"[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}")
QUERY_SECRET = re.compile(
    r"(?i)([?&](?:token|session|auth|jwt|password|reset)=)[^&\s]+"
)
HEADERISH = re.compile(
    r"(?i)(session=|Authorization:\s*Bearer\s+)[^;\s]+"
)


def hash_ip(raw: str) -> str:
    try:
        ipaddress.ip_address(raw)
    except ValueError:
        return raw
    digest = hashlib.sha256(raw.encode("utf-8")).hexdigest()[:12]
    return f"ip_{digest}"


def redact_line(line: str) -> str:
    parts = line.split(" ", 1)
    if parts and parts[0]:
        line = hash_ip(parts[0]) + (" " + parts[1] if len(parts) > 1 else "")
    line = JWT.sub("[REDACTED_JWT]", line)
    line = UUID.sub("[REDACTED_UUID]", line)
    line = EMAIL.sub("[REDACTED_EMAIL]", line)
    line = QUERY_SECRET.sub(r"\1[REDACTED]", line)
    line = HEADERISH.sub(r"\1[REDACTED]", line)
    return line


def main() -> int:
    p = argparse.ArgumentParser()
    p.add_argument("--in", dest="src", required=True)
    p.add_argument("--out", dest="dst")
    p.add_argument("--canary", default="CANARY_")
    args = p.parse_args()
    src = Path(args.src)
    try:
        raw = src.read_text(encoding="utf-8", errors="replace").splitlines()
    except OSError as exc:
        print(exc, file=sys.stderr)
        return 1
    redacted = [redact_line(line) for line in raw]
    text = "\n".join(redacted) + ("\n" if redacted else "")
    if args.dst:
        Path(args.dst).write_text(text, encoding="utf-8")
    else:
        sys.stdout.write(text)
    survivors = [line for line in redacted if args.canary in line]
    if survivors:
        print(f"canary survived in {len(survivors)} line(s)", file=sys.stderr)
        return 2
    return 0


if __name__ == "__main__":
    raise SystemExit(main())
Enter fullscreen mode Exit fullscreen mode

CI shape I actually want. Two steps. No model in the loop.

python3 redact_waf_log.py --in fixtures/negative.log --out /tmp/neg.out --canary CANARY_
test $? -eq 0
grep -E 'POST|403|/admin/users' /tmp/neg.out
python3 redact_waf_log.py --in fixtures/positive.log --out /tmp/pos.out --canary CANARY_
grep 'GET /healthz' /tmp/pos.out
Enter fullscreen mode Exit fullscreen mode

If the negative fixture still prints CANARY_, you do not have a redactor. You have a comment in a wiki.

Prevent at the log format, not in the chat window

Regex after the fact is a seatbelt. The cheaper control is not writing the secret.

A noisy nginx-style format that I treat as the prevent-side failing config:

# failing example — do not ship
log_format too_much '$remote_addr $request $status '
                    '$http_cookie $http_authorization $request_body';
Enter fullscreen mode Exit fullscreen mode

A format that still lets me triage a block:

log_format waf_triage '$remote_addr $request_method $uri $status '
                      '$body_bytes_sent $http_user_agent $upstream_status';
Enter fullscreen mode Exit fullscreen mode

If you run SafeLine or any reverse-proxy WAF, read the access-log template the same way you would read a verbose debug flag. I am not describing a product bug. I am describing the extra field someone enabled at 2 a.m. Cookie, authorization, and raw body do not belong in the default deny export you later feed to a classifier.

Tradeoff, because dropping fields hurts investigations:

Field Keep for triage? Send to a model?
Method + path template Yes Yes
Status + rule id Yes Yes
User-agent family Yes Yes
Full cookie header Forensics only No
Authorization Rotate, do not log No
Raw body Packet capture, not INFO logs No
Source IP Maybe Hash, then decide

Detect on the model host anyway

You will still have a human who bypasses the script. So I plant the same canary in a synthetic deny line and grep the other side.

Unexecuted template against a local OpenAI-compatible proxy. Label it that way. Swap the URL for whatever you actually run.

# template — do not point this at production logs
CANARY='CANARY_SESS_7f3a'
python3 redact_waf_log.py --in fixtures/negative.log --out /tmp/prompt.txt --canary CANARY_
# After a human-shaped classify call, on the model host:
grep -R --line-number -e "$CANARY" /var/log /tmp 2>/dev/null || true
# expected: no match in proxy access logs or worker journals
Enter fullscreen mode Exit fullscreen mode

If the canary shows up in the proxy access log, you did not only leak a cookie to the model. You leaked it to log_format on the model hop. That is boundary six. Self-hosting does not delete it.

Recover if it already left

Assume the paste happened. Do not argue with the journald timestamp.

  1. Rotate the session, the JWT signing key if the token was a service credential, and any reset token in the referrer.
  2. Delete or tombstone the prompt store, container logs, and object-storage export that held the raw line. Retention is part of the incident, not a footnote.
  3. Add the canary fixture to the log-export job that failed. A one-off grep is not a control.
  4. Record whether the model host was yours. If it was not, treat the prompt as a third-party disclosure and follow that vendor's deletion path. Do not pretend a chat “unsend” rewrites their disk.

Prevent / detect / recover

Layer Prevent Detect Recover
WAF log format Drop cookie, auth, body from the default export Lint log_format in CI against a deny-list of variables Rewrite the template, roll the access log
Export job Run redact_waf_log.py before any classify step Canary must be absent; exit 2 fails the job Re-run export from the unredacted vault you still control
Analyst laptop No raw deny files in shell history snippets grep CANARY_ on the clipboard file you were about to send Rotate the canary-mapped credential
Model proxy Accept only the feature JSON, not the raw line Grep worker journals for canaries after a synthetic request Wipe prompt cache, rotate host disks if the raw line landed

Where a free model actually helps

I want a classifier on the redacted feature JSON. Method, path template, status, rule id. That job is boring and cheap. It does not need the cookie.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I have used MonkeyCode's free model access and free server option as one place to classify already-redacted deny lines. The redactor is ordinary Python. It does not depend on that platform. If the line is still raw, a free server is just another disk that now holds a session token.

The honest split: run the gate on a machine you administer. Send only the redacted fixture to whatever model you can afford. If even the redacted path templates are too identifying, do not send them off-box at all.

Who should not use this

Do not use regex redaction as anonymization. /accounts/1842/invoices still names a tenant after the cookie is gone.

Do not use this as a legal-hold pipeline. Forensics needs the raw packet in a vault with access control, not in a chat transcript.

Do not point the template proxy call at production logs. The fixtures are the point. Production comes after the canary stays dead.

Do not skip the gate because the model is free, local, or “just for triage.” Cost is not a trust boundary. Logs are.

Limitations I will not hand-wave: encodings beat regex. Base64 in a query string, JSON inside ?payload=, multipart bodies, and log-format drift will all slip. Path templates still leak identifiers. A free or self-hosted model operator can still read the redacted prompt. This fixture proves a canary died. It does not prove every secret died.

Which invariant belongs in CI?

The canary check belongs in CI, on the export job, with the two fixtures above. That is a regression test. It does not need a GPU.

The proxy allow-list — feature JSON in, raw access line out — belongs on the model hop. Humans in a hurry will bypass a wiki. They will not bypass a sidecar that rejects CANARY_ and eyJ.

So: which layer do you want to fail first when the next 403 looks interesting? The export job, or the model proxy? I want the export job red. The proxy is the backstop, not the policy.

Top comments (0)