DEV Community

Cover image for Building an AI-Powered Auth Log Anomaly Detector with MITRE ATT&CK
PyHackSecGP
PyHackSecGP

Posted on

Building an AI-Powered Auth Log Anomaly Detector with MITRE ATT&CK

Why Auth Logs Matter

Auth logs are underrated.

Every brute-force attempt, privilege escalation, suspicious sudo usage, and unexpected SSH login leaves a trace in /var/log/auth.log.

The problem is volume and signal-to-noise.

A busy server can generate thousands of authentication log lines every day. Most are legitimate logins, sudo usage, SSH keepalives, automated jobs, and monitoring activity.

The attack patterns are there, but they look like noise without context.

I wanted a tool that could read auth logs like a security analyst — detecting patterns, adding context, and helping identify attacker behavior.

What I Built

P3 — a Python tool that:

  • Parses Linux authentication logs
  • Applies rule-based detections
  • Maps findings to MITRE ATT&CK
  • Enriches source IPs with reputation data
  • Uses a local LLM for threat assessment
  • Alerts through Slack or Discord
  • Supports continuous watch mode
  • Provides machine-readable exit codes and JSON output

And the best part:

Zero third-party Python dependencies. Standard library only.


Detection Rules

SSH Brute Force — T1110

The first detection uses a sliding window:

≥5 failed SSH authentication attempts from one IP within 10 minutes.

window = [
    t for t in self._ssh_failures[ip]
    if t > cutoff
]

count = len(window)

if count >= 5:
    severity = (
        "CRITICAL" if count >= 50
        else "HIGH" if count >= 20
        else "MEDIUM"
    )
Enter fullscreen mode Exit fullscreen mode

Severity scales with the number of failures.

Five failures might be a user repeatedly entering the wrong password.

Fifty failures in ten minutes is a very different situation.


Distributed Brute Force — T1110

Single-IP detection misses distributed attacks.

A botnet can make only two or three authentication attempts from each IP — never enough to trigger a per-IP threshold.

So P3 also looks at the aggregate pattern:

≥10 unique source IPs with ≥20 combined failures.

This matters because the mitigation can be different.

A single noisy IP might justify an IP block.

A distributed attack may require broader controls such as rate limiting, geo-blocking, MFA enforcement, or investigation of the targeted accounts.


Credential Stuffing — T1110

This is one of the most important detections.

If an IP has authentication failures and then successfully authenticates, that pattern deserves immediate attention.

Conceptually:

if ip in self._ssh_failures and ip in self._ssh_successes:
    # Failure followed by success
    # Investigate as potential credential compromise
Enter fullscreen mode Exit fullscreen mode

P3 treats this as a CRITICAL finding.

The reasoning is simple: repeated failures followed by a successful login can indicate that the attacker eventually obtained or guessed valid credentials.

The detection itself is strong, but in a production environment I'd still recommend correlating it with account identity, source history, MFA events, and other telemetry before declaring compromise.


Privilege Escalation — T1548

P3 monitors failed sudo authentication attempts per user.

It also looks for suspicious su activity.

For example, unexpected attempts to switch to root from an unusual account can indicate privilege escalation.


Account Creation — T1136

Account creation events such as useradd are flagged.

An attacker who already has shell access may create a new account to establish persistence.

These findings are therefore treated as HIGH severity by default.


Root Login — T1078

Direct SSH authentication as root is flagged as CRITICAL.

On a properly hardened Linux server, direct root SSH access is normally disabled.

If you suddenly see successful root authentication, that's worth investigating immediately.


MITRE ATT&CK Mapping

Every finding includes a technique ID and technique name.

For example:

[BF-001] CRITICAL — SSH Brute Force

MITRE: T1110 (Brute Force)
Confidence: 87%
Source IP: 185.234.218.x
AbuseIPDB: 97% abuse score
Country: CN
Enter fullscreen mode Exit fullscreen mode

This is useful during incident response.

A finding that says:

"Many failed logins detected."

isn't particularly actionable.

A finding mapped to T1110 — Brute Force gives the analyst a known attack technique to investigate, along with documented detection guidance, mitigations, and related procedures.

The goal is to turn raw log activity into something that can feed an incident-response workflow.


Local AI Analysis

The AI component runs against a local Ollama instance.

Instead of sending authentication logs to a cloud API, P3 builds a summary containing the detected findings and representative log lines.

The local model returns:

  1. Overall threat assessment — CRITICAL/HIGH/MEDIUM/LOW
  2. Key findings
  3. Attack pattern analysis
  4. Immediate recommendations
  5. IOCs — IPs, usernames, and patterns worth blocking or investigating

No cloud service.

No API keys.

No authentication data leaving the machine.

That's important because authentication logs can contain usernames, source IPs, hostnames, and other sensitive operational information.


Watch Mode + Alerting

P3 can run continuously as a systemd service.

For example:

python3 main.py /var/log/auth.log \
  --watch \
  --interval 60 \
  --webhook "$SLACK_WEBHOOK" \
  --allowlist allowlist.yaml
Enter fullscreen mode Exit fullscreen mode

Every 60 seconds it:

  1. Parses the log
  2. Detects anomalies
  3. Compares them with previously seen findings
  4. Alerts only on new HIGH or CRITICAL events

This prevents a brute-force campaign that runs for several hours from generating the same alert every minute.

Slack and Discord webhooks are supported.


Alert Context

A useful alert should contain enough information to act without opening the raw log immediately.

P3 includes context such as:

  • Detection rule
  • MITRE ATT&CK technique
  • Severity
  • Confidence score
  • Source IP
  • Username
  • IP reputation
  • Number of attempts
  • AI-generated summary
  • Recommended next steps

The objective is simple:

Don't just tell me that something happened. Tell me why it matters.


Exit Codes as an API

P3 also exposes simple exit codes:

0 — No anomalies / only low severity
1 — HIGH severity finding
2 — CRITICAL severity finding
Enter fullscreen mode Exit fullscreen mode

That makes it easy to integrate with other automation.

For example:

python3 main.py /var/log/auth.log \
  --no-ai \
  --json /tmp/daily.json

[ $? -ge 2 ] && page_oncall
Enter fullscreen mode Exit fullscreen mode

No SIEM required.

You can wire it into cron jobs, monitoring systems, incident-response scripts, or other automation.


IP Reputation Enrichment

P3 optionally integrates with AbuseIPDB.

For each source IP, it can retrieve:

  • Abuse confidence score
  • Country code
  • Number of reports in the last 90 days

Context matters.

A brute-force attack from an IP with a 97% abuse confidence score and thousands of reports is more suspicious than the same number of failures from an IP with no reputation history.

Both events should be detected.

The reputation data simply helps determine how urgently to escalate them.


Allowlisting

Internal infrastructure can generate authentication events that look suspicious.

Jump boxes, monitoring agents, automation systems, and backup servers are common examples.

P3 supports an allowlist.yaml:

trusted_ips:
  - 10.0.1.5     # Prometheus exporter

trusted_users:
  - ansible
  - backup
Enter fullscreen mode Exit fullscreen mode

Allowlisted events are filtered before detection runs.

That prevents known internal activity from creating unnecessary alerts.


What I Learned

The interesting engineering problems weren't the detection rules.

Those were relatively straightforward.

The difficult parts were everything around them.

1. Sliding Windows Across Log Parses

The detection window needs to survive across watch-mode cycles.

That means storing timestamps, not just counters.

Otherwise, every scan starts from zero and the detector loses historical context.

2. Deduplication in Watch Mode

A continuously running detector shouldn't alert on the same anomaly every minute.

P3 hashes the identifying fields of an anomaly and alerts only when it sees a new event.

3. Log Format Variance

Linux authentication logs aren't completely uniform.

Debian-based systems commonly use:

/var/log/auth.log
Enter fullscreen mode Exit fullscreen mode

while RHEL-based systems commonly use:

/var/log/secure
Enter fullscreen mode Exit fullscreen mode

The syslog formatting can also vary.

P3 handles both formats and also supports journalctl JSON export.

4. Local LLM Latency

Local AI is great for keeping sensitive logs on the machine, but inference isn't free.

P3 uses a timeout of around 120 seconds for AI analysis.

That means watch-mode intervals need to account for model latency, otherwise one scan can overlap with the next.


What Enterprise SIEMs Do Better

Splunk, Datadog, and other enterprise SIEM platforms obviously have P3 beat in several areas:

  • Scale
  • Cross-source correlation
  • Long-term data retention
  • Behavioral baselining
  • Enterprise identity integration
  • Team workflows
  • Dashboards
  • Alert management

P3 isn't trying to compete with them.

It's a small, transparent tool for understanding authentication activity on a Linux system without requiring an entire SIEM stack.


Why Build This?

The biggest benefit isn't just the detection engine.

It's visibility.

I can see exactly:

  • How a detection works
  • Why an event was classified as suspicious
  • Which MITRE technique it maps to
  • What reputation data influenced the assessment
  • What the local LLM recommended
  • What gets sent to the alerting system

There's no black box hiding the detection logic.

You can read the Python, change the thresholds, add a detection, modify the scoring, or remove the AI component entirely.

That's the point.

Build the security logic you understand.


What's Next?

A few things I'd like to add:

  • More MITRE ATT&CK techniques
  • Baseline-based anomaly detection
  • Geo/IP clustering for distributed attacks
  • Better correlation between successful and failed authentication
  • Automatic firewall integration
  • A lightweight web dashboard
  • More structured incident-response output
  • Feedback from analyst decisions to improve AI assessments

The long-term goal is to make P3 useful as a lightweight first-response layer before an event reaches a full SIEM.


Final Thought

Auth logs contain a surprising amount of security intelligence.

The challenge isn't collecting them.

It's turning thousands of individual events into a small number of high-confidence, actionable signals.

That's what I wanted P3 to do.

Code: https://github.com/PyHackSecGP/p3-log-anomaly-detector

If you're building something similar, I'd be interested in how you're detecting authentication anomalies and handling the false-positive problem.

Top comments (0)