i build security tools as a cybersec student, mostly so i actually understand the stuff i'm studying instead of just memorizing it for Security+. ThreatLens started because i got tired of scrolling raw windows event logs looking for the one line that mattered.
the idea is simple. point it at a folder of logs, it runs detections, it tells you what looks like an attack. offline, no server, no agent. just a CLI.
the problem
if you've ever opened a security.json export with 900k events in it you know the feeling. somewhere in there is a brute force, or a service getting installed it shouldn't, or someone dumping creds. but you're not finding it by eye. you need something to do the first pass so you can spend your time investigating instead of reading.
i wanted that first pass to be local. a lot of the triage stuff assumes you've already shipped everything to a SIEM. i don't always have one running on my own boxes, and i wanted something i could just pip install and run.
what i tried
first version just regex'd through lines and counted failed logins. that catches the dumbest brute force and nothing else. real attacks are multi step. someone gets in with valid creds (T1078), runs something (T1059), sets up persistence (T1543), then moves sideways. a single rule never sees the whole picture.
so the detections became modules, 12 of them, each mapped to a MITRE technique. that part was fine. the part that took longer was correlation. a failed-login spike on its own is noise. a failed-login spike followed by a success followed by a new service on the same host is a story. linking those across detection boundaries is where most of the actual work went.
one design choice i'm happy with: separating targeted brute force from credential spray. they look similar if you just count failures. but a brute force hammers one account in a burst, and a spray tries one password across many accounts slowly. burst analysis on the timing tells them apart, and they're different investigations, so calling them the same thing would be lying to the analyst.
what worked
YAML rules with a handful of operators (equals, contains, regex, threshold) cover most of what i wanted to write without touching python. and it reads Sigma rules directly, which mattered because i didn't want to reinvent a rule format the whole industry already uses. selections, filters, field modifiers, conditions. not the full sigma-rs engine, but enough.
output goes to terminal, json, csv, or an html report with severity charts and an attack timeline. it can also push to elastic, splunk HEC, wazuh, or spit out an ATT&CK Navigator layer so you can see coverage on the matrix.
the numbers
synthetic benchmark, single core, python 3.11 on windows 11:
- 9,009 events (2.3 MB): 0.13s, about 69k events/sec
- 90,145 events (22.6 MB): 1.27s
- 901,341 events (226 MB): 14.24s, about 63k events/sec
on the sample dataset it hit every embedded technique and threw zero false positives on the benign activity. small sample, so i'm not going to pretend that's a real detection-rate claim, but it's a start.
what's broken / what i'd change
here's the honest part. at 63k events/sec it's slower than Chainsaw and Hayabusa, which are compiled. for a single-core python tool reading EVTX i think that's fine, but if you've got hundreds of millions of events you'd want one of those instead. i'm not pretending to compete on raw speed.
the parser also wants event IDs and syslog fields mapped to formats it knows. throw it something weird and it'll shrug. that's the next thing i want to fix, a more forgiving field mapper so onboarding a new log source doesn't mean editing code.
and the false-positive testing needs real ugly data, not synthetic. synthetic logs are too clean. real environments are full of weird-but-benign stuff that trips naive rules, and i won't trust the detection numbers until i've run it against messier input.
it's defensive only. no remote access, no capture, no exploit anything. just reads logs you already have on systems you're allowed to look at.
it works. not perfect, but it works, and i learned more about how attacks actually chain together building this than i did from any single chapter of studying.
repo: https://github.com/TiltedLunar123/ThreatLens
if you do detection engineering for real, i'd genuinely like to know what log source you'd want supported first.
Top comments (0)