the size cap on my log scanner turns itself off exactly when it can't measure the file

I maintain a small python tool called SIEMForge. it builds and converts SIEM detection content, Sigma rules plus a Sysmon config plus Wazuh rules, and it ships a log scanner so you can test a rule against a raw log file before you push it to a real SIEM.

a while back i added a size cap to the scanner. every parse path reads the whole file into memory, so a 5GB syslog or some attacker-supplied json blob would eat all the RAM the process can grab before matching even starts. the cap is 100MB by default, overridable with an env var. boring, and exactly the kind of thing you add once and forget.

last week i was reading the coverage report properly instead of skimming the total, and the size guard had a branch nothing tested. here's the whole function:

def _check_log_size(path: Path) -> None:
    limit = _resolve_max_bytes()
    try:
        size = path.stat().st_size
    except OSError:
        return
    if size > limit:
        raise LogFileTooLargeError(
            f"Log file {path} is {size} bytes, which exceeds the "
            f"{limit}-byte limit. Set SIEMFORGE_MAX_LOG_BYTES to raise it."
        )

look at the except. if stat() throws, the function returns None and the scan keeps going. so the safety check disables itself precisely when it can't measure the file. and "can't stat" is not a friendly condition. it's a permissions problem, or a race where the file changed under you, or some weird mount. the kind of thing that shows up when someone is poking at your box, not when everything is calm.

i wrote a test to pin the behavior down, not to change it:

def test_check_log_size_ignores_stat_error(self, tmp_path, monkeypatch):
    f = tmp_path / "x.json"
    f.write_text("[]", encoding="utf-8")

    def boom(self, *a, **k):
        raise OSError("stat blocked")

    monkeypatch.setattr(Path, "stat", boom)
    assert _check_log_size(f) is None

that's a characterization test. it says "this is what the code does today," and if someone changes it the test yells. i left the behavior in place because there's a real argument for it: don't refuse to scan a file just because you couldn't read its size. but for a security tool the other side of that argument is louder. the guard exists to stop a memory blowup, and the one case where it steps aside is the adversarial one.

then i kept pulling the thread, and the same question showed up everywhere. what does this scanner do when it meets something it doesn't understand? the answer wasn't consistent.

the size-too-big path fails loud. it raises, scan_logs catches it, prints "Failed to parse", returns -1. you know something went wrong. but an unknown file extension doesn't raise. the format resolver just falls back to json, the json parser reads a not-json file, gets nothing back, and you see "No alerts, all events are clean." drop a .evtx in by mistake and the tool hands it a clean bill of health.

CSV is the same shape. if the header sniffer can't make up its mind, the parser treats every row as data and names the columns col_0, col_1, col_2:

def test_csv_sniffer_error_falls_back_to_positional(self, tmp_path, monkeypatch):
    def boom(self, sample):
        raise csv.Error("could not determine delimiter")

    monkeypatch.setattr(csv.Sniffer, "has_header", boom)
    f = tmp_path / "ambiguous.csv"
    f.write_text("""a,b
1,2
""", encoding="utf-8")
    events = parse_log_file(f, fmt="csv")
    assert events[0] == {"col_0": "a", "col_1": "b"}

the data is all there, but your Sigma rule is keyed on CommandLine and Image, and there is no CommandLine anymore, there's col_4. so the rule matches nothing and the scan comes back quiet.

the part that bugs me is the split. some of these fail loud and some fail silent, and there's no principle behind which is which. it's just wherever i happened to write a raise versus a return. for most tools that's fine. for a thing whose entire job is answering "did anything match," silent-zero is the worst possible default, because "0 alerts" now means one of two opposite things: your logs are clean, or my scanner quietly gave up. an analyst can't tell those apart from the outside, and the second one is the one that gets someone owned.

the tests i added don't fix any of this. they took the suite from 90% to 96%, but the number isn't the point. the point is they made the inconsistency legible. i can now see, in one file, every place the scanner swallows a problem and returns empty. that list is the actual to-do.

what i'd change, roughly in order: make the unknown-suffix fallback print a warning instead of silently guessing json, log a one-line notice when a CSV drops to positional keys, and reconsider the stat() swallow so a file it can't measure gets flagged rather than waved through. none of that is hard. the hard part was noticing that "0 alerts" had been lying to me by omission.

honest gaps, since i'd want them if i were reading this:

• the stat() fail-open only triggers if stat() actually throws, which is rare. rare is not never, and rare tends to line up with "under attack."
• the quantifier parser fails closed on a malformed count, which is defensible. a broken rule condition shouldn't invent an alert. but it's still a silent no-match with no warning that the rule was bad, so a typo in a condition just detects nothing, forever.
• the positional-key CSV fallback keeps the data addressable, which really is better than crashing, but only if something downstream knows to read col_N. nothing does.

repo if you want to see the rest: https://github.com/TiltedLunar123/SIEMForge

it works. the detection logic is fine. i just learned that the scariest output a detection tool can give you is a calm, confident zero.