DEV Community

Sam Rivera
Sam Rivera

Posted on

Turn Your Side Project's Logs Into a Daily Triage Digest Instead of Reading Them Raw

#python #ai #productivity #indiedev

My side project runs fine most days, which is exactly the problem. When it breaks, I find out from a user or from my own bank statement, because reading raw logs every evening is a chore I stopped doing around week three.

So I built the smallest possible fix: a nightly job on a free hosted box that pulls the day's logs, asks an LLM to group the errors into a digest, and posts me a summary I actually read. This post is the whole setup, including the part that failed first.

The constraint set

  • Budget: $0. This runs on free model access and a free server option from MonkeyCode. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used it here because the job is small, stateless, and a bad reason to pay for anything — but the script below takes any OpenAI-compatible endpoint, so nothing is locked in.
  • Time: about 40 minutes to set up, including the failure I'll show you.
  • Risk ceiling: if the digest is wrong, I lose nothing — the raw logs are untouched. That property mattered to me more than summary quality.

The artifact

digest.py, ~55 lines, no dependencies beyond requests:

import json, os, sys, requests
from collections import Counter

LOG_DIR = os.environ.get("LOG_DIR", "/var/log/myapp")
API_BASE = os.environ["LLM_API_BASE"]      # any OpenAI-compatible endpoint
API_KEY = os.environ["LLM_API_KEY"]
MODEL = os.environ.get("LLM_MODEL", "default")
MAX_CHARS = 12_000                          # hard cap, see failure below

def today_lines():
    import glob, datetime
    day = datetime.date.today().isoformat()
    lines = []
    for path in glob.glob(f"{LOG_DIR}/*.log"):
        with open(path, errors="replace") as f:
            for line in f:
                if day in line and ("ERROR" in line or "WARN" in line):
                    lines.append(line.strip())
    return lines

def pre_aggregate(lines):
    # Cheap local grouping first: normalize numbers/UUIDs, count patterns.
    import re
    pats = Counter()
    for l in lines:
        norm = re.sub(r"[0-9a-f-]{8,}", "<id>", l)
        norm = re.sub(r"\d+", "<n>", norm)
        pats[norm[:200]] += 1
    return pats.most_common(30)

def summarize(patterns):
    payload = "\n".join(f"{c}x {p}" for p, c in patterns)[:MAX_CHARS]
    r = requests.post(
        f"{API_BASE}/chat/completions",
        headers={"Authorization": f"Bearer {API_KEY}"},
        json={
            "model": MODEL,
            "messages": [
                {"role": "system", "content":
                 "You triage app logs. Output: 1) top 3 issues by count, "
                 "2) anything NEW (count 1-2) that looks actionable, "
                 "3) one line: ignore-worthy noise. Be terse."},
                {"role": "user", "content": payload},
            ],
        },
        timeout=60,
    )
    r.raise_for_status()
    return r.json()["choices"][0]["message"]["content"]

if __name__ == "__main__":
    lines = today_lines()
    if not lines:
        sys.exit(0)  # silent day is a good day
    digest = summarize(pre_aggregate(lines))
    requests.post(os.environ["WEBHOOK_URL"], json={"text": digest}, timeout=10)
Enter fullscreen mode Exit fullscreen mode

And the cron line on the free server:

30 21 * * * LOG_DIR=/var/log/myapp LLM_API_BASE=... LLM_API_KEY=... WEBHOOK_URL=... /usr/bin/python3 /opt/digest.py >> /var/log/digest-cron.log 2>&1
Enter fullscreen mode Exit fullscreen mode

Why the pre-aggregation step exists (the failure fixture)

First version sent raw log lines straight to the model. On a bad deploy day my app logged the same connection error 4,000 times with different request IDs. The model got a truncated wall of identical noise and produced a confident, useless summary about "database connectivity" — the actual problem was a misconfigured retry loop, visible only in the count of the pattern, which truncation had destroyed.

The fix is boring and local: normalize IDs and numbers, count with Counter, send the top 30 patterns with their counts. The model now sees 4000x connect timeout to <id> instead of 4000 lines. This also caps my input size at MAX_CHARS, which is the entire cost-control story.

Limitations, honestly

  • This is triage, not diagnosis. The digest tells me where to look, never what to do. I verify against raw logs before changing anything.
  • Free tiers are a canary, not a foundation. If the endpoint is slow or rate-limited one night, I get no digest and I don't notice. My rollback criterion: if digests silently fail twice in a week, I add a dead-man's-switch ping (a heartbeat that alerts when the digest doesn't arrive). Until then, missing a night is acceptable because logs persist.
  • Don't send logs with user PII through any hosted model. Mine are request IDs and stack traces. If yours aren't, add a redaction pass in pre_aggregate or skip this approach entirely.
  • Who shouldn't use this: anything with an SLA, on-call rotation, or compliance requirements. This is a solo-builder convenience layer over grep, not monitoring. If you need alerting, you need alerting.

What I'd do next

The digest currently has no memory, so a recurring low-count error gets flagged as "NEW" every night. A tiny SQLite table of seen-pattern hashes would fix that. If you've built something similar, what's the one field you track per pattern to keep repeat noise down — first-seen date, count delta, something else?

Top comments (0)