DEV Community

Charlie Zhu
Charlie Zhu

Posted on

From Log Sprawl to a Two-Sentence Morning Report

The application server wrote 2.3 GB of logs last night. The team ignored them, because reading them would take three hours. Then a payment outage went unnoticed for forty minutes because the relevant error was buried under routine warnings.

That is the standard failure mode of logging. Volume grows faster than attention, so teams either ignore logs entirely or page on every keyword and then disable the pager. The middle path is a small summarizer that runs before anyone is awake.

This article builds one. The tool reads the last 24 hours of error and warning lines, sends them to a free model tier for grouping and explanation, and writes a short Markdown brief. A free server option hosts the cron job. The whole thing is about sixty lines of Python.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The design has three stages: extract, summarize, deliver. Extraction is deliberately crude. The script scans each log file for lines containing ERROR, WARN, Exception, or Traceback, and keeps the last 200 matches. That cap protects the prompt from overflowing and forces the model to focus on recent activity.

# nightly_log_brief.py
import os
import re
from datetime import datetime, timedelta
from pathlib import Path
from openai import OpenAI

LOG_DIR = Path('/var/log/myapp')
SINCE = datetime.now() - timedelta(hours=24)

def extract_candidates(log_path: Path) -> list[str]:
    candidates = []
    with log_path.open(errors='ignore') as fh:
        for line in fh:
            if any(level in line for level in ('ERROR', 'WARN', 'Exception', 'Traceback')):
                candidates.append(line.strip())
    return candidates[-200:]
Enter fullscreen mode Exit fullscreen mode

A structured log parser would be nicer, but most projects do not have one. The regex-free substring check works on plain text, JSON lines, and even stack traces. It will miss a log line that says 'something went wrong' without a keyword, but it will catch the lines that matter for a morning triage.

The summarization step is one prompt. It asks the model to group the lines by root cause, give a one-line explanation per group, and suggest a next step. The temperature is low, and the instruction to avoid inventing details is explicit.

def summarize(candidates: list[str]) -> str:
    client = OpenAI(
        api_key=os.environ['MONKEYCODE_API_KEY'],
        base_url=os.environ.get('MONKEYCODE_BASE_URL'),
    )
    prompt = f'''Below are the last 200 error/warning lines from a 24-hour log window.
Group them by root cause. For each group, give a one-line explanation and a suggested next step.
Keep the whole answer under 200 words. Do not invent details not present in the lines.

Lines:
{chr(10).join(candidates)}'''
    resp = client.chat.completions.create(
        model=os.environ.get('MONKEYCODE_MODEL'),
        messages=[{'role': 'user', 'content': prompt}],
        temperature=0.2,
    )
    return resp.choices[0].message.content
Enter fullscreen mode Exit fullscreen mode

The prompt is the part you will tune. Adding a line like the service is called payment-worker helps the model use the right names. Removing the word 'WARN' from the extraction pattern reduces noise if warnings are too chatty. The script is meant to be edited, not worshipped.

The delivery stage writes the summary to a file that the team reads with coffee. A cron entry on the free server option runs the script at 6 a.m. every day.

0 6 * * * cd /opt/log-brief && python nightly_log_brief.py >> /var/log/log-brief.log 2>&1
Enter fullscreen mode Exit fullscreen mode

To test the script without waiting for a cron run, point LOG_DIR at a sample directory and run it manually. The output file will appear in the current directory if you change the output path. For teams that prefer chat notifications, replace the file write with a webhook call; the rest of the script stays the same.

Here is the output from a controlled test with a 24-hour log sample. The model grouped 163 error lines into three root causes.

# Log brief for 2026-08-25T06:00:01

1. **Timeout errors in payment worker** (142 occurrences): The worker is hitting a 30s timeout on the upstream payment API. Suggested next step: check the API's status page and increase the timeout to 45s if the incident continues.
2. **Database connection pool exhaustion** (18 occurrences): The pool size is too low for the morning batch job. Suggested next step: raise `max_connections` from 10 to 20 and monitor.
3. **Single recurring stack trace in auth service** (3 occurrences): A null pointer in session renewal. Suggested next step: add a null check and deploy a patch.
Enter fullscreen mode Exit fullscreen mode

The summary is not perfect. The model guessed that the timeout was caused by an upstream incident, which was plausible but unverified. That is the right failure mode: the tool flags, a human confirms, and the team spends five minutes instead of three hours.

The limitations are real. The model sees only the lines you feed it, so logs without context produce shallow summaries. The free tier is fine for a once-a-day job, but it is not a real-time alerting system. And if logs contain customer data, sending them to an external model requires redaction first. A simple redaction step could replace email addresses and IPs before the prompt is built.

Teams with a dedicated observability platform do not need this. Teams with real-time paging requirements should not use a nightly cron. And teams with zero error logs should fix their logging before adding a summarizer.

The lesson is that a small, narrow automation beats a grand platform that never ships. The free model and free server are enough for this job, and the job is worth having. If you have a log directory and a morning coffee, the script above is a good starting point.

Top comments (0)