Weekend Build Log: Redact the Log Before You Send It

Friday, 11:40pm. A staging deploy fails and the CI log runs to 412 lines. You copy the whole thing into a chat window to ask what broke. Halfway through the paste you notice a bearer token and an internal hostname.

That pause is the entire project. This weekend's build is a small log-triage tool with one hard rule. Redact first, ask a model second, validate the answer third.

I kept the scope brutal on purpose. If a side project cannot ship in a weekend, it usually ships never.

The scope cut, as a table

Every weekend project dies from a feature list, not from a hard bug. So the first artifact is the cut itself.

Idea	Kept?	Why
Regex redaction gate	Yes	It is the only part that protects you
One model call per log	Yes	Simple to reason about, simple to test
Strict JSON output check	Yes	Bad output must fail loudly
Offline golden-file test	Yes	You need to run this without a key
Web UI	No	A CLI plus cron is enough
Database	No	Flat files on disk are fine here
Retry and queue logic	No	One retry, then write to failed/
Multi-model routing	No	One endpoint, one model per run
User accounts and auth	No	Nothing ever calls it manually

Read the right column twice. Every "No" is a bug you will not have to fix on Sunday night.

Step 1: Write the redaction gate first

The gate is the product. Everything else is plumbing around it.

# redact.py
import re

PATTERNS = [
    ("aws_key",   re.compile(r"AKIA[0-9A-Z]{16}")),
    ("gh_token",  re.compile(r"gh[pousr]_[A-Za-z0-9]{20,}")),
    ("bearer",    re.compile(r"(?i)bearer\s+[A-Za-z0-9._\-]{16,}")),
    ("jwt",       re.compile(r"eyJ[A-Za-z0-9_\-]{8,}\.[A-Za-z0-9_\-]{8,}\.[A-Za-z0-9_\-]{8,}")),
    ("email",     re.compile(r"[\w.+-]+@[\w-]+\.[\w.]+")),
    ("ipv4",      re.compile(r"\b(?:\d{1,3}\.){3}\d{1,3}\b")),
    ("long_hex",  re.compile(r"\b[0-9a-fA-F]{32,}\b")),
]

def redact(text: str) -> tuple[str, dict[str, int]]:
    counts: dict[str, int] = {}
    for name, rx in PATTERNS:
        text, n = rx.subn(f"<{name.upper()}>", text)
        if n:
            counts[name] = n
    return text, counts

def assert_clean(text: str) -> None:
    # Second pass must find nothing. Catches overlapping patterns.
    _, residual = redact(text)
    if residual:
        raise ValueError(f"redaction leaked: {residual}")

That second pass is the part people skip. Patterns overlap, and order matters.

Run the gate twice on the output. If the second run still matches, you have a bug, not a private log.

Step 2: Trim before you spend anything

A 400-line log is mostly noise. Keep the head, the tail, and every error-ish line.

# trim.py
import re

NOISE = re.compile(r"(?i)\b(error|exception|fail|fatal|timeout|denied|refused)\b")

def trim(log: str, head: int = 40, tail: int = 200) -> str:
    lines = log.splitlines()
    keep = set(range(min(head, len(lines))))
    keep |= set(range(max(0, len(lines) - tail), len(lines)))
    keep |= {i for i, l in enumerate(lines) if NOISE.search(l)}
    return "\n".join(lines[i] for i in sorted(keep))

Order matters here too. Trim first, then redact, then trim again if needed.

Trimming exposes one trade-off. You can cut the line that explains the failure.

So keep a 10-line window around every match. Do not keep only the matching line.

Step 3: One call, one strict contract

Ask for a fixed shape and refuse anything else. No prose parsing, no "best effort".

# ask.py
import json, os, urllib.request

PROMPT = """You triage CI logs. Return JSON only.
Fields: cause (string, <=300 chars), next_step (string, <=300 chars),
confidence (low|medium|high), quoted_evidence (0-3 strings copied
verbatim from the log). Do not guess. If unclear, use confidence=low."""

def call_model(log: str) -> dict:
    body = json.dumps({
        "model": os.environ["TRIAGE_MODEL"],
        "messages": [
            {"role": "system", "content": PROMPT},
            {"role": "user", "content": log},
        ],
        "temperature": 0,
    }).encode()
    req = urllib.request.Request(
        os.environ["TRIAGE_BASE_URL"].rstrip("/") + "/chat/completions",
        data=body,
        headers={
            "Content-Type": "application/json",
            "Authorization": "Bearer " + os.environ["TRIAGE_API_KEY"],
        },
    )
    with urllib.request.urlopen(req, timeout=60) as r:
        return json.loads(r.read())["choices"][0]["message"]["content"]

Model names change constantly, so never hardcode one. Read it from the environment.

Now validate the shape with a small hand-rolled check. Skip the heavy dependency.

# validate.py
ALLOWED = {"low", "medium", "high"}
REQUIRED = {"cause", "next_step", "confidence"}

def validate(obj: dict) -> dict:
    if not REQUIRED <= obj.keys():
        raise ValueError(f"missing: {REQUIRED - obj.keys()}")
    if obj["confidence"] not in ALLOWED:
        raise ValueError("bad confidence")
    for k in ("cause", "next_step"):
        if not isinstance(obj[k], str) or not obj[k].strip():
            raise ValueError(f"empty {k}")
        if len(obj[k]) > 300:
            raise ValueError(f"too long: {k}")
    ev = obj.get("quoted_evidence", [])
    if not isinstance(ev, list) or len(ev) > 3:
        raise ValueError("bad evidence")
    return obj

Any failure writes the raw output to failed/. You review those by hand on Monday.

Step 4: The test that needs no API key

This is the artifact that makes the weekend worth it. Record one real response, then test against it forever.

# test_pipeline.py
import json, pytest
from validate import validate
from redact import redact, assert_clean

def test_token_is_gone():
    out, counts = redact("auth: Bearer abcdef1234567890XYZ")
    assert "abcdef1234567890XYZ" not in out
    assert counts["bearer"] == 1

def test_second_pass_is_clean():
    out, _ = redact("host 10.0.3.41 token AKIAABCDEFGHIJKLMNOP")
    assert_clean(out)

def test_rejects_missing_field():
    with pytest.raises(ValueError):
        validate({"cause": "x", "confidence": "low"})

def test_rejects_bad_confidence():
    with pytest.raises(ValueError):
        validate({"cause": "x", "next_step": "y", "confidence": "sure"})

def test_fixture_still_valid():
    with open("fixtures/response.json") as f:
        validate(json.load(f))

Run pytest -q with the network off. If it passes, the contract holds.

The failing cases matter more than the passing one. Every bad model output becomes a permanent test.

Step 5: Put it on a small box and walk away

This tool has no users waiting. A cron entry is the correct deployment.

ssh you@your-box
git clone https://example.com/your/triage.git && cd triage
python3 -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt
mkdir -p inbox out failed fixtures
export TRIAGE_BASE_URL=... TRIAGE_MODEL=... TRIAGE_API_KEY=...

*/30 * * * * cd /srv/triage && .venv/bin/python run.py >> run.log 2>&1

run.py is fifteen lines. Read a file from inbox/, redact, trim, call, validate, write JSON to out/.

Add a flock around it. Cron does not care that the last run is still going.

*/30 * * * * flock -n /tmp/triage.lock cd /srv/triage && .venv/bin/python run.py >> run.log 2>&1

That one flag prevents the only outage this design can cause.

Where the free tier actually fits

Here is the honest split. The gate, the trimmer, and the tests need nothing from anyone. They run offline and they are the real work.

MonkeyCode is relevant for the two parts you would otherwise pay for. It offers free model access for the single call in Step 3, plus a free server option for the box in Step 5. As of this writing the operator advertises a 10,000,000-token allotment on the free tier. Treat that number as time-sensitive, not as a spec.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Why this pairing suits a weekend build: one call per log keeps usage boring and predictable. A log triage job on a small box is bursty and idle most of the day. You do not need a large instance, and you do not need a fat token budget.

Verify the current free-tier terms on the project page before you plan around them. Quotas, model lists, and server limits change, and this post will not.

What I deliberately skipped

Skipping is a design decision, so write it down. Otherwise you will add it back at 1am.

• No web UI. The output is a JSON file you read in your editor.
• No database. Files are easier to grep and easier to delete.
• No retry ladder. One retry, then quarantine to failed/.
• No streaming. The whole point is a small, bounded answer.
• No auth layer. Nothing calls this over HTTP.
• No prompt tuning loop. One prompt, versioned in git, changed only with a test.

That last skip is the hardest one. Prompt fiddling feels like progress and produces nothing you can verify.

Limits you should accept up front

The regex gate is not a guarantee. It catches shapes it knows and misses everything else.

A secret in an unusual format will pass through. So will a token split across two log lines.

Add a deny-list of your own hostnames and project names. That is the highest-value ten minutes you will spend.

Free tiers are best-effort by nature. Assume the box can restart and the quota can change.

Your cron job must survive an empty inbox/ and a 500 from the provider. Both are normal.

Who should not use this

Do not build this if any of the following is true for you.

• You handle regulated data, where a regex gate is not an acceptable control.
• You need per-user access control or an audit trail per request.
• You process thousands of logs a day, where a cron loop and flat files collapse.
• You need a guaranteed uptime number from your log triage tool.

In those cases, the redaction pattern is still useful. The weekend architecture is not.

Sunday night check

The demo is boring on purpose. Drop a log into inbox/, wait thirty minutes, read a three-field JSON file.

If the fields are wrong, you have a fixture to add. If the file is missing, read failed/.

That is the whole loop. Redact, trim, ask once, validate, quarantine the misses.

If you want to try it this weekend, the free model access and free server option cover the two paid parts. Everything that protects you runs locally, with no key at all.