One night the free server logged an incident that resolved before my alarm went off. The next morning the free model that summarizes those logs handed me a root cause that matched nothing in the raw lines. That mismatch kicked off a 48-hour experiment in how far I could trust a cheap summarizer with a limited attention span.
The short version: the model wasn't lying, and it wasn't hallucinating in the usual sense. It was only ever seeing the tail of the story, and the longer my pipeline ran, the more confident it sounded about the part that survived. Here are the field notes: what I tried, what broke, and what I would repeat tomorrow.
What I tried: a nightly digest job that stays inside free options
I wanted an overnight report that stays inside free limits, and the setup was deliberately boring. A scheduled job on MonkeyCode's free server option pulls the day's log lines, trims them to fit a token budget, and calls the free model access it provides to explain the result. Disclosure: This article was prepared as part of MonkeyCode's product outreach.
The budget looked like a harmless detail in the first draft, and the budget turned out to be the whole product. My rule was simple: keep the newest lines, summarize the rest, and join the digest to the next day's context so history never scrolls away. Night one produced a plausible summary of the day's noise. Night two produced an intricate one, and intricate was the warning sign, because the real incident had happened hours earlier and the context could no longer see it.
What broke: the summaries started citing each other
By day two the digest wasn't reading logs anymore; it was reading yesterday's digest plus today's tail. That is the classic compression trick for a fixed window, and it worked in the narrow sense that the job always finished. It failed in the important sense that errors became assumptions: one wrong phrase in the old summary quietly became the premise for the next one.
After 40 hours I stopped trusting vibes and ran a control test on the same day of logs. Same model, same files, three different orderings of context — and three confident explanations of different incidents. Nothing had changed except which chunk of evidence had been evicted, and the model defended each answer with equal conviction. That's when I stopped calling it a model bug and started calling it a budgeting failure.
Why does a summarizer get more confident as it sees less? Because the remaining text is self-consistent, and nothing reminds the model what went missing.
The artifact: a digest pipeline that fails loudly
The fix is a two-pass digest. First, split raw logs into chunks that each fit a per-chunk budget. Then summarize each chunk, simulate the final context squeeze, and count how many early digests get evicted before the model ever sees them.
def estimate_tokens(text: str) -> int:
# Fallback heuristic; prefer the API's reported usage if your client has it.
return max(1, len(text) // 4)
def chunk_lines(lines: list[str], budget: int):
chunks, current, used = [], [], 0
for line in lines:
cost = estimate_tokens(line)
if current and used + cost > budget:
chunks.append(current)
current, used = [], 0
current.append(line)
used += cost
if current:
chunks.append(current)
return chunks
def two_pass_digest(lines, chunk_budget, final_budget, summarize):
chunks = chunk_lines(lines, chunk_budget)
digests = [summarize("\n".join(chunk)) for chunk in chunks]
kept, evicted, used = [], [], 0
for digest in reversed(digests):
cost = estimate_tokens(digest)
if kept and used + cost > final_budget:
evicted.append(digest)
continue
kept.append(digest)
used += cost
kept.reverse()
return {
"digest": "\n".join(kept),
"evicted_count": len(evicted),
"chunks_total": len(digests),
"warning": f"{len(evicted)} early digest(s) evicted" if evicted else None,
}
Plug in any summarize function; mine sends the chunk to the free model with a one-line instruction. The rule that saved me is short: never trust a digest with an evicted_count above zero, and make the job surface the number instead of hiding it.
if (run := two_pass_digest(lines, 6000, 4000, summarize_model))["evicted_count"]:
alert("early evidence dropped", run["warning"])
One caution: the chunk budgets above are example settings for my logs, not a product spec. Read your model's actual limit from the API response and size the budgets from that number.
What held, what broke, and what I'd repeat
- Budget discipline made reruns deterministic; the eviction counter turned a vague fear into a number I could alert on.
- My first version trusted the raw tail, and my second version trusted the summaries. Only the explicit counter caught the drift between the two.
- I would repeat the handoff line: every digest starts with the time window it actually covers, not the window I wished it covered.
- I would not chain summaries more than one level deep. A summary of a summary is where the free model got most creative with the facts.
Limitations and who should not use this
This workflow is triage, not forensics. If your postmortem feeds a compliance record or a customer report, keep the raw logs and the full pipeline, because a squeezed context is the wrong place for evidence you can lose.
Chunking by line order also breaks when services interleave, since two related events can land in different chunks. Filter by correlation ID first, or the ordering test will fail before the model ever gets a chance to.
The token heuristic is exactly wrong at the boundary where eviction starts, so replace it with real usage numbers whenever the client reports them. And be honest about who shouldn't use this: anyone who needs completeness, anyone who needs to reproduce a specific incident timeline, and anyone who will present the digest as proof. Use it to route attention, not to settle arguments.
I'm still running the job on MonkeyCode's free server option with the free model access it provides, mostly because the cost of leaving it on overnight is zero. The eviction counter is now the first line of every summary I read, and it's the one line I'd keep even if the model and the server changed next week.
Top comments (0)