The refactor started clean. Twenty minutes later the same tool call that had succeeded at minute two was producing the same truncated JSON, with the same missing key, three calls in a row. I pasted the same task into a fresh session and it passed on the first attempt. The model had not degraded. The session had.
That distinction is easy to miss when you read AI-generated diffs all day. The model is the only moving part you can see, so it gets the blame. After a 48-hour run on a free coding server, I stopped trusting that default.
I ran these notes on MonkeyCode's free server tier while it offered free model access and a free hosted server option. I chose it because the server exports plain JSONL conversation history, which lets me inspect what actually entered each tool call instead of guessing from a chat window. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The goal was not to benchmark the server. The goal was a reproducible way to find why tool calls break in long sessions, on any server.
My working hypothesis was simple: if failure comes from context, then a fixed model will fail more often as a session gets older. I designed a small workload: twelve coding tasks, each needing five to nine tool calls, all run in one long session. The model stayed fixed for the whole run. Failures did not climb in a clean slope. They arrived in steps: quiet for a stretch, then two or three broken calls around the same boundary, then quiet again.
I kept looking at what changed at those boundaries, and three patterns came back again and again.
Entity drift. The first call received a value such as run_id = 'exp_104'. Twenty calls later a different value existed, but the pending tool description still carried an old copy. The model had not forgotten the new value. Both values were in history, and the stale one was simply closer to the active window. Cleaning that stale value out of the thread fixed the failure.
Repeated output noise. Some tool calls attached large identical metadata blocks every time they ran. The metadata was irrelevant, but it stayed in context and became visible filler. Past a certain amount of retained filler, the model started copying those blocks into its own arguments. Shortening the history removed the failure.
Truncated tool edges. When the server dropped old messages to fit the window, the visible part of an earlier tool schema got clipped. Optional parameters started disappearing from generated calls. No error message; the model simply produced weaker calls. That one is invisible unless you look at the full history export.
These are patterns from one long run, not a controlled benchmark. The useful part is the check, which takes about five minutes.
The check takes any JSONL conversation export where each line is one message and prints a small table: window index, approximate token volume, tool-call failures, and the maximum number of times a single message repeated inside the window. Run it on your own export.
#!/usr/bin/env python3
"""session_health.py: track tool-call failures against session age."""
import argparse
import json
import re
import sys
from collections import Counter
from pathlib import Path
FAIL_MARKERS = re.compile(
r'(fail(?:ed|ure)?|error|exception|invalid|unexpected|could not|missing)',
re.IGNORECASE,
)
def normalize(message: dict) -> str:
parts = [message.get('role', '?'), message.get('content') or '']
for call in message.get('tool_calls') or []:
parts.append(call.get('function', {}).get('name', ''))
parts.append(call.get('function', {}).get('arguments', ''))
return '\n'.join(parts)
def load(path: Path):
documents = []
for line in path.read_text().splitlines():
if not line.strip():
continue
try:
documents.append(json.loads(line))
except json.JSONDecodeError as exc:
print(f'warning: skipping malformed line: {exc}', file=sys.stderr)
return documents
def approximate_tokens(text: str) -> int:
return len(text.split())
def analyze(messages, window=12, step=6):
rows = []
for start in range(0, max(1, len(messages) - window + 1), step):
chunk = messages[start:start + window]
normalized = [normalize(message) for message in chunk]
joined = '\n'.join(normalized)
repeats = max(Counter(normalized).values(), default=1)
failures = sum(1 for text in normalized if FAIL_MARKERS.search(text))
rows.append({
'start_index': start,
'approx_tokens': approximate_tokens(joined),
'failures': failures,
'repeat_peak': repeats,
})
return rows
def main() -> int:
parser = argparse.ArgumentParser(
description='Check whether tool-call failures track session age.'
)
parser.add_argument('history', type=Path, help='JSONL conversation export')
parser.add_argument('--window', type=int, default=12)
parser.add_argument('--step', type=int, default=6)
args = parser.parse_args()
messages = load(args.history)
if not messages:
print('no messages found', file=sys.stderr)
return 1
print(f"{'start':>5} {'approx_tokens':>13} {'failures':>8} {'repeat_peak':>12}")
for row in analyze(messages, args.window, args.step):
print(
f"{row['start_index']:>5} {row['approx_tokens']:>13} "
f"{row['failures']:>8} {row['repeat_peak']:>12}"
)
return 0
if __name__ == '__main__':
raise SystemExit(main())
Example output from one of my sessions. Your numbers will differ; this shows the shape I kept seeing:
start approx_tokens failures repeat_peak
0 1,280 0 1
6 4,340 1 1
12 9,610 1 2
18 15,030 3 3
24 18,100 7 4
Read the table from top to bottom. If failures jump in steps while tokens grow, the session has reached an age where old values, repeated output, or clipped schema edges start to leak into new calls. If failures stay flat, the model or the code is a better suspect.
The fix I would repeat is boring: checkpoint at task boundaries. Start a new session and paste a tight summary of the previous one. Keep key facts and concrete return values in the summary; drop the full transcript. When a long session must continue, prune earlier tool outputs before the window overflows. If an important value was set twenty calls ago, write it explicitly into the new tool input instead of expecting the model to pull it from memory. After any pruning, check that optional parameters still appear in generated calls.
Limitations: this was 48 hours, one free tier, one fixed model, one small workload. Real differences between models exist, and I did not run separate control sessions for each pattern. If your failure distribution is flat instead of stepped, this article does not apply to your case.
Who should not use this: sessions short enough to finish in a few minutes do not need a health check. Production automation needs retries, timeouts, and schema validation; a session-age table does not replace any of those. And if you already have a concrete error message, debug that message instead of trying to match it to a pattern here.
The lesson I am keeping: instrument the export on day one and prune before the first step failure, not after. The cheapest way to try it is to export your next long session and run the script above.
MonkeyCode makes that rehearsal easy: the current offering includes free model access and a free hosted server option, and its history export drops straight into this JSONL check. If your failure table climbs in steps the way mine did, spend ten minutes pruning context before you spend a week auditioning models. That is the change I would repeat.
Top comments (0)