My Agent Forgot Everything Mid-Task: A Debugging Retrospective

Last Tuesday I watched an agent do something infuriating: it gathered requirements, confirmed a plan, and then completely ignored that plan during implementation. It asked the same clarifying question it had already answered, dropped a budget constraint it had repeated twice, and produced code that contradicted its own earlier decisions. The output looked fine in isolation, which made the failure harder to spot, and I spent the first hour assuming the model was simply bad. It wasn't. The model was fine; my context management was the problem, and the debugging trail that exposed it is worth sharing because the same trap will catch you too.

The first rule of debugging an agent is to stop trusting your memory and start trusting a transcript. I had been watching the agent live, which meant I only saw the last few turns, and my own mental model filled in the gaps the agent had already lost. So I added a simple logger that wrote every request and response to a JSONL file, including token counts and timestamps, and then I replayed the whole session from the beginning. That replay changed everything: the agent was running with a sliding window that kept only the most recent turns, and the turns where it had committed to the plan were long gone by the time implementation began.

To confirm the hypothesis I built a tiny reproduction harness that simulates exactly what a sliding-window truncation does to a conversation. The script takes a transcript, assigns each turn a token count, and drops the oldest turns whenever the cumulative total exceeds a budget, which is the same naive strategy many agent scaffolds use by default.

# retention_check.py
import json
import sys

def simulate_truncation(turns, token_budget=8000):
    kept = []
    used = 0
    for turn in turns:
        used += turn["tokens"]
        kept.append(turn)
        while used > token_budget and len(kept) > 1:
            dropped = kept.pop(0)
            used -= dropped["tokens"]
    return kept

def main(path):
    turns = json.load(open(path))
    kept = simulate_truncation(turns)
    key_facts = ["budget under $50", "Python", "Docker"]
    for fact in key_facts:
        survived = any(fact in t["text"] for t in kept)
        print(f"{fact}: {'survived' if survived else 'LOST'}")

if __name__ == "__main__":
    main(sys.argv[1])

Run it with python retention_check.py session.jsonl and you will see exactly what I saw: the budget constraint was lost three turns before implementation, the stack decision survived by luck, and the deployment choice was gone entirely. The reproduction took about ten minutes, and it turned a vague feeling that something was wrong into a precise, repeatable fact, which is the difference between guessing and debugging.

The fix was not a bigger window, because a bigger window only delays the same failure and makes every request more expensive. The fix was to compress decisions instead of discarding them: every few turns, a summarizer reads the full transcript and writes a short decision ledger that stays pinned in the context as a system message. The agent still receives the recent turns verbatim, but it also gets a stable record of what it already decided, so the budget constraint survives even when the original turn is dropped.

def compress_and_keep(turns, model, keep_recent=5):
    full_text = "\n".join(t["text"] for t in turns)
    summary = model.complete(
        "List only the confirmed decisions and constraints so far."
    )
    return [{"role": "system", "text": summary}] + turns[-keep_recent:]

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I ran this exact harness on MonkeyCode's free server using its free model access, and the experiment cost me nothing beyond the time to write the script. The free server meant I did not have to provision a virtual machine just to test a hypothesis, and the project advertises a free allowance of 10 million tokens, which covered dozens of reproduction runs without me watching a meter. That convenience matters for debugging because the whole point of a retrospective like this is to make failure cheap to reproduce, and a free tier lowers the barrier to running the experiment instead of reasoning about it in your head.

That said, there are real limits you should respect. Free server capacity and token allowances change, so check the repository for the current numbers before you build a workflow around them, and never point a production workload at a free tier that offers no SLA. The summarization fix also has its own failure mode: a summary can lose nuance, so you should keep recent turns verbatim and only compress the older ones, and you should log the summaries themselves so you can audit what the agent believes it decided. If your agent only runs for a few turns, none of this matters, and you should not add the complexity; if your agent runs long enough to forget, the ledger approach is the difference between a tool you can trust and a demo that works once.

The broader lesson is that AI-assisted debugging is still debugging: you form a hypothesis, you build a minimal reproduction, and you fix the root cause instead of the symptom. The only difference is that the component you are debugging is a context window, and the failure mode is silent memory loss rather than a crash. If you are hitting the same wall with a long-running agent, try the transcript replay first, then the retention check, and if you want a free place to run that experiment, MonkeyCode's server option is a reasonable starting point — just verify the current limits before you rely on them.