- Book: LLM Observability Pocket Guide
- The series: Pocket Guides for Developers — 8 short books on agents, RAG, prompts, observability, events, databases and system design — the whole series
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
It's 3 p.m. and your agent just put back the bug you told it about before lunch. The session started at nine. Since then it has fixed a flaky test, renamed a module, chased a CSS bug and argued with you twice about a migration. The context meter says there is plenty of room left.
That room is the trap. Claude Code now runs its current models (Fable 5.1 and Fable 5, Sonnet 5 and later, Haiku 5.5, Opus 4.7 and later) with a 1 million token window by default. A window that large invites one session per day. Anthropic's own best-practices guide builds most of its advice on the opposite idea: "Claude's context window fills up fast, and performance degrades as it fills."
My take: treat the 1M window as a ceiling you should rarely get near. Give each session one sub-goal, carry state between sessions in files you wrote, and compact only when you have no better option.
What Chroma measured
In July 2025, Chroma published Context Rot, a technical report by Kelly Hong, Anton Troynikov and Jeff Huber. They ran 18 models from Anthropic, OpenAI, Google and Alibaba through tasks that should be easy at any length, such as finding one fact in a long text or copying a run of repeated words with one odd word in it. The summary line: "Across all experiments, model performance consistently degrades with increasing input length." Performance "varies significantly as input length changes, even on simple tasks."
Two of their results map straight onto a coding session.
The first is LongMemEval, a question-answering test over a long chat history. Chroma asked the same questions two ways: with a focused prompt of about 300 tokens that held only the relevant part, and with the full prompt of about 113k tokens. "Across all models, we see significantly higher performance on focused prompts compared to full prompts." The answer was in the input both times. The extra history was the difference. And 113k tokens is roughly a ninth of the window Claude Code now opens by default.
The second is distractors, text that looks like the answer but isn't. "Even a single distractor reduces performance relative to the baseline," the report says, and the effect grows as the input gets longer.
Be fair about the limits. The models in the report are the ones available in mid-2025, from GPT-3.5 Turbo up to Claude Opus 4, GPT-4.1, Gemini 2.5 Pro and Qwen3. Today's models weren't tested. I haven't seen a study showing the effect went away since, and Anthropic's current Claude Code docs still name it as the constraint the rest of the advice rests on.
Your all-day session is a haystack you built yourself
Map the study onto your day. The morning's work is the haystack. The approach you rejected at 10:40 is a distractor: code that looks like a solution, still in context, with a "no, not like that" somewhere after it. The afternoon task is the question, and it has little to do with anything before it.
That mapping is my reading. Chroma didn't test coding agents. But Anthropic's guide describes the same failure and gives it a name, "the kitchen sink session": "You start with one task, then ask Claude something unrelated, then go back to the first task. Context is full of irrelevant information." The fix it gives is /clear.
On repeated corrections the guide is blunter. After two failed corrections, clear and start over, because "a clean session with a better prompt almost always outperforms a long session with accumulated corrections."
It costs you, too. The context window page: "Old conversation crowds out the files you need next and costs tokens on every message."
One sub-goal per session
The working rule fits in four steps.
- Scope each session to one change. "Add the refund endpoint" is one. "The refund endpoint, then the CSS bug, then a look at that migration" is three sessions.
- Write the brief before you start. For bigger features, the Claude Code guide suggests having Claude interview you and write a spec to
SPEC.md, then: "start a fresh session to execute it. The new session has clean context focused entirely on implementation." - Close the session when the sub-goal is done, even with most of the window free.
- Before you close it, write down what the next session needs. Facts about this task go in the brief. Facts that hold for every task go in
AGENTS.md.
A brief can be short. This one fits on a screen:
# Task: refund endpoint for paid orders
## Goal
POST /orders/:id/refund returns 200 for a paid
order and 409 for any other status.
## Done when
- npm test passes
- a new test covers the 409 case
## Known from the last session
- Order statuses live in src/db.ts
- The payments client is mocked in test/helpers
## Out of scope
- Partial refunds
- Changes to the payments client
Claude Code takes a first prompt on the command line (CLI reference), so the brief can be the whole start of the session:
claude "$(cat briefs/refund.md)"
AGENTS.md is the shared file for the facts that outlive one task. agents.md lists Codex, Gemini CLI, Cursor, opencode, Aider, goose and many more as tools that read it. Codex builds its instruction chain "once per run," capped at 32 KiB by default (Codex docs). Claude Code reads AGENTS.md when there is no CLAUDE.md on the path, from v2.1.277 on. If you keep a CLAUDE.md, put an @AGENTS.md import in it (memory docs). Keep the file short: the test command, the conventions, the gotchas a fresh session would hit. Every session pays for it from its first token.
Compaction is a summary someone else wrote
Compaction is the alternative to a fresh session, and it is lossy by design. Claude Code's docs list what survives: the project-root CLAUDE.md and auto memory are re-injected from disk, the plan from plan mode comes back, and up to five recently modified files are re-read. Context that hooks added is summarized with the rest. The memory docs list three reasons a rule can vanish after compaction, and the first is that it was given only in conversation.
Anthropic's engineering post on context engineering names the risk: "overly aggressive compaction can result in the loss of subtle but critical context." The same post calls context "a finite resource with diminishing marginal returns."
A fresh session with a brief starts from what you chose to write down. A compacted one starts from what a summarizer decided mattered. If you have to compact, steer it. The docs show /compact <instructions>, for example:
/compact focus on the refund endpoint and its tests
If you want a forcing function, the model config docs let you cap sessions at a 200K window:
export CLAUDE_CODE_DISABLE_1M_CONTEXT=1
One catch from the same page: with auto-compaction on, the session compacts itself at the 200K boundary, and with it off the session stops there with a context-limit error. So the cap only works as a cue if you watch /context or a status line and end the sub-goal before you reach it.
Where Hermes fits
I build Hermes IDE, so weigh this section with that in mind. Everything above works in a plain terminal. Hermes makes it cheaper to follow.
Press ⌘N (Ctrl+Shift+N on Windows and Linux), paste the brief and press Enter. Per the README, the agent starts in its own terminal on a new hermes/<name> branch in its own worktree, with the task as its first prompt. The next sub-goal is another ⌘N. Ending one session and starting the next doesn't mean stashing, switching branches or finding a free terminal.
The ⌘N New task sheet: the task, the agent, the project, a new hermes/ worktree, and the command Hermes will run.
Hermes 2.3.4, released October 15, added a context meter that shows how full the conversation is and offers Compact when it gets close to full. Project memory and pinned files reach every later session's agent, which covers some of what a brief carries between sessions.
Each agent runs its own interface in a real terminal, so /clear and /compact work the way the vendor ships them. Hermes doesn't decide when a sub-goal is done or write the brief for you. Claude Code alone covers a lot of this well: /context, a custom status line for context use, and parallel sessions in worktrees (best-practices guide). Hermes is free, source-available under BSL 1.1, runs on macOS, Windows and Linux, and isn't affiliated with Nous Research, Anthropic, OpenAI or any agent vendor.
Claude Code's /context breakdown (44.4k of 1M tokens), a context gauge per session in the sidebar, and the token total with estimated cost.
When a long session is the right call
Anthropic's guide gives the counterpoint itself: "Sometimes you should let context accumulate because you're deep in one complex problem and the history is valuable." A long debugging hunt where every failed hypothesis is evidence is one sub-goal, and it can earn a long session. The rule is one sub-goal per session. The clock has nothing to do with it.
What doesn't earn it is a session that stays open because closing it feels like losing work. If the work is written down in the brief, the repo and AGENTS.md, closing the session loses nothing you need.
How long does your longest agent session run before you notice it getting worse?
If this was useful
If you want splitting work to cost one keystroke, Hermes IDE is free on macOS, Windows and Linux, with the source on GitHub.
A context meter is observability at the smallest scale: what went into the model, what it cost, and how quality moves as the input grows. The LLM Observability Pocket Guide is about doing that for a whole LLM system, with a short tour of the tracing and evals tools and how to pick the right ones for your team. It's one of 8 short books in the Pocket Guides series, on agents, RAG, prompts, observability, events, databases and system design: the whole series is here.
Top comments (0)