DEV Community: Vilva Athiban P B

The complete 2 part series of How Claude code handles context/memory is out. https://dev.to/vilvaathibanpb/memory-management-in-claude-code-context-pipeline-1j1p

Vilva Athiban P B — Thu, 30 Apr 2026 14:49:33 +0000

Vilva Athiban P B

Apr 22

Memory management in Claude Code: Context Pipeline

#ai #claude #mcp #agents

Comments

8 min read

Memory management in Claude Code: Session Memory and Safe Compaction

Vilva Athiban P B — Sun, 26 Apr 2026 17:58:43 +0000

This is Part 2 of the series. If you have not read the first post yet, start with Part 1.

In Part 1, I looked at the first half of the memory pipeline:

slicing history after the last compact boundary
budgeting large tool results
using microcompact for cheap cleanup
triggering auto-compact with a policy instead of a last-second token check

That already solves a big part of the problem.

But once those cheaper layers are no longer enough, the system needs to do something more expensive without losing important working context, breaking message structure, or getting stuck in recovery loops.

This is where Claude Code’s design becomes especially useful. It treats compaction as a transaction, and it keeps a separate session-memory plane so overflow handling does not always start from scratch.

In this post, I will go through that second half:

how session memory is maintained
how full compaction works
why invariant protection matters more than it sounds
how recovery paths are kept bounded
what needs to be cleaned up after compaction

Two-part series

Part 1: Context Pipeline

Part 2: Session Memory and Safe Compaction

1) Keep session memory in parallel

A useful idea in Claude Code is that durable memory extraction does not begin at the moment the prompt overflows.

Instead, the system maintains a separate session-memory file in parallel and updates it only when it is actually worth doing.

That matters because summarization during an overflow path is expensive and time-sensitive. If you already have a distilled session memory available, compaction becomes cheaper and more predictable.

The extraction path is gated by thresholds such as:

initialize only after around 10k tokens
require at least 5k token growth between updates
require enough activity, such as 3 tool calls

The exact numbers are less important than the pattern. Session memory is updated opportunistically, not continuously.

Here is a TS sketch:

type Message = {
  id: string;
  token_count: number;
};

type SessionMemoryState = {
  content: string;
  last_update_tokens: number;
  last_summarized_message_id: string | null;
  extraction_started_at: number | null; // seconds since epoch
  extraction_task: Promise<void> | null;
};

const INIT_THRESHOLD_TOKENS = 10_000;
const GROWTH_THRESHOLD_TOKENS = 5_000;
const TOOL_CALL_THRESHOLD = 3;
const WAIT_TIMEOUT_SECONDS = 15;
const STALE_EXTRACTION_SECONDS = 60;

export function createSessionMemoryState(): SessionMemoryState {
  return {
    content: "",
    last_update_tokens: 0,
    last_summarized_message_id: null,
    extraction_started_at: null,
    extraction_task: null,
  };
}

export function shouldExtractSessionMemory(
  totalTokens: number,
  newTokensSinceLast: number,
  toolCalls: number
): boolean {
  if (totalTokens < INIT_THRESHOLD_TOKENS) {
    return false;
  }

  if (newTokensSinceLast < GROWTH_THRESHOLD_TOKENS) {
    return false;
  }

  if (toolCalls < TOOL_CALL_THRESHOLD) {
    return false;
  }

  return true;
}

function nowSeconds(): number {
  return Date.now() / 1000;
}

function withTimeout<T>(promise: Promise<T>, timeoutSeconds: number): Promise<T> {
  return new Promise<T>((resolve, reject) => {
    const timeoutId = setTimeout(() => {
      reject(new Error("Timeout"));
    }, timeoutSeconds * 1000);

    promise.then(
      (value) => {
        clearTimeout(timeoutId);
        resolve(value);
      },
      (error) => {
        clearTimeout(timeoutId);
        reject(error);
      }
    );
  });
}

export async function waitForInflightExtraction(
  state: SessionMemoryState
): Promise<void> {
  const task = state.extraction_task;

  if (!task) {
    return;
  }

  if (state.extraction_started_at !== null) {
    const age = nowSeconds() - state.extraction_started_at;
    if (age > STALE_EXTRACTION_SECONDS) {
      return;
    }
  }

  try {
    await withTimeout(task, WAIT_TIMEOUT_SECONDS);
  } catch {
    return;
  }
}

// Replace this with your actual durable-memory summarizer
function summarizeForDurableMemory(messages: Message[]): string {
  return `Summarized ${messages.length} messages`;
}

export async function updateSessionMemory(
  state: SessionMemoryState,
  messages: Message[]
): Promise<void> {
  const run = async (): Promise<void> => {
    state.content = summarizeForDurableMemory(messages);
    state.last_update_tokens = messages.reduce(
      (sum, m) => sum + (m.token_count || 0),
      0
    );

    if (messages.length > 0) {
      state.last_summarized_message_id = messages[messages.length - 1].id;
    }
  };

  state.extraction_started_at = nowSeconds();
  state.extraction_task = run();

  await state.extraction_task;
}

What this example shows is the separation of concerns.

background extraction handles durable knowledge distillation
compaction handles overflow when it happens

That split is one of the strongest parts of the design.

Why wait for in-flight extraction?

Before using session memory for compaction, Claude Code waits for a currently running extraction to finish, but only up to a limit.

That is a good practical choice.

It gives the system a chance to use a fresher memory snapshot without risking deadlock. If the extraction appears stale, it skips waiting and continues.

This is a small detail, but it shows the right mindset: use better context when available, but keep the runtime moving.

2) Treat compaction as a transaction

Once the system decides to compact, the easy thing to say is “summarize history.”

That is not enough.

A better way to think about compaction is as a transaction with clear phases:

prepare the compact input
generate the summary
clear state that is no longer valid
restore critical operational context
continue in the same turn

This is the part that makes the design production-grade.

A summary by itself rarely preserves everything the runtime needs. The agent may still need a small set of high-value context after compaction, such as:

currently relevant files
plan or plan mode state
invoked skills
deferred-tool deltas
agent listing deltas
MCP instruction deltas

That is why Claude Code rehydrates operational context after summarization, with strict budgets.

From the visible source snapshot, example restore budgets include:

max restored files: 5
max tokens per file: 5k
total file restore budget: 50k
max tokens per skill: 5k
total skills budget: 25k

Those limits matter. Rehydration without budgets will just recreate the original problem.

Here is a TypeScript sketch:

interface Message {
  id: string;
  role: "user" | "assistant";
  kind?: "text" | "tool_use" | "tool_result" | "boundary" | "summary";
  content: string;
  attachments?: string[];
}

interface ModelInfo {
  contextWindow: number;
  maxOutputTokens: number;
}

interface CompactionResult {
  summary: string;
  boundaryMessage: Message;
  restored: Message[];
}

async function fullCompaction(
  messages: Message[],
  model: ModelInfo
): Promise<CompactionResult | null> {
  const prepped = stripHeavyAttachments(messages);

  let summary: string | null = null;
  let working = [...prepped];

  for (let attempt = 0; attempt < 3; attempt++) {
    try {
      summary = await summarizeConversation(working, model);
      break;
    } catch (err) {
      if (!isPromptTooLong(err)) throw err;
      working = truncateOldestApiRounds(working);
    }
  }

  if (!summary) return null;

  clearReadCaches();
  clearNestedMemoryPathCaches();

  const restored = rehydrateCriticalContext(messages, {
    maxFiles: 5,
    maxTokensPerFile: 5_000,
    totalFileBudget: 50_000,
    maxTokensPerSkill: 5_000,
    totalSkillBudget: 25_000,
  });

  return {
    summary,
    boundaryMessage: {
      id: crypto.randomUUID(),
      role: "assistant",
      kind: "boundary",
      content: "[compact-boundary]",
    },
    restored,
  };
}

function buildPostCompactMessages(compacted: CompactionResult): Message[] {
  return [
    compacted.boundaryMessage,
    {
      id: crypto.randomUUID(),
      role: "assistant",
      kind: "summary",
      content: compacted.summary,
    },
    ...compacted.restored,
  ];
}

This code is useful for two reasons.

First, it makes compaction bounded. If the compact prompt is still too long, the system truncates oldest API rounds and retries only a limited number of times.

Second, it makes the restore step explicit. The compacted state is not just “smaller history.” It is “smaller history plus the operational context needed to continue.”

That is a much more reliable model.

3) Protect structure before saving tokens

This is probably the most important practical point in the whole design.

When context compression fails, the first failure is often structural, not semantic.

In other words, the problem is not always that the summary forgot something. The problem is that the resulting message set is invalid or inconsistent.

There are two invariants that matter a lot.

Tool-use and tool-result integrity

If a compaction boundary cuts through a tool interaction, the model can end up seeing a tool_result without the matching tool_use, or vice versa.

That breaks the structure of the conversation.

Assistant chunk integrity

A single assistant message may be represented as multiple chunks internally, such as thinking text and tool-use blocks that share the same assistant message ID. If compaction splits them incorrectly, the merged prompt becomes inconsistent.

Here is a TS sketch that adjusts a keep-start index so these invariants stay intact:

type Message = {
  kind?: string;
  tool_use_id?: string | null;
  assistant_message_id?: string | null;
};

export function adjustKeepStartIndex(
  messages: Message[],
  candidateIdx: number
): number {
  let idx = Math.max(0, candidateIdx);

  // Keep assistant chunk groups together
  if (idx < messages.length) {
    const currentGroup = messages[idx].assistant_message_id;

    while (
      idx > 0 &&
      currentGroup &&
      messages[idx - 1].assistant_message_id === currentGroup
    ) {
      idx -= 1;
    }
  }

  // Collect tool_result IDs after idx
  const toolResultIdsAfter = new Set<string>();
  for (let i = idx; i < messages.length; i++) {
    const m = messages[i];
    if (m.kind === "tool_result" && m.tool_use_id) {
      toolResultIdsAfter.add(m.tool_use_id);
    }
  }

  // Adjust boundary to avoid breaking invariants
  while (idx > 0) {
    const prev = messages[idx - 1];

    // Avoid orphaning tool results
    if (
      prev.kind === "tool_use" &&
      prev.tool_use_id &&
      toolResultIdsAfter.has(prev.tool_use_id)
    ) {
      idx -= 1;
      continue;
    }

    // Keep assistant chunks together
    const prevGroup = prev.assistant_message_id;
    if (
      prevGroup &&
      idx < messages.length &&
      messages[idx].assistant_message_id === prevGroup
    ) {
      idx -= 1;
      continue;
    }

    break;
  }

  return idx;
}

What this example shows is that compaction boundaries cannot be chosen by token count alone. They also need to respect structure.

That is why invariant protection should be treated as a first-class part of memory design, not as cleanup after the fact.

4) Recovery must stay bounded

Even with a good compaction pipeline, model calls can still fail.

Some examples are:

prompt-too-long
media-size failures
max-output-tokens

A common mistake is to handle these with open-ended retries. That usually makes incidents worse.

A better design is to use a bounded recovery ladder:

try the cheapest valid recovery
escalate in a limited way
stop after a hard ceiling
track repeated failures and trip a circuit breaker

Here is a TypeScript example:

async function retryWithRecoveryLadder(
  err: unknown,
  msgs: Message[],
  history: Message[],
  model: ModelInfo,
  state: { autoCompactFailures: number }
): Promise<Message[]> {
  if (isPromptTooLong(err)) {
    const compacted = await fullCompaction(msgs, model);
    if (compacted) {
      const retryMsgs = buildPostCompactMessages(compacted);
      const response = await callModel(retryMsgs, model);
      return appendToHistory(history, retryMsgs, response);
    }
  }

  if (isMediaTooLarge(err)) {
    const stripped = stripHeavyAttachments(msgs);
    const response = await callModel(stripped, model);
    return appendToHistory(history, stripped, response);
  }

  if (isMaxOutputTokens(err)) {
    const bumpedModel = {
      ...model,
      maxOutputTokens: Math.min(model.maxOutputTokens * 2, 8192),
    };
    const response = await callModel(msgs, bumpedModel);
    return appendToHistory(history, msgs, response);
  }

  state.autoCompactFailures += 1;
  throw err;
}

The idea is simple. Recovery paths should help the system escape bad states, not keep it stuck in them.

That is why the circuit breaker on repeated compaction failures matters too. If compaction keeps failing, the runtime should stop trying the same thing over and over.

5) Cleanup is part of correctness

Compaction changes the transcript, but it also invalidates derived state.

That means a compaction flow is not complete until the system clears or resets the state that no longer matches the new prompt.

In the visible Claude Code paths, post-compact cleanup resets things like:

microcompact state
system prompt sections
session message cache
approvals
some user and memory caches for main-thread compactions

At the same time, it intentionally preserves some invoked-skill content so future compactions can still use it.

That is a good reminder that cleanup is not just cache invalidation for performance. It is part of correctness.

Here is a small TypeScript sketch:

function runPostCompactCleanup(source: "user" | "compact" | "session_memory") {
  resetMicrocompactState();
  resetSystemPromptSections();
  clearSessionMessageCache();
  clearApprovals();

  if (source !== "compact") {
    clearUserCaches();
    clearMemoryCaches();
  }

  // Preserve whichever skill cache your runtime wants available
  // for future compaction passes.
}

If this step is missing, the agent can continue with ghost state that refers to a history shape that no longer exists.

That kind of bug is subtle and hard to debug later.

6) Measure the transformed prompt, not raw history

One detail I especially like in this design is that context diagnostics mirror the same transformations used before the model call.

That means a /context command should not look at raw stored history. It should look at:

messages after the compact boundary
with tool-result budgeting applied
with microcompact applied
with any context-collapse projection applied

That way, the numbers shown to operators are the numbers that actually matter.

Here is a TS sketch:

type Message = {
  token_count?: number;
};

type Model = {
  context_window: number;
  max_output_tokens: number;
};

type State = unknown; // adjust based on your actual state shape

// Assume these exist in your pipeline
declare function getMessagesAfterCompactBoundary(history: Message[]): Message[];
declare function simulateToolResultBudget(msgs: Message[], state: State): Message[];
declare function maybeHistorySnip(msgs: Message[]): Message[];
declare function microcompact(msgs: Message[]): Message[];
declare function maybeContextCollapseProjection(msgs: Message[]): Message[];

export function contextDiagnostics(
  history: Message[],
  model: Model,
  state: State
) {
  let msgs = getMessagesAfterCompactBoundary(history);
  msgs = simulateToolResultBudget(msgs, state);
  msgs = maybeHistorySnip(msgs);
  msgs = microcompact(msgs);
  msgs = maybeContextCollapseProjection(msgs);

  const totalTokens = msgs.reduce(
    (sum, m) => sum + (m.token_count || 0),
    0
  );

  const effectiveContext =
    model.context_window - Math.min(model.max_output_tokens, 20_000);

  const threshold = effectiveContext - 13_000;

  return {
    input_tokens: totalTokens,
    auto_compact_threshold: threshold,
    remaining_before_compact: Math.max(0, threshold - totalTokens),
    message_count: msgs.length,
  };
}

This is a small implementation choice, but it improves debugging immediately because the reported state matches the model-facing prompt.

7) The reusable blueprint

At a high level, the full control plane looks like this:

async function runQueryTurn(history: Message[]) {
  let msgs = messagesAfterLastCompactBoundary(history);
  msgs = enforceToolResultBudget(msgs);
  msgs = optionalHistorySnip(msgs);
  msgs = microcompact(msgs);
  msgs = optionalContextCollapseProjection(msgs);

  if (shouldAutoCompact(msgs)) {
    const compacted =
      trySessionMemoryCompaction(msgs) || fullCompaction(msgs);

    if (compacted) {
      msgs = buildPostCompactMessages(compacted);
      runPostCompactCleanup();
    }
  }

  let response = await callModel(msgs);

  if (recoverableOverflow(response)) {
    response = await retryWithRecoveryLadder();
  }

  return append(
    history,
    msgs,
    response,
    getToolResults(response),
    getAttachments(response)
  );
}

This is why the most transferable lesson from Claude Code is architectural.

Memory efficiency in agents is not one algorithm. It is a coordinated system made of:

staged context reduction
stable replacement decisions
asynchronous durable memory extraction
transaction-style compaction
invariant-preserving boundaries
bounded recovery
observability based on the real prompt view

That is what makes long sessions manageable.

Closing

If you only implement summarization, your agent may work for a while.

If you implement a layered memory control plane — cheap reductions first, durable memory in parallel, compaction as a transaction, and bounded recovery — the agent has a much better chance of staying stable over long sessions.

That is the real lesson here.

Not better summaries.

Better memory architecture.

Read Part 1: Context Pipeline

Memory management in Claude Code: Context Pipeline

Vilva Athiban P B — Wed, 22 Apr 2026 14:43:03 +0000

If you are building an AI agent that needs to survive longer sessions, context management becomes a systems problem very quickly.

At first, the implementation looks simple. You append messages, keep tool outputs around, maybe summarize occasionally, and keep going. But once the session becomes longer, the same problems start showing up: prompts get too large, tool outputs dominate the context, cache reuse drops, and retries start making the situation worse.

That is why Claude Code’s memory design is interesting. The useful part is not one summarization trick. It is the layered control system around context.

This post covers the first half of that system:

how the model-facing prompt is built on each turn
why tool-result budgeting happens early
where microcompact fits in
how auto-compact is triggered safely

This is Part 1 of a two-part series. In Part 2(Coming soon), I will look at session memory, full compaction, invariant protection, cleanup, and bounded recovery.

Two-part series

Part 1: Context Pipeline

Part 2: Session Memory and Safe Compaction

Why context becomes a systems problem

A long-running coding agent keeps collecting context from different places:

user and assistant messages
tool calls
large tool outputs
file attachments
memory files and instruction overlays
subagent and MCP state

The mistake is to treat all of this as one flat history buffer.

It is not.

A recent plan, an active file, and a large terminal output from twelve turns ago should not be treated the same way. If the system has only one strategy — keep appending and summarize later — it will eventually hit one of these failure modes:

prompt-too-long loops
broken tool_use / tool_result structure
duplicate reinjection of context
poor prompt-cache reuse
brittle resume behavior

So the real problem is not just “how do I summarize history?”

The real problem is:

How do I keep the prompt useful over long sessions without breaking message invariants, hurting cache stability, or making recovery worse?

That is the part Claude Code gets right.

1) Start with a layered pipeline

The implementation uses a layered memory pipeline.

Instead of jumping directly to summarization, it reduces context in stages:

keep only the relevant slice of history
shrink large tool results
do lightweight cleanup
compact only when cheaper steps are not enough

That ordering matters. Summarization is the expensive path, so it should be the fallback, not the default.

A simplified version of the per-turn flow looks like this:

slice messages after the last compact boundary
apply tool-result budget
optionally snip history
run microcompact
optionally apply context-collapse projection
check auto-compact
call the model
use bounded recovery if the call still fails

This is the shape that makes the whole system stable.

2) The per-turn context pipeline

Here is a generic TypeScript version of that control flow.

type QuerySource = "user" | "compact" | "session_memory";

interface Message {
  id: string;
  role: "user" | "assistant";
  kind?: "text" | "tool_use" | "tool_result" | "boundary" | "summary";
  content: string;
  toolUseId?: string;
  assistantMessageId?: string;
  tokenCount?: number;
  timestampMs?: number;
}

interface ModelInfo {
  contextWindow: number;
  maxOutputTokens: number;
}

interface RuntimeState {
  autoCompactFailures: number;
  userConfig: { autoCompactEnabled: boolean };
  env: {
    disableCompact: boolean;
    disableAutoCompact: boolean;
  };
  toolBudgetState: {
    seenIds: Set<string>;
    replacements: Map<string, string>;
  };
}

async function runQueryTurn(
  history: Message[],
  model: ModelInfo,
  source: QuerySource,
  state: RuntimeState
): Promise<Message[]> {
  let msgs = getMessagesAfterCompactBoundary(history);

  msgs = await applyToolResultBudget(msgs, state.toolBudgetState);
  msgs = maybeHistorySnip(msgs);
  msgs = microcompact(msgs);
  msgs = maybeContextCollapseProjection(msgs);

  if (shouldAutoCompact(msgs, model, source, state)) {
    const compacted =
      (await trySessionMemoryCompaction(msgs, state, model)) ??
      (await fullCompaction(msgs, state, model));

    if (compacted) {
      msgs = buildPostCompactMessages(compacted);
      runPostCompactCleanup(state, source);
    } else {
      state.autoCompactFailures += 1;
    }
  }

  try {
    const response = await callModel(msgs, model);
    return appendToHistory(history, msgs, response);
  } catch (err) {
    return retryWithRecoveryLadder(err, msgs, history, model, state);
  }
}

The important thing here is not the exact function names. It is the order.

The model does not receive raw history. It receives a transformed view of the history. And that transformed view is built in stages.

That is also why observability should mirror this pipeline. If your /context command looks at raw history but the model sees a compacted version of it, your debugging numbers will not match reality.

3) Budget tool results before anything expensive

The cheapest and most effective reduction is usually tool output.

In long coding sessions, tool results are often the biggest tokens in the prompt:

terminal output
search results
file diffs
generated logs
structured JSON returned by tools

The system usually does not need the full payload forever. It needs enough inline context for the model to stay grounded, and a durable reference to the full content outside the prompt.

This is where tool-result budgeting comes in.

A good implementation should do three things:

persist the full output somewhere durable
replace the in-context payload with a stable preview
replay the same replacement text on later turns

That third point matters for prompt caching. If the replacement text changes from turn to turn, the cache key changes too. So once a tool result has been replaced, the decision should be frozen by tool_use_id.

Here is a generic TypeScript version:

interface ToolStorage {
  store(toolUseId: string, content: string): Promise<string>;
}

interface BudgetState {
  seenIds: Set<string>;
  replacements: Map<string, string>;
}

const MAX_TOOL_RESULT_BYTES_PER_MESSAGE = 64_000;

function buildPreview(ref: string, original: string): string {
  return [
    `Tool result stored externally: ${ref}`,
    ``,
    `Preview:`,
    original.slice(0, 500),
    ``,
    `[truncated for context budget]`,
  ].join("\n");
}

async function applyToolResultBudget(
  messages: Message[],
  state: BudgetState,
  storage?: ToolStorage
): Promise<Message[]> {
  return Promise.all(
    messages.map(async (msg) => {
      if (msg.kind !== "tool_result" || !msg.toolUseId) return msg;

      const bytes = Buffer.byteLength(msg.content, "utf8");
      if (bytes <= MAX_TOOL_RESULT_BYTES_PER_MESSAGE) return msg;

      if (state.seenIds.has(msg.toolUseId)) {
        const replacement = state.replacements.get(msg.toolUseId);
        return replacement ? { ...msg, content: replacement } : msg;
      }

      const ref = storage
        ? await storage.store(msg.toolUseId, msg.content)
        : `tool-result://${msg.toolUseId}`;

      const replacement = buildPreview(ref, msg.content);

      state.seenIds.add(msg.toolUseId);
      state.replacements.set(msg.toolUseId, replacement);

      return { ...msg, content: replacement };
    })
  );
}

What this example shows is simple: large tool outputs are converted into small, stable prompt payloads before the system reaches full compaction.

That is a practical optimization, and it pays off early.

Why budget at the API message level?

One useful production detail from Claude Code is that budgeting is applied at the merged API-message level, not just on raw internal message objects.

Why does that matter?

Because tool outputs can arrive in parallel. If you budget each internal fragment separately, several large results can slip through simply because they are split across message objects. Budgeting at the merged message level closes that gap.

That is the kind of detail that usually only becomes obvious once you run real sessions at scale.

4) Use microcompact for cheap cleanup

So what happens after tool-result budgeting?

You still need a lightweight cleanup path before full compaction. That is what microcompact is for.

Claude Code uses two useful ideas here.

Cached microcompact

One path tracks compactable tool_result blocks and edits them in a cache-aware way instead of eagerly rewriting everything locally.

The point is not only to remove tokens. It is also to preserve cache reuse as much as possible.

Time-based microcompact

There is also a time-based path for idle sessions.

If enough time has passed since the last assistant message, older tool results become cheaper to remove because the cache may already be stale. In that case, the system can keep only the most recent few tool results and clear the rest.

This is a useful production idea because not all context pressure is token pressure. Some of it is cache-lifecycle pressure.

Here is a minimal version of a time-based microcompact pass:

type Message = {
  id: string;
  role: string;
  kind?: string;
  content: string;
  timestamp_ms?: number;
};

const IDLE_THRESHOLD_MS = 60 * 60 * 1000;
const KEEP_RECENT_TOOL_RESULTS = 5;

export function microcompact(
  messages: Message[],
  nowMs: number = Date.now()
): Message[] {
  // Find last assistant message (from end)
  const lastAssistant = [...messages]
    .reverse()
    .find((m) => m.role === "assistant");

  if (!lastAssistant || !lastAssistant.timestamp_ms) {
    return messages;
  }

  const idleTooLong =
    nowMs - lastAssistant.timestamp_ms > IDLE_THRESHOLD_MS;

  if (!idleTooLong) {
    return messages;
  }

  // Get all tool_result messages
  const toolResults = messages.filter(
    (m) => m.kind === "tool_result"
  );

  // Keep only last N tool results
  const keepIds = new Set(
    toolResults.slice(-KEEP_RECENT_TOOL_RESULTS).map((m) => m.id)
  );

  // Build compacted messages
  return messages.map((msg) => {
    if (msg.kind === "tool_result" && !keepIds.has(msg.id)) {
      return {
        ...msg,
        content:
          "[microcompacted: old tool result removed after idle period]",
      };
    }
    return msg;
  });
}

This example is intentionally simple. It is not trying to summarize anything. It is just removing stale low-value content with very low latency.

That is exactly where microcompact fits.

5) Auto-compact is a policy decision

Eventually the lightweight paths are not enough. At that point the system needs to decide whether to run a full compaction.

The common mistake is to trigger compaction too late:

if used_tokens > context_window:
    summarize()

That looks fine in a toy implementation. In production it is usually too late.

Claude Code uses a more realistic threshold model.

First, it computes an effective context window:

effectiveContext = contextWindow - min(modelMaxOutput, 20k reserve)

This makes room for the response instead of pretending the whole context window is available for input.

Then it applies an additional buffer:

autoCompactThreshold = effectiveContext - 13k buffer

That extra buffer matters. Compaction itself needs space. Recovery paths also need space. Waiting until the last possible turn is usually a bad policy.

Here is a simple version of the same logic:

type Message = {
  token_count: number;
};

type Model = {
  context_window: number;
  max_output_tokens: number;
};

type State = {
  disable_compact: boolean;
  disable_auto_compact: boolean;
  auto_compact_enabled: boolean;
  auto_compact_failures: number;
};

export function effectiveContextWindow(
  contextWindow: number,
  modelMaxOutput: number
): number {
  const reserve = Math.min(modelMaxOutput, 20_000);
  return contextWindow - reserve;
}

export function autoCompactThreshold(
  contextWindow: number,
  modelMaxOutput: number
): number {
  return effectiveContextWindow(contextWindow, modelMaxOutput) - 13_000;
}

export function shouldAutoCompact(
  messages: Message[],
  model: Model,
  querySource: string,
  state: State
): boolean {
  if (state.disable_compact || state.disable_auto_compact) {
    return false;
  }

  if (!state.auto_compact_enabled) {
    return false;
  }

  if (querySource === "compact" || querySource === "session_memory") {
    return false;
  }

  if (state.auto_compact_failures >= 3) {
    return false;
  }

  const totalTokens = messages.reduce(
    (sum, m) => sum + (m.token_count || 0),
    0
  );

  const threshold = autoCompactThreshold(
    model.context_window,
    model.max_output_tokens
  );

  return totalTokens >= threshold;
}

What this example shows is that compaction is a policy decision, not just a token-count check.

The policy is guarded by:

environment flags
user config
query source
a circuit breaker for repeated failures

That is the right way to think about it. Once compaction becomes part of the runtime, it needs the same safety rails as any other production control path.

6) Why this layering works

At this point the overall structure should be clear:

tool-result budgeting handles the biggest cheap wins
microcompact handles lightweight cleanup
auto-compact is the expensive fallback

This is what layered degradation means in practice.

Instead of going from “everything is in the prompt” to “summarize history,” the system has intermediate steps that are cheaper, safer, and easier to reason about.

That matters for two reasons.

First, it keeps latency lower in the common case.

Second, it reduces the chances of structural mistakes because the system does not compact more than it needs to.

7) In Part 2

Part 1 focused on the normal turn pipeline and the cheap reductions that happen before summarization.

In Part 2, I will look at the second half of the architecture:

session memory as a separate durable memory plane
why full compaction should be treated as a transaction
how structural invariants are preserved
how bounded recovery avoids retry spirals
why post-compact cleanup and observability matter

That is where the design goes from prompt management to stable long-session behavior.

Follow Vilva Athiban for more such production-grade AI/Agentic content

Read next: Session Memory and Safe Compaction

How Claude Code Uses React in the Terminal

Vilva Athiban P B — Wed, 15 Apr 2026 04:01:01 +0000

React handles reconciliation. A custom renderer handles layout, screen buffers, diffing, and high-FPS terminal updates.

Most React engineers know React through the DOM.

You write components, React reconciles them, and the browser takes care of layout, painting, input, and rendering. That flow is so familiar that it is easy to think of React as a browser UI library.

Claude Code’s terminal renderer is a good reminder that React is really a reconciliation engine.

In this setup, React is not rendering to the DOM. It is rendering into a custom terminal host with its own layout engine, screen buffer, diff pipeline, and terminal patch writer. The interesting part is not that React works in a terminal. The interesting part is how much renderer infrastructure is required to make that terminal UI feel fast, stable, and predictable.

That is what makes this design worth studying. It is not a novelty renderer. It is a rendering engine built around React.

React handles reconciliation. The renderer handles the rest.

At the top, this is still a React renderer built with react-reconciler.

React owns the usual things: state updates, component lifecycle, effects, and reconciliation. But instead of committing into DOM nodes, it commits into terminal-native host nodes.

From there, the custom renderer takes over.

The pipeline looks like this:

React commit
  -> mutate terminal host nodes
  -> run layout with Yoga
  -> paint into a screen buffer
  -> diff against the previous frame
  -> compile terminal patches
  -> flush a single buffered terminal write

That separation is the main architectural idea.

React gives the codebase a clean component model and update system. The renderer keeps control over layout cost, paint cost, write volume, and output correctness. In a browser, the platform hides most of that. In a terminal, the renderer has to own it directly.

The host node model is small on purpose

The renderer does not try to recreate the browser.

Its host elements are terminal-specific primitives such as ink-root, ink-box, ink-text, and ink-raw-ansi. That already tells you something important: this renderer is designed around the terminal’s constraints, not around DOM compatibility.

The node model is also selective about where work happens.

Not every node gets layout. Text and raw ANSI nodes use custom measurement paths. Event handlers are stored separately from normal visual attributes. Dirty flags are explicit and propagate upward when something visually relevant changes.

That separation is doing real work.

A handler identity change should not trigger repaint logic. It changes behavior, not output. If the renderer mixes handler updates into visual invalidation, it destroys useful fast paths. By keeping those concerns separate, it avoids unnecessary work and preserves reuse opportunities.

This is the kind of design choice that matters a lot once frame cost starts to matter.

Terminal input is mapped into React scheduling priorities

One of the strongest ideas in this renderer is the event-priority bridge.

Terminal events are classified before they reach React scheduling:

Discrete   -> keydown, click, focus, paste
Continuous -> resize, scroll, mousemove
Default    -> everything else

The host config then exposes that classification through React 19 scheduling hooks such as update-priority resolution and event-type lookup.

This is more important than it looks.

In a terminal UI, keyboard input usually needs immediate response. Resize and scroll events can arrive in bursts. Mouse movement can be noisy. If all of those are treated the same way, responsiveness starts to fall apart.

The smart part is not just that events are handled. It is that input semantics are translated into scheduler semantics.

That is why keyboard interactions can stay responsive while high-frequency event streams remain stable.

Render timing is chosen carefully

Terminal UIs are very sensitive to timing mistakes.

If rendering happens too early in the lifecycle, you can get one-frame lag bugs around cursor movement, focus visuals, caret placement, or layout-dependent output. These bugs are small, but they are immediately noticeable in a terminal.

This renderer avoids that by deferring rendering to a microtask after commit, so layout effects have already run.

At the same time, it still uses synchronous container updates where deterministic timing matters.

That hybrid model is a good fit for terminal rendering. React keeps its modern reconciliation model, but the renderer still gets tightly controlled output timing. The result is better correctness without giving up throughput.

This is one of those details that sounds subtle until you have to debug a renderer that paints one frame too early.

The screen buffer is built for hot loops

A terminal renderer spends a lot of time doing repetitive, low-level work. That makes data layout important.

This renderer does not use an object-per-cell screen model. Instead, it uses packed typed arrays. The screen stores character, style, hyperlink, and width data in Int32Array words, with a BigInt64Array view used for fast bulk clears and fills.

On top of that, it uses intern pools to avoid repeated allocations and repeated string work:

CharPool
HyperlinkPool
StylePool

The StylePool is especially interesting because it caches transition strings between style IDs. That means style changes do not have to repeatedly compute ANSI transitions. They become a cached lookup plus an append.

That is exactly the kind of optimization terminal renderers need.

In browser React, style updates usually stop at DOM mutation and browser paint. In a terminal renderer, style changes become terminal byte sequences. That makes string reuse and packed memory layout much more important.

Diffing is damage-aware instead of full-screen by default

A naive terminal renderer can redraw the full screen on every frame. That works for small demos and falls apart once updates become frequent.

This renderer tracks damage bounds for each frame and focuses diff work on changed regions instead of brute-forcing the full viewport every time. It also unions relevant previous damage so it can reason about what actually needs attention.

That already saves work, but the more interesting part is that it also uses blitting.

If a subtree is clean and its layout has not changed, the renderer can copy cells from the previous frame instead of re-rendering text. That is much cheaper than repainting the subtree from scratch.

Just as importantly, the renderer does not trust blitting blindly.

There are explicit guards that disable blit paths when overlays, removals, or absolute positioning could cause stale cells to be restored. That balance is what makes the system robust: reuse previous work aggressively, but only where correctness still holds.

That is a strong pattern in any renderer. Cache when it is safe. Recompute when it is not.

Scroll performance uses terminal-native acceleration

Scrolling is another place where terminal renderers can either be efficient or wasteful.

Instead of rewriting an entire scroll region every tick, this renderer can use terminal-native scroll operations such as DECSTBM, then patch only the rows that become newly exposed.

That is dramatically cheaper than repainting the full region.

But once again, the fast path is capability-aware.

If synchronized output is unreliable, or if atomicity is not guaranteed in the current environment, the renderer can fall back to safer diff-based behavior to avoid visible intermediate artifacts.

This is a good production habit. Fast paths should be gated by runtime capability detection, not assumed globally.

Terminals vary too much for anything else to be reliable.

Long-running sessions need strict lifecycle control

A renderer like this is not judged only by how it behaves in the first few seconds.

It has to stay correct over long sessions, with repeated updates, mounts, unmounts, focus changes, resizes, and scrolling. That is where weak cleanup turns into real runtime drift.

This codebase is careful about that.

Yoga references are cleared before frees. Removed nodes trigger focus cleanup. Pool growth is controlled with resets and ID migration. Terminal modes are restored synchronously on unmount and exit.

That is not background maintenance. It is part of the renderer design.

For long-running terminal applications, cleanup discipline is directly tied to correctness and frame consistency. Without it, performance slowly degrades and bugs become much harder to reason about.

What is worth copying from this design

Even if you are not building a terminal UI, there are solid engineering lessons here.

1. Build an explicit bridge from input to scheduling

Not all events should enter the system with the same urgency. If the runtime supports prioritization, use it intentionally.

2. Keep non-visual updates out of visual invalidation paths

Behavior changes and render changes are different kinds of work. Treating them the same creates unnecessary paint cost.

3. Optimize the data model, not just the algorithm

Typed-array buffers and intern pools matter because they remove cost from the hottest rendering loops.

4. Track damage and reuse work carefully

Damage-aware diffing plus selective blitting is a powerful combination, as long as the correctness checks stay strict.

5. Gate fast paths by runtime capability

An optimization is only good if it is safe in the environment where it runs.

6. Treat teardown as part of the architecture

Cleanup, resource release, and terminal mode restoration are part of renderer correctness, not afterthoughts.

The bigger takeaway

The interesting thing about Claude Code’s terminal UI is not just that it uses React.

It is that React is only one layer in a larger rendering system.

React handles reconciliation and the component model. The renderer handles layout, painting, diffing, scheduling, memory behavior, terminal patch generation, and output correctness. Once you look at it that way, this stops being "React in a terminal" and starts looking like what it really is: a custom rendering engine built around React.

That is the real lesson here.

When the platform is constrained, the renderer matters more than the framework. And when that renderer is designed well, React can be a very good fit far outside the browser.

Follow https://www.linkedin.com/in/vilvaathiban/ for more such technical content