DEV Community

Christo
Christo

Posted on

Stop your Grok 4.5 agents from blowing past the context window

#ai #agents #langchain #typescript

Long-running agents have a boring failure mode: they accumulate conversation until they hit the model's context window and fall over. xAI ships a fix for Grok 4.5 — a context-compaction endpoint — but you have to call it yourself, on the right cadence, and feed the result back in a very specific way.

grok-loop-kit does that for you.

npm install grok-loop-kit     # Node (CJS + ESM + types)
pip install grok-loop-kit     # Python
Enter fullscreen mode Exit fullscreen mode

The loop 

import { GrokLoopClient } from 'grok-loop-kit';

const client = new GrokLoopClient(process.env.XAI_API_KEY!, {
  compactEvery: 8,          // compact every 8th turn...
  compactAtTokens: 8000,    // ...or when the rendered context crosses this
});

for (const turn of userTurns) {
  const res = await client.sendMessage(turn, tools);
  // res._grokLoopKit -> { turnsSinceCompact, estimatedTokensSaved, totalCompactions }
}
Enter fullscreen mode Exit fullscreen mode

Every sendMessage appends the user turn, calls POST /v1/responses, and — when the turn count hits a boundary or the rendered context crosses your token budget — calls POST /v1/responses/compact and rebuilds the transcript from the result.

The gotcha that actually matters

xAI's compaction endpoint returns an item you must feed back into the next request's input verbatim:

{ "type": "compaction", "id": "cmp_…", "encrypted_content": "…" }
Enter fullscreen mode Exit fullscreen mode

Not as a summarized user message — the exact item, unmodified. Get this wrong and you either lose context or corrupt the record. grok-loop-kit handles it (and still degrades gracefully against gateways/mocks that return a plain string).

Does compaction actually preserve information, or does it quietly forget? I tested it adversarially: plant five unguessable vault codes early, run enough turns to force four real compactions (dropping ~10 messages each), then demand exact recall.

✅ 5/5 exact codes recalled after 4 compactions
Enter fullscreen mode Exit fullscreen mode

Since the codes are random, the model can only answer if the compacted encrypted_content genuinely carried them forward. It did.

Native LangGraph

GrokLoopChatModel is a real BaseChatModel, so it drops straight into a LangGraph agent:

import { GrokLoopChatModel } from 'grok-loop-kit/langgraph';
import { createReactAgent } from '@langchain/langgraph/prebuilt';

const model = new GrokLoopChatModel({
  apiKey: process.env.XAI_API_KEY,
  grokLoop: { compactEvery: 8, compactAtTokens: 120_000 },
});

const agent = createReactAgent({ llm: model, tools: [getWeather] });
const out = await agent.invoke({ messages: [new HumanMessage('weather in Tokyo?')] });
Enter fullscreen mode Exit fullscreen mode

Because LangChain invokes a model with the full history each turn, the adapter compacts that history (a pure function of the incoming messages) rather than keeping a parallel transcript. Tool calls are parsed onto AIMessage.tool_calls. It's verified end-to-end against a live createReactAgent tool loop.

(One 1.0 lesson: extend BaseChatModel, not ChatOpenAI — on LangChain 1.x, ChatOpenAI.withConfig/bindTools clone into a plain ChatOpenAI and silently drop a subclass's _generate.)

Also in 1.0

  • Streaming (SSE) with a token callback
  • Retries with exponential backoff, honoring Retry-After
  • Per-call AbortSignal + request timeout
  • getState() / loadState() for durable, resumable agents
  • An AsyncGrokLoopClient for asyncio
  • 25 Node + 16 Python tests; MIT licensed

When does compaction help?

Compaction replaces the transcript with one dense record. That's a win once the raw transcript outweighs the record — i.e. long conversations. With tiny messages the record can cost more than it saves, so keep compactAtTokens realistic (thousands, not hundreds). It's a large-context optimization, and it keeps your agent inside the window indefinitely.

Repo: https://github.com/Booyaka101/grok-loop-kit · npm i grok-loop-kit

Built with the xAI context-compaction docs.

Top comments (0)