Agent Memory in TypeScript: Short-Term, Long-Term, and What to Throw Away

#ai #typescript #agents #node

Book: AI That Plans
The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here
My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
Me: xgabriel.com | GitHub

"Agent memory" gets implemented as one array of messages that grows
until something truncates it. That single array is doing three jobs
that have different lifetimes, different access patterns, and
different failure modes — which is why the truncation always feels
arbitrary.

Separate them and the eviction question answers itself.

Three stores, three lifetimes

Working memory is the message window sent on this turn. Lifetime:
one run. Bounded by the context limit. It is the only one the model
sees directly.

Long-term memory is durable facts about the user or the domain.
Lifetime: indefinite. Not sent wholesale — retrieved selectively.
"Prefers metric units", "primary account is acct_x", "is on the
enterprise plan".

Episodic memory is the record of previous runs. Lifetime: audit
retention. Almost never sent to the model. Used for debugging,
analytics, and evals.

Conflating the first two produces the common failure: a fact the user
stated in message three is still in the window at message forty
because someone was afraid to drop it, and it is crowding out the
last ten turns that actually matter.

export interface WorkingMemory {
  window(): MessageParam[];
  append(m: MessageParam): void;
  compact(budget: number): Promise<void>;
}

export interface LongTermMemory {
  recall(query: string, k: number): Promise<Fact[]>;
  remember(f: NewFact): Promise<void>;
  forget(id: string): Promise<void>;
}

export interface EpisodicMemory {
  record(run: RunRecord): Promise<void>;
  find(filter: RunFilter): Promise<RunRecord[]>;
}

Three interfaces with almost nothing in common. That is the point —
a single Memory interface with get/set is how the three get
conflated in the first place.

Working memory: the window and its budget

export class Window implements WorkingMemory {
  private msgs: MessageParam[] = [];
  private summary: string | null = null;

  constructor(private readonly system: string) {}

  window(): MessageParam[] {
    const head: MessageParam[] = this.summary
      ? [{ role: "user", content: `Earlier context:\n${this.summary}` }]
      : [];
    return [...head, ...this.msgs];
  }

  append(m: MessageParam) {
    this.msgs.push(m);
  }
}

The summary sits at the front as a synthetic turn. Everything after
it is verbatim recent history.

Compaction is where the design decision lives:

async compact(budget: number) {
  if (estimateTokens(this.window()) <= budget) return;

  const keepRecent = 6;
  const older = this.msgs.slice(0, -keepRecent);
  const recent = this.msgs.slice(-keepRecent);
  if (!older.length) return;

  const res = await model.invoke([
    { role: "user", content: SUMMARISE_PROMPT(this.summary, older) },
  ]);

  this.summary = textOf(res.content);
  this.msgs = recent;
}

Two properties matter more than the summarisation prompt.

Keep the recent turns verbatim. Summarising the last exchange
loses exactly the detail the next turn depends on. Compress the
distant past, never the present.

Never split a tool pair. A tool_use block whose matching
tool_result was evicted produces a malformed request. Whatever your
eviction rule, it operates on complete exchanges:

function safeCut(msgs: MessageParam[], from: number): number {
  while (from < msgs.length && isOrphanedToolResult(msgs[from])) {
    from++;
  }
  return from;
}

That guard is one of the more common causes of a mysterious 400 from
the API after a long conversation.

What to summarise for

A generic "summarise the conversation" prompt produces prose that
reads well and loses what the agent needs. Ask for the operational
residue instead:

const SUMMARISE_PROMPT = (prev: string | null, msgs: MessageParam[]) => `
${prev ? `Previous summary:\n${prev}\n\n` : ""}
Summarise the exchange below. Preserve, as a list:
- decisions made and their stated reasons
- identifiers, values, and constraints the user supplied
- tools already called and what they returned
- anything the user explicitly asked to remember
Omit pleasantries and restatements. Under 200 words.

${render(msgs)}
`;

Feeding the previous summary back in is what stops the tenth
compaction from being a summary of a summary of a summary. Each pass
revises one document rather than re-compressing its own output.

Long-term memory is written, not scraped

The tempting implementation runs after each conversation, extracts
"facts", and stores them. It fills up with noise — restatements,
transient context, things the user said once about a specific
request.

Make writing an explicit action the agent takes:

const remember = tool({
  name: "remember",
  description: ""
    "Store a durable fact about this user. Use only for stable " +
    "preferences, identifiers, or constraints that will matter in " +
    "future conversations. Do not store transient request details.",
  schema: z.object({
    fact: z.string().max(200),
    category: z.enum(["preference", "identifier", "constraint"]),
  }),
  async run({ fact, category }, ctx) {
    await ctx.longTerm.remember({
      userId: ctx.userId,
      fact,
      category,
      sourceRunId: ctx.runId,
    });
    return { stored: true };
  },
});

Two benefits beyond precision. The write is attributable — every fact
carries the run that created it, so you can audit where a wrong
memory came from. And it is inspectable and deletable, which you need
the first time a user asks what you have stored about them.

Retrieval is selective, and belongs at the start of a run:

const facts = await longTerm.recall(task, 5);
const system = [BASE_SYSTEM, facts.length
  ? "Known about this user:\n" + facts.map((f) => `- ${f.fact}`).join("\n")
  : ""].join("\n\n");

Five, not fifty. Long-term memory injected wholesale is just a
context leak with extra steps.

Episodic memory stays out of the window

export type RunRecord = {
  runId: string;
  userId: string;
  startedAt: Date;
  outcome: "done" | "turn_limit" | "budget" | "error";
  turns: number;
  costUsd: number;
  toolCalls: { name: string; ok: boolean }[];
  finalSummary: string;
};

This is not memory the model uses. It is memory you use — to answer
which tool fails most, whether cost per run is drifting, which
outcomes correlate with turn limits.

Keeping it in a separate store with its own retention is what stops
it leaking into the context window, which is the one thing that would
make it expensive.

The eviction policy, stated

Working memory: compact when over budget, keep the last N turns
verbatim, never split a tool pair, feed the previous summary forward.

Long-term: written deliberately by a tool, retrieved top-k per run,
deletable by the user.

Episodic: never in the window, retained per your audit policy.

Testing the part that breaks

Compaction is where the bugs are, and it is testable without a model
if you inject the summariser.

it("never leaves an orphaned tool_result", async () => {
  const w = new Window(SYSTEM, fakeSummariser);
  for (const m of longConversationWithToolPairs) w.append(m);

  await w.compact(1000);

  const win = w.window();
  for (const [i, m] of win.entries()) {
    if (isToolResult(m)) {
      expect(hasMatchingToolUse(win, i)).toBe(true);
    }
  }
});

That test catches the malformed-request bug before a user has a
conversation long enough to trigger it — which, in practice, is
always a real user rather than anyone on the team.

If this was useful

AI That Plans covers memory
as separate systems — window management, compaction that keeps what
matters, durable facts written deliberately, and the run history you
keep for yourself.