If you've built anything with an LLM agent — a coding assistant, a browser agent, a research bot — you've probably watched it get worse the longer a task runs. It starts sharp, then a few dozen tool calls in, it starts repeating itself, forgets a constraint you gave it three turns ago, or burns through your token budget doing something it already tried and failed at.
The obvious fix everyone reaches for is a bigger context window. A recent paper out of Fudan University, GenericAgent, argues that's the wrong instinct entirely — and backs it up with numbers that are hard to ignore.
The real problem isn't length, it's density
The paper's core claim: agent performance isn't governed by how much context you can fit, it's governed by how much of that context is actually decision-relevant at any given moment. Stuffing more tokens into the window doesn't help if most of them are noise, and the paper cites three concrete failure modes:
- Positional bias — information buried in the middle of a long context is harder for the model to retrieve than information at the start or end (the well-known "lost in the middle" effect).
- Active interference — irrelevant content doesn't just sit there inertly, it competes with and dilutes the model's attention away from the evidence that actually matters.
- Shrinking effective context — the usable context length of an LLM is meaningfully smaller than its advertised window size. Much of a 1M-token context is, in practice, functionally inaccessible during generation.
On top of that, most agent frameworks treat every task as stateless — useful knowledge like "this tool has a weird edge case" or "this user always wants CSV, not JSON" gets learned through trial and error and then thrown away the moment the session ends. Even frameworks with memory tend to store raw logs, not distilled lessons, so token spend keeps climbing per task while the agent's actual capability stays flat.
What GenericAgent does differently
The system is built around a deliberately narrow toolset — just 9 atomic tools (file read/patch/write, code execution, web scan/execute, checkpoint memory, ask-user) compared to Claude Code's 53 or OpenClaw's 18 tool factories. The design rule is that each tool represents one irreducible capability with zero overlap, which cuts down the decision overhead of choosing between tools that do almost the same thing.
The more interesting piece is a four-layer memory hierarchy:
- L1 (index) — compact pointers and keyword mappings, always loaded, minimal by design. Think of it as a table of contents the agent consults to decide what's worth retrieving in full.
- L2 (facts) — verified, stable knowledge, admitted only after it's proven reusable across tasks.
- L3 (SOPs) — reusable procedures: workflows, common failure cases, recovery strategies.
- L4 (raw archive) — full session logs, kept for auditing but not injected into the live context.
New information only gets promoted into L2/L3 under a rule the paper calls "No Execution, No Memory" — nothing gets remembered unless it was verified through an actual successful tool call. Guesses and failed attempts get discarded during consolidation rather than cluttering future context.
On top of that sits a graduated compression pipeline: tool outputs get head-tail truncated, repeated context blocks get replaced with placeholders every few turns, and once history exceeds the token budget, the oldest messages get evicted on a rolling basis — with a persistent "working memory anchor" (recent turn summaries plus key facts) surviving the eviction so the agent doesn't lose the thread. The whole system targets a context budget under 30k tokens, deliberately rejecting the industry trend toward ever-larger windows.
The self-evolution loop
The part that makes this more than "just add caching" is how the agent's knowledge compounds over repeated tasks, in three stages:
- Natural-language execution — first attempt at a task, reasoning in plain text, trial and error.
- SOP distillation — once something works, the successful path gets compressed into a structured procedure, stripped of the dead ends.
- Code-based execution — a proven workflow eventually gets crystallized into executable code, skipping the reasoning step entirely on future runs.
This isn't manually triggered — the system escalates through these stages on its own as evidence accumulates.
Does it actually work? The numbers
This is the part that made me pick this paper over others trending this week — it's not just an architecture proposal, it has real head-to-head benchmarks against Claude Code and OpenClaw:
- On a 5-task long-horizon benchmark (doc generation, SQL copilot, experiment reporting, etc.), GenericAgent matched Claude Code's 100% success rate while using 188,829 tokens versus Claude Code's 537,413 — about a third of the token spend.
- In a repeated-task test (the same data-download task run 5 times), GenericAgent's runtime dropped from 102s to ~66s and token use fell from ~200k to ~100k, while baseline frameworks stayed flat run over run — direct evidence the "self-evolution" claim isn't just theoretical.
- In a longer 9-round longitudinal study on a GitHub research task, the fully-evolved (code-based) stage cut runtime by 78%, LLM calls by 84%, and total tokens by 90% compared to round 1.
- On long-term factual memory (the LoCoMo benchmark), GenericAgent beat both Mem0 and A-MEM — without using any embedding model or vector database at all, which is a genuinely counterintuitive result given how much of the "agent memory" space has converged on embeddings as the default.
- After installing 20 skills and heavy usage, a bare "Hello" request cost GenericAgent 2,298 tokens versus Claude Code's 22,821, CodeX's 23,932, and OpenClaw's 43,321 — a rough order-of-magnitude difference in baseline overhead.
Where it's honest about its limits
The authors flag real, specific limitations rather than hand-waving: a hard 30-round execution cap that forces long tasks to span multiple sessions via written reports rather than true continuous state; a self-adjusting weighting mechanism for task exploration that they call "preliminary" and not yet proven at scale; and a skill-tree maintenance process (merging, deprecating outdated skills) that is currently entirely manual. They also float the idea of the agent eventually modifying its own codebase — feasible in principle since the whole system is ~3,300 lines versus OpenClaw's ~530,000 — but they're explicit that this is unvalidated future work, not something the paper demonstrates.
Why this matters if you're building agents
The takeaway that stuck with me: the paper reframes token consumption as a symptom rather than a cost. As they put it, "in long-horizon agentic settings, token consumption reflects the symptoms of an agent's context management quality, rather than the thoroughness of reasoning." If your agent is burning tokens, that's usually a sign it's carrying dead weight in its context, not that it's "thinking harder." That's a genuinely useful lens to bring to any agent system you're debugging, independent of whether you adopt this specific architecture.
Paper: GenericAgent (arXiv:2604.17091) · Code
Top comments (2)
One thing I'd want to know about the No Execution, No Memory rule: a tool call can succeed for the wrong reason, so one lucky run could promote a bad lesson into the fact layer, and then it gets loaded into every future task. Does the paper describe any way to demote a fact once it stops working?
I found the concept of a four-layer memory hierarchy in GenericAgent particularly intriguing, as it addresses the issue of context density and noise in LLM agents. The idea of promoting information to higher layers only after it's proven reusable across tasks through the "No Execution, No Memory" rule resonates with my experience in optimizing knowledge graph-based systems. By filtering out unverified information and focusing on verified, stable knowledge, GenericAgent's approach could significantly reduce the cognitive load on the model and improve its overall performance. I'm curious to see how this design would handle complex, multi-step tasks that require a deeper understanding of context and intent - do you think GenericAgent's narrow toolset and memory hierarchy would be sufficient to tackle such tasks, or would additional mechanisms be needed?