Long-term Agent Memory is not only about retaining more history. It is about retrieving evidence that remains useful when facts, decisions, and tasks change.
Agent Memory Challenge 2026 — Cycle 2 is now open.
Textual Memory · Coding Memory · Multimodal Memory
Open-source Methods · Commercial Products
Join the evaluation
Everyone agrees that AI agents need memory.
But “memory” can mean very different things in practice.
One system may preserve raw conversation history. Another may summarize it into structured facts. A third may retrieve repository traces, prior debugging attempts, or images from earlier sessions. Some systems use dense retrieval; others use graphs, databases, rerankers, or generative memory consolidation.
The difficult question is not whether an Agent can store information.
The difficult question is whether it can retrieve the information that should govern its next action.
A memory system can retrieve a highly similar statement from the past and still mislead an agent if that statement has since been corrected. It can return every related record and still fail if the agent cannot distinguish a current decision from an obsolete one.
That is the problem Agent Memory Challenge 2026 Cycle 2 is designed to examine.
What is the Agent Memory Challenge?
The Agent Memory Challenge is the public evaluation program of the Agent Memory Leaderboard (AML), an open benchmark for long-term Agent Memory.
Participants provide the memory layer through two interfaces:
- Add — receives content that the system should store, organize, index, update, or consolidate.
- Search — returns relevant memory evidence for a later query and user scope.
AML then runs the shared downstream workflow:
- Answer — generates task responses from retrieved evidence.
- Eval — scores results under a standardized protocol.
- Review — verifies submitted versions, evaluation conditions, and compliance before publication.
This separation matters.
In many public comparisons, different systems use different answer models, prompts, datasets, scoring methods, and evaluation procedures. It can become difficult to tell whether a result reflects a better memory system, a stronger answer model, or simply a different test setup.
AML creates a clearer evaluation boundary:
Participants build the memory layer.
AML standardizes the downstream Answer and Eval process.
The goal is not to claim that one architecture is universally the “best” memory system. It is to make different approaches more comparable under a shared, reproducible evaluation flow.
Why “current evidence” matters
Long-running agents do not operate in a static world.
A release date changes. A preference is corrected. A debugging hypothesis is disproven. A repository evolves. A previous decision becomes invalid after new evidence appears.
In these situations, retrieving the most similar old memory is not enough.
A useful memory system should help an agent answer questions such as:
- What was previously believed?
- What changed later?
- Which source supports the current state?
- Which older information remains historically relevant?
- Which information should no longer influence the next task?
- Can the returned evidence be traced back to its source?
This is why Agent Memory should not be evaluated only as a search problem.
It is also a problem of temporal reasoning, provenance, relevance, scope, and decision support.
Three tracks in Cycle 2
Cycle 2 expands the evaluation across three independent tracks.
Textual Memory
The Textual Memory track evaluates long conversations and cross-session history.
It includes capabilities such as:
- Explicit fact recall
- Multi-hop relations and reasoning
- Temporal events and changing states
- User personalization
- Rule and workflow following
- Memory governance and safety boundaries
Cycle 2 also includes Streaming Memory conditions, where Add and Search calls occur as events unfold. This tests whether a system can use the right information at the right time, rather than relying only on a static archive of the full conversation.
Coding Memory
Coding agents accumulate a different kind of history.
Past implementation decisions, repository conventions, failed debugging attempts, test results, error messages, module relationships, and development trajectories can all matter to a later task.
The Coding Memory track evaluates whether historical engineering experience can help an agent solve subsequent software-engineering tasks.
The formal evaluation contains 150 software-engineering tasks, tested under both relevant-history and noisy-history settings—for 300 task-condition units in total.
The question is not simply whether a system can find a file path or a past error message.
It is whether it can retrieve useful engineering evidence without overwhelming the agent with irrelevant historical noise.
Multimodal Memory
Agents increasingly work across text, images, screenshots, documents, and other visual context.
The Multimodal Memory track evaluates how systems write, retrieve, and use text-and-image memory. It tests whether a system can preserve ordered multimodal context, connect evidence across modalities, and retrieve the information needed for a later task.
Participants may use original images, captions, or both, while disclosing the media types their systems actually process.
Core principles of the evaluation
The benchmark is built around a few practical rules.
Search returns evidence, not a disguised final answer
The participant’s Search API should return memory evidence relevant to the query. AML handles downstream answer generation and scoring.
This keeps the memory layer distinct from the final response model.
User isolation is a strict boundary
Memory from one user_id, task, sample, team, or evaluation run must not leak into another.
Long-term memory is useful only if it is also appropriately scoped.
A successful write must be searchable
A synchronous Add request should return success only after the content has been persisted and can be retrieved through Search.
Retries should not create duplicate memories
Systems should handle request_id idempotently, so a network retry does not write the same memory multiple times.
Formal evaluations use a fixed system version
Once a Full evaluation begins, the submitted code, image, endpoint, authentication method, and operating conditions must remain stable.
Results are published in the context of the corresponding track, division, system version, evaluation suite, and cycle.
Cycle 1: a shared evaluation is already attracting attention
AML published its first leaderboard cycle on August 12, 2026.
During the first cycle:
- 136 teams submitted applications
- 67 representative memory systems completed official evaluation
- The AML website received more than 300,000 cumulative visits
- AML’s Hugging Face Space reached the top of the Spaces trending list during the first cycle
These figures reflect participation and community interest in Cycle 1. They do not imply that any single system is universally superior outside its submitted version, track, and evaluation conditions.
What they do show is that the Agent Memory community is actively looking for more transparent ways to compare systems.
Who can participate?
Cycle 2 is open globally to:
- Universities and research institutions
- Open-source maintainers
- Independent research teams
- Commercial product teams
- Individual developers
- Cross-organization teams
Each track includes two separate divisions:
- Open-source Methods
- Commercial Products
Participants may enter multiple tracks, while rankings remain separate by track and division.
Participation is free. Participants operate their own APIs, databases, bandwidth, and compute resources; AML provides the standardized evaluation orchestration, downstream Answer and Eval process, result review, and leaderboard publication.
Key dates
- September 20, 2026, 00:00 UTC+8 — Cycle 2 opens
- September 20–October 31 — Registration, integration, Smoke tests, Full evaluations, and review
- October 31, 2026, 23:59 UTC+8 — Submission-material deadline
- November 4, 2026, 23:59 UTC+8 — Evaluation closes
- Mid-November 2026 — Official results planned for release
Teams should submit stable APIs early. A Full evaluation typically takes approximately 0.5–2 days, depending on the implementation and evaluation scale.
Open-source awards
Cycle 2 provides a total prize pool of approximately USD 22,400 for eligible Open-source Methods teams across the three tracks.
Each track includes:
- First Prize — approximately USD 2,980
- Two Second Prizes — approximately USD 1,190 each
- Three Third Prizes — approximately USD 450 each
- Best Technical Innovation Award — approximately USD 750
Awards are denominated and paid in RMB. USD figures are approximate and may vary with the exchange rate.
Commercial Products are ranked in a separate division and are not eligible for the prize pool.
Explore AML and participate
- Official website
- Evaluation and registration
- Participation rules
- API integration guide
- GitHub repository
Long-term Agent Memory should be more than a growing context window.
It should help an agent retrieve the evidence that is relevant now, understand what has changed, and act on a record that can be inspected and verified.
Make memory measurable. Make progress verifiable.
Top comments (0)