On October 20th, we are officially releasing and open-sourcing the memory-native foundation model series, MemT 1.0.
MemT 1.0 integrates persistent, updatable internal memory into model computation, so that historical information and task experience can be retained and recalled in subsequent related tasks. Our goal is to make "remember, update, and apply experience" a capability the model can learn.
This release includes MemT-1.0-Flash and MemT-1.0-Nano, with 288B and 36B total parameters and 17B and 3B active parameters, respectively. Across 11 memory benchmarks, the two models achieve average scores of 50.6 and 47.0, ranking first and second among the parametric memory methods evaluated in this assessment — all under conditions where the query context does not provide the original history text.
Figure: Memory performance of MemT 1.0 across benchmarks
01 Memory-Native: From Information Retention to a Learnable Capability
Foundation models are internalizing capabilities such as reasoning and multimodality into themselves; memory is another critical capability. It enables a model to carry forward prior information and experience, accumulating and using them across sustained interactions.
Earlier, Metis introduced memory foundation models, exploring memory as a native capability of the model. MemT 1.0 builds on this foundation with an upgrade: native memory modules are added to selected Transformer blocks, so that the original attention and memory attention run in parallel, jointly supporting information storage and use through dynamic local memory and trainable hyper memory.
Figure: The memory-native architecture of MemT 1.0
Textual memory methods provide the original history or retrieved evidence at query time; MemT 1.0 instead leverages previously written information through internal memory states, without requiring the raw history text to be provided again.
The key innovation is teaching the model how to retain, update, and read memory. During online interaction, the backbone and hyper-memory weights remain fixed; what changes is the dynamic memory state maintained independently for each interaction trajectory. Generation in the current turn reads existing memory; once the interaction completes, the input and answer are written back to memory for use in subsequent interactions.
02 The Memory Mechanism: How It Updates, How It Scales
The memory mechanism of MemT 1.0 is built around three key pillars.
- Token-wise stable updates
A stable update mechanism based on Gated DeltaNet-2 updates memory token by token during the storage process after each interaction, merging new information into existing memory through erase, write, and forget mechanisms.
- Mixture-of-Storage for capacity scaling
Mixture-of-Storage (MoS) expands the native memory module into multiple memory experts, distributing information through independent storage and utilization routing. MoS employs dense routing in which all experts participate, providing a mechanism for expanding memory capacity. In this main evaluation, each memory layer uses a single expert; multi-expert scaling is examined separately in ablation experiments.
Figure: Mixture-of-Storage (MoS)
- Two-stage training
Continual pretraining builds language modeling capability that relies on native memory, followed by post-training that learns to complete tasks based on memory. The report constructs a 34B-token training corpus, comprising 20B tokens for continual pretraining and 14B tokens for post-training. Post-training adopts a "context–query–answer" format, in which the student model learns to reproduce the full-context teacher's answering behavior using only its memory state.
03 Measured Performance: It Remembers, and It Stays Reliable
A memory-native model must not only make good use of historical information, but also maintain its original capabilities under memory noise. We evaluated MemT 1.0's actual performance along two dimensions: memory capability and general capability.
Memory capability: can it use history well without the source text?
The experiments span 11 memory benchmarks, evaluating information retention and retrieval across different history lengths, dynamic updates and management, and personalized understanding. History is written into memory in chronological order; the model then answers related questions. Memory is strictly isolated across different users and histories.
With no original history text in the query context, MemT-1.0-Flash and MemT-1.0-Nano score 50.6 and 47.0 on average, ranking first and second among the parametric memory methods in this evaluation — 12.5 and 8.9 points above the best average of other parametric memory methods, respectively.
The two models have different strengths: Flash performs better on memory operations, personalized understanding, dialogue recall, and implicit learning; Nano scores higher on PerMemSafe and Memora. The strongest textual memory systems still hold an overall advantage, particularly in long-history tests. Ensuring that important details survive continuous updates and are accurately read back when needed remains a key direction for memory-native models.
*Memory capability evaluation across 11 benchmarks. Benchmarks are grouped by interaction trajectory length. DS denotes DeepSeek-V4-Flash-0731; QW denotes Qwen3.6-35B-A3B. In the original table, bold and underline denote the best and second-best results within each memory category, respectively.
General capability: can it hold up against irrelevant memory?
Models must also cope with interference from irrelevant memory. We write irrelevant information into memory before evaluation to simulate interference during sustained interaction, then compare against the corresponding backbone models under the same noise setting across four dimensions — knowledge and scientific reasoning, instruction following, long-context understanding, and code generation — on six benchmarks. Flash's average score is roughly on par with its backbone, while Nano drops by 0.9 points.
First, instruction following: can the model accurately execute the current request? In the table, IFE refers to IFEval, which tests whether a model complies with explicit requirements such as specified formats, length limits, and required keywords — matters that determine whether an answer can be put directly to work. Under irrelevant information, Flash improves 0.9 points over its backbone to 93.9; Nano improves 3.3 points to 92.6, showing that both models still execute current instructions well in this test.
*General capability evaluation under the noise setting. Values for MemT 1.0 denote score differences relative to the corresponding backbone under the same setting.
Second, the practical programming everyone cares about: can generated code actually get the job done? BCB-H refers to BigCodeBench-Hard, which requires the model to understand complex requirements, combine software libraries, and generate code that passes execution tests — testing whether the model can turn programming knowledge into runnable solutions. In this test, Flash and Nano each improve 2.0 points over their backbones, reaching 34.4 and 27.7, respectively. However, another code test, LiveCodeBench v6, drops by 0.7 and 5.8 points, indicating that different types of programming tasks are affected differently, and a single improvement should not be extrapolated into an overall gain in programming capability.
Long-text understanding asks: faced with a large body of material, can the model sort out the information and answer questions? LB refers to LongBench v2, which covers long documents, multi-document settings, long conversations, and code repositories, testing the model's comprehension and reasoning over long-form material. Under irrelevant information, Flash improves 0.2 points over its backbone to 64.4, roughly on par overall, while Nano drops 2.0 points. The retention of historical information in internal memory and the comprehension of current long-form material need to be examined separately.
These results show that memory capability and general capability can largely coexist, but some tasks still incur performance loss. Keeping models reliable while they continuously accumulate information is the direction memory-native models need to keep optimizing.
04 Putting Experience to Work: How Memory Changes What Happens Next
MemT 1.0 explores a new path for growing foundation model capabilities: beyond parameters, data, and context, effective experience is retained and put to work in subsequent tasks.
In a Coding Agent, operational experience — methods for reproducing problems, failure modes, and verification workflows — is retrieved, written into the model's internal memory, and then guides subsequent actions, without continuously occupying task context in textual form. The current experiments use an externally constructed experience library and apply task-specific post-training on top of Nano.
We compared no experience, textual experience, and memory-native approaches on 500 tasks from SWE-bench Verified and 300 tasks from SWE-bench Multilingual covering nine programming languages, running each task independently three times.
On Verified, with experience provided at task start, Nano raises the average success rate from 66.07% to 69.33%, versus 68.87% with textual experience. When the execution framework triggers experience writing at key breakpoints, the average success rate reaches 69.73%, and the proportion of tasks succeeding all three times rises from 54.00% to 62.80%, an improvement of 8.80 percentage points.
On Multilingual, the framework-triggered setting achieves an average success rate of 63.56%, higher than 60.00% without experience and 63.00% with textual experience; the proportion of tasks succeeding all three times rises from 45.33% to 55.33%. The gains mainly appear in average success rate and consistency across repeated runs; the proportion of tasks succeeding at least once did not consistently improve.
*Agentic coding performance on SWE-bench Verified and SWE-bench Multilingual. In the original table, bold and underline denote the best and second-best results within each benchmark and experience-setting combination.
05 Storage and Runtime Efficiency: Persistent Accumulation with a Controllable State
With a fixed memory configuration, new information updates the existing matrix, and the persistent state size does not keep growing with history. In the main evaluation configuration, Nano's FP32 memory state is 40 MiB per interaction trajectory, and Flash's is 1,280 MiB, excluding shared model weights and the current turn's KV cache.
In the efficiency evaluation on BEAM history, at a 128K-token history length, Nano achieves roughly 1.86x end-to-end inference speed compared with the full-context approach, with a persistent state about 1/66 of its size; at a 256K history, end-to-end speed is about 3.99x.
After history preparation, across the evaluated range from 512 to 256K tokens, Nano's average query first-token latency is approximately 104–112 ms, remaining essentially stable. End-to-end latency includes history preparation and the generation of a fixed 128-token answer; query first-token latency excludes history preparation.
*Efficiency evaluation results: end-to-end latency and query first-token latency.
*Persistent storage footprint per interaction trajectory.
These advantages mainly appear in long-history scenarios; with short histories, Nano is usually slower. A fixed state also means limited information capacity: total storage still grows with the number of sessions and the configured capacity. The efficiency experiments above should not be directly equated with lower costs on programming tasks — experience writing and additional verification still incur overhead.
06 Limitations and Next Steps
We see that fixed-capacity memory in memory-native models must strike a balance between preserving early details and accommodating new information, and long-term fidelity of precise information remains a challenge. Current experience application still relies on external experience construction and retrieval; a complete loop of autonomously acquiring, verifying, and continuously integrating experience has not yet formed.
Next, we will improve long-term memory, expand memory capacity, and reduce the cost of memory updates. We will also further explore autonomous experience acquisition and continuous cross-task memory integration, gradually moving toward agents that learn from interaction through memory-native evolution.
Memory-native capability provides a foundation for continual learning and recursive self-improvement (RSI); feedback, verification, and update mechanisms still need to evolve together.
07 Learn More
Parameters set where a model's capability starts; memory opens up the space for intelligence to grow.
Developers and researchers are welcome to explore the MemT 1.0 models and the possibilities of memory-native capability.
Hugging Face: https://huggingface.co/collections/MemTensor/memt-1
Top comments (0)