AI/ML Research Digest — Sep 19, 2026

Efficient long‑context memory and token compression

Continual‑learning anchors combined with merged low‑rank LoRA updates lift retention on a 100‑task benchmark from about 1 % to roughly 35 % [1]. The gain shows that simple architectural tricks can stave off catastrophic forgetting in very long horizons. Grouped Value Attention stores only grouped values and reconstructs keys when needed, cutting the key‑value cache by ~45 % while keeping GQA accuracy intact [2]. A parallel line of work compresses agent caches to one‑quarter of their original size and aligns end‑of‑sequence handling, enabling million‑token contexts with modest compute cost [3][4]. Together these methods make it feasible to run large models over extended texts or streams without exploding memory use.

Mixture‑of‑Experts scaling for embodied and inference models

PhysBrain 1.5 unifies vision and language in a single MoE‑scaled backbone that tackles 28 embodied benchmarks, establishing state‑of‑the‑art performance across tasks despite not targeting diffusion backbones or real‑time video generation. A dedicated sparse‑MoE world simulator accelerates rollouts and improves visual fidelity for agents interacting with simulated environments [5]. On the inference side, a learned routing predictor lets a 35 B MoE model be served directly from SSD while keeping active memory under 3 GiB, narrowing the gap between storage capacity and compute at extreme scales [6].

Recursive self‑improvement (RSI) architectures for LLM agents

The Generalized Agent Iteration framework formalizes RSI as a spectrum of internal updates (meta‑learning) and external updates (fine‑tuning), providing concrete metrics to evaluate autonomous policy upgrades [7]. Building on this, Dream‑RSI and RSIAgent use replay simulators and multi‑agent curricula to achieve self‑enhancement without additional training data, demonstrating practical pipelines for continual improvement [8][9]. Recursive auto‑research loops reduce token traffic by up to 49 % and lower API costs, proving that RSI can translate into real‑world savings.

Unified physical foundation model for embodied diffusion

A single MoE‑scaled diffusion backbone now processes video, robotic, and 3D inputs together, supporting real‑time streaming generation and control for embodied agents [10]. This consolidation removes the need for separate models per modality and opens a path to more coherent multimodal planning.

RelateAnything open‑vocabulary relation predictor

The lightweight predicate‑driven module accepts arbitrary relation strings and region sources, achieving 2.3–3.5× higher mean recall than prior approaches while running at roughly 20 ms per frame [11]. Faster, more flexible relation prediction can improve downstream tasks such as scene graph generation and visual reasoning.

Orthus speculative decoding under BF16

Speculative decoding with Orthus reproduces exact FP32 trajectories; switching to BF16 introduces only minor divergences that do not affect benchmark performance [12]. This confirms that reduced‑precision inference can retain quality, which matters for cost‑constrained deployments.

Additional insights

Latent‑interface training separates perception from actuation in robot policies, preventing shortcut learning and yielding more robust manipulation under cluttered conditions [13].

Sparse‑MoE world simulator not only speeds up rollouts but also delivers higher‑quality video streams for embodied agents, tightening the loop between simulation fidelity and policy performance [5].

ActionSplice inserts corrected actions into chunked video models without replaying completed evaluations, slashing LPIPS scores and reducing latency dramatically [14].

Lightning Weave composes specialist policies on‑policy via distillation, improving accuracy while cutting token consumption, a useful recipe for scaling multitask agents efficiently [15].

References

1. Continual Learning Mechanisms Compose for Long-Horizon Memorization
2. Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction
3. DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
4. When EOS Tokens Disagree: Understanding Length Inflation in On-Policy Distillation
5. Pelican-Sim 1.0: A General World Model Simulator for Embodied Intelligence
6. The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction
7. Generalized Agent Iteration: One Formal Framework for Iterative Policy Improvement and Recursive Self-Improvement
8. Dream-RSI: Recursive Self-Improvement through Evolving Worlds
9. RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments
10. PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models
11. RelateAnything: Real-Time Open-Vocabulary Relation Prediction From Any Inputs
12. How Lossless Is Lossless Speculative Decoding? The Role of Numerical Precision in Orthrus
13. Breaking the Vision-Action Shortcut: Latent Interface Training for Generalizable Robotics Foundation Models
14. ActionSplice: In-Flight Action Editing for Interactive World Models
15. Lightning Weave: Improving the Accuracy-Efficiency Frontier of Reasoning Models through Capability Composition