DEV Community: Papers Mache

KV cache cut by ~45% with near‑same accuracy

Papers Mache — Fri, 25 Sep 2026 05:00:00 +0000

Grouped Value Attention slashes transformer KV memory by roughly 45 % without hurting benchmark scores. By storing only grouped values and reconstructing keys on the fly, it eliminates the need to materialize a full key matrix during decode, cutting the persistent cache scalars dramatically.

Before these works, long‑context decoding relied on full key–value caches that grew linearly with sequence length. Even techniques like grouped‑query attention still kept a key for every token, so memory pressure remained the dominant bottleneck in autoregressive inference.

On a 350 M‑parameter model, GVA cuts persistent cache scalars by about 46 % while keeping average task accuracy within 0.01 points of GQA. The authors report “this representation reduces persistent cache scalars by approximately 45–47 % relative to matched GQA” and note that “the 16‑dimensional positional variant reaches 44.35 average accuracy across five tasks, compared with 44.36 for GQA and 43.88 for MLA” [1].

DeepSeek‑V4.1‑Flash pushes KV compression further, shrinking the global HBM footprint to 890 bytes per token – roughly one quarter of its predecessor. Moreover, “through a dedicated deployment optimization known as SWA Bounded Replay … reduces its persistent KV cache footprint … to roughly 1/8 of that of DeepSeek‑V4‑Flash,” and despite these cuts the model “delivers substantially better performance than the baseline” [2].

Both papers leave open how these memory savings translate into end‑to‑end latency reductions on commodity GPUs. GVA’s custom decoding kernels are still under evaluation, so real‑world throughput gains remain unproven; DeepSeek’s design includes FP4 caching and cross‑layer reuse, which could benefit from hardware supporting low‑precision arithmetic.

If the reported cuts hold in production, the KV cache allocation for decoder‑only models could potentially be reduced substantially while maintaining quality. These reductions suggest that re‑evaluating long‑context benchmarks with smaller cache budgets may enable longer token contexts on more modest hardware.

References

Open‑vocab relation predictor triples recall

Papers Mache — Thu, 24 Sep 2026 05:00:00 +0000

RelateAnything shatters the low‑recall ceiling that has limited zero‑shot visual relationship detection, delivering 2.3–3.5× higher mean recall while staying under a 20 ms per‑frame budget. Prior models were shackled to a fixed set of 50‑56 predicates and relied on object‑label conditioning, preventing them from handling arbitrary relation vocabularies at inference time.

The field has long been constrained by three intertwined obstacles: no free‑text, verified relation corpus; architectures that condition on learned object labels; and evaluation metrics that reward agreement with the training taxonomy rather than true open‑vocab performance. Consequently, scene‑graph systems remained tied to a single detector and could not ingest novel predicate strings.

RelateAnything achieves a mean recall 2.3–3.5× higher than the strongest open‑vocabulary baseline of comparable scale across three cross‑dataset benchmarks and one zero‑shot setting [1]. The boost persists even when paired with a real detector, and it outperforms a 3B‑parameter VLM scene‑graph model while using under 2 % of its parameters.

The relation model runs at 20 ms per frame on a single GPU, alongside a real‑time object detector [1]. This speed translates to roughly 49 FPS end‑to‑end, enabling truly live video processing without sacrificing the recall gains.

The authors acknowledge that training still depends on the RA‑4M corpus covering 10 k free‑text predicates and that positive‑unlabeled supervision may limit performance on completely novel vocabularies; moreover, in‑domain measurements can overstate transfer benefits by up to fivefold. This suggests further work is needed to verify robustness when the predicate bank expands beyond the curated set and to devise metrics that capture true open‑vocab generalisation without bias toward the training corpus.

If these numbers hold under broader scrutiny, developers should replace fixed‑predicate scene‑graph pipelines with RelateAnything for any real‑time video stream, re‑evaluating their systems on OV‑SGG‑Bench or similar cross‑dataset suites to expose hidden recall deficits.

References

RelateAnything: Real-Time Open-Vocabulary Relation Prediction From Any Inputs

Recursive self‑improvement cuts token traffic 49%

Papers Mache — Wed, 23 Sep 2026 05:00:00 +0000

Recursive self‑improvement has been reported to reduce token generation overhead, with over 15% improvement in token efficiency for speculative decoding as noted in the accompanying GitHub summary of [1]. The twist is that the very loops that promise such savings often backtrack, with many search trajectories ending worse than their peak performance.

Today’s LLM deployments often employ speculative decoding to reduce token usage; the cited works discuss speculative decoding and recursive self‑improvement as distinct approaches [1], [2].

Recursive auto‑research loops have been associated with over 15% token‑generation efficiency improvement for speculative decoding and reported 20% lower end‑to‑end serving cost in the GitHub summary of [1].

A recent benchmark where agents revise their own training‑data strategies under a fixed target model reports that 58.33% of settings improve on the first valid attempt, yet 78.26% of searches that continue past their best score end with a lower‑scoring final attempt (2609.13406): the improvement step is not monotone even when the standard outside is fixed.

Generalized Agent Iteration introduces two orthogonal axes—internal vs. external improvement and anchored vs. self‑referential evaluation—that partition the design space into four distinct categories of agent iteration, providing a unified language for both classic policy iteration and recursive self‑improvement [2].

The reported token‑efficiency gains (over 15%) are observed for speculative decoding workloads; it remains unclear how these improvements translate to other downstream tasks such as code synthesis or embodied control [1].

Moreover, the improvement step’s non‑monotonicity means that extending a recursive search can more often degrade performance than improve it, raising reliability concerns for production deployments [2].

Benchmarks that track token efficiency should incorporate recursive self‑improvement loops as a standard baseline, ensuring that reported savings reflect both raw token counts and the stability of performance gains.

References

35 B MoE runs under 3 GiB RAM

Papers Mache — Tue, 22 Sep 2026 05:00:00 +0000

Learned routing slashes the active memory of a 35‑billion‑parameter Mixture‑of‑Experts model to under 3 GiB, turning laptop‑scale inference from fantasy into practice. By predicting which experts will be needed one token ahead, the engine streams only the relevant weights from SSD and never holds the full parameter matrix in RAM.

Until now MoE inference on consumer hardware has been bottlenecked by weight storage rather than compute, with naive offloading offering no speed gain because layer N+1’s routing cannot start until layer N’s output is produced. A fully‑resident engine therefore demands more than 18 GiB of memory just to hold the int4‑quantized weights.

Edge0 reduces peak active memory from 18.2 GiB to 2.9 GiB while boosting decode throughput to 20.4 tok/s across five public benchmarks, staying within a few points of its fp16 teacher on average [1]. The quoted comparison—“the Mac mini M4 Pro has 24 GB…while Edge0’s profile decodes at 20.4 tok/s occupying 2.9 GiB”—captures the five‑fold speedup and the order‑of‑magnitude memory saving.

The quality gap introduced by int4 quantization is largely recovered through an unmerged LoRA trained on the student path, so the routed set exactly matches the staged expert set without dropping any weights [1]. This “prerouter” trick preserves most of the fp16 model’s accuracy while keeping memory tiny.

The technique hinges on a per‑layer prerouter that must be pre‑trained and on fast SSD streaming, which limits portability to machines lacking high‑throughput storage; moreover, 20 tok/s may still be too slow for real‑time latency‑critical workloads. These constraints leave open whether the same memory savings can be achieved on lower‑end disks or with larger MoE configurations.

If the memory wall no longer excludes extreme‑scale models from commodity inference, benchmark suites such as MMLU should add Edge0’s 35 B MoE as a baseline, reshaping how we evaluate accessibility versus performance.

References

The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction

AI/ML Research Digest — Sep 19, 2026

Papers Mache — Mon, 21 Sep 2026 05:00:00 +0000

Efficient long‑context memory and token compression

Continual‑learning anchors combined with merged low‑rank LoRA updates lift retention on a 100‑task benchmark from about 1 % to roughly 35 % [1]. The gain shows that simple architectural tricks can stave off catastrophic forgetting in very long horizons. Grouped Value Attention stores only grouped values and reconstructs keys when needed, cutting the key‑value cache by ~45 % while keeping GQA accuracy intact [2]. A parallel line of work compresses agent caches to one‑quarter of their original size and aligns end‑of‑sequence handling, enabling million‑token contexts with modest compute cost [3][4]. Together these methods make it feasible to run large models over extended texts or streams without exploding memory use.

Mixture‑of‑Experts scaling for embodied and inference models

PhysBrain 1.5 unifies vision and language in a single MoE‑scaled backbone that tackles 28 embodied benchmarks, establishing state‑of‑the‑art performance across tasks despite not targeting diffusion backbones or real‑time video generation. A dedicated sparse‑MoE world simulator accelerates rollouts and improves visual fidelity for agents interacting with simulated environments [5]. On the inference side, a learned routing predictor lets a 35 B MoE model be served directly from SSD while keeping active memory under 3 GiB, narrowing the gap between storage capacity and compute at extreme scales [6].

Recursive self‑improvement (RSI) architectures for LLM agents

The Generalized Agent Iteration framework formalizes RSI as a spectrum of internal updates (meta‑learning) and external updates (fine‑tuning), providing concrete metrics to evaluate autonomous policy upgrades [7]. Building on this, Dream‑RSI and RSIAgent use replay simulators and multi‑agent curricula to achieve self‑enhancement without additional training data, demonstrating practical pipelines for continual improvement [8][9]. Recursive auto‑research loops reduce token traffic by up to 49 % and lower API costs, proving that RSI can translate into real‑world savings.

Unified physical foundation model for embodied diffusion

A single MoE‑scaled diffusion backbone now processes video, robotic, and 3D inputs together, supporting real‑time streaming generation and control for embodied agents [10]. This consolidation removes the need for separate models per modality and opens a path to more coherent multimodal planning.

RelateAnything open‑vocabulary relation predictor

The lightweight predicate‑driven module accepts arbitrary relation strings and region sources, achieving 2.3–3.5× higher mean recall than prior approaches while running at roughly 20 ms per frame [11]. Faster, more flexible relation prediction can improve downstream tasks such as scene graph generation and visual reasoning.

Orthus speculative decoding under BF16

Speculative decoding with Orthus reproduces exact FP32 trajectories; switching to BF16 introduces only minor divergences that do not affect benchmark performance [12]. This confirms that reduced‑precision inference can retain quality, which matters for cost‑constrained deployments.

Additional insights

Latent‑interface training separates perception from actuation in robot policies, preventing shortcut learning and yielding more robust manipulation under cluttered conditions [13].

Sparse‑MoE world simulator not only speeds up rollouts but also delivers higher‑quality video streams for embodied agents, tightening the loop between simulation fidelity and policy performance [5].

ActionSplice inserts corrected actions into chunked video models without replaying completed evaluations, slashing LPIPS scores and reducing latency dramatically [14].

Lightning Weave composes specialist policies on‑policy via distillation, improving accuracy while cutting token consumption, a useful recipe for scaling multitask agents efficiently [15].

References

Hybrid-precision attention reduces compute cost with minimal accuracy loss

Papers Mache — Sun, 20 Sep 2026 05:00:00 +0000

Mixed‑precision quantization can halve the compute cost of LLM attention while keeping accuracy loss below 1 %. By preserving only a narrow set of critical tokens in full precision, HyQuant sidesteps the catastrophic degradation that plagued earlier low‑bit attempts.

Previous efficiency work often applied uniform 4‑ or 8‑bit quantization to the entire attention matrix or used smoothing techniques to handle outliers, which can lead to notable accuracy degradation. Attention computation constitutes a major component of inference latency in both prefill and decode phases.

HyQuant delivers between 1.32× and 3.58× decode‑kernel speedup while preserving near‑full‑precision accuracy across multiple long‑context and reasoning benchmarks — "Experimental results show that HyQuant achieves 1.32 to 3.58 decode‑kernel speedup and 1.04 to 1.17 end‑to‑end decode speedup while maintaining near‑full‑precision accuracy across multiple long‑context and reasoning benchmarks." [1] The method quantizes the bulk of query, key, and value tensors to low‑bit formats but keeps vertical‑line tokens and a local sliding window in full precision, identified by a lightweight pattern detector. This hybrid design works unchanged for vision‑language models that share the same attention topology, and the authors ship an in‑place Triton kernel patch that drops into existing pipelines with a single import.

The extra cost of vertical‑line identification consumes only 3% to 5% of total runtime — "Our evaluation shows that this additional vertical‑line identification overhead accounts for only 3% to 5% of the total runtime." [1] The detector merely accumulates query vectors, runs a matrix multiplication every 64 tokens, and sums columns, operations that are dwarfed by the savings from quantizing the remaining context. Consequently, the hybrid precision layer adds negligible overhead even on dense GPU workloads.

HyQuant also applies to the prefilling stage, though its latency benefits may vary across model sizes and task types. The paper evaluates primarily on decode‑focused benchmarks and provides custom Triton kernels for integration. This suggests an open question: can the same hybrid principle be generalized to encoder‑only architectures without bespoke kernel support?

Production stacks should replace uniform attention quantizers with HyQuant’s hybrid operator and re‑measure end‑to‑end throughput on their target workloads. The reported 1.04–1.17× overall decode speedup suggests services could increase request‑per‑second throughput by up to ~17% while maintaining near‑full‑precision quality. Updating benchmark suites such as LongBench to include a HyQuant baseline will make future comparisons reflect this practical latency frontier.

References

HyQuant: Hybrid-Precision Quantization for LLM Attention

Beacon queries cut KV memory by 40 %

Papers Mache — Sat, 19 Sep 2026 05:00:00 +0000

Predicting which past key‑value (KV) entries will be revisited lets the attention module skip redundant lookups, cutting inference memory dramatically. BeaconKV’s beacon query mechanism turns that prediction into a 40 % reduction in peak KV consumption without harming answer quality.

Before BeaconKV, KV cache compression relied on recent queries as proxies for future attention patterns, assuming that the next token would attend primarily to the most recent context. That assumption collapses on long‑horizon reasoning where Thought Revisiting Tokens (TRT) re‑attend to distant plan fragments, leaving existing compressors blind to upcoming reuse.

BeaconKV achieves up to a 40 % reduction in peak KV memory across four open‑source large reasoning models by maintaining compact “beacon” queries that forecast revisits instead of storing the full query history [1]. The authors show that these beacons capture the small set of similarity groups formed by TRT queries, enabling selective retention of only the KV pairs likely to be needed again.

The approach also preserves answer quality, delivering the largest reported accuracy gain of 31.7 percentage points on Qwen3‑14B for AIME24 when the KV cache is limited to 1024 tokens “observed on Qwen3-14B for AIME24 with a maximum KV cache budget of 1024” [1]. This shows that aggressive compression need not sacrifice performance on demanding chain‑of‑thought tasks.

The method is training‑free but requires users to choose a target compression ratio via flags such as --decode-max-budget, and the reported gains come from benchmarks that already exhibit frequent TRT patterns; workloads without such revisiting behavior may see smaller savings. One open question is how beacon clustering scales when the number of similarity groups grows with more heterogeneous prompts.

If these memory savings hold in practice, the GPU memory required for a given KV cache size could be reduced proportionally; for example, a 40 % reduction would lower the VRAM needed for a 4096‑token cache by roughly the same factor, enabling larger batch sizes or cheaper hardware deployments.

References

BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference

World‑model RL accelerates LLM training by 3‑4

Papers Mache — Fri, 18 Sep 2026 05:00:00 +0000

Replacing the environment with a learned world model shrinks wall‑clock training time for LLM agents by several times (approximately 3–4×) while leaving benchmark scores intact. The field has long assumed that high‑fidelity sandbox execution is unavoidable once an agent leaves the generation phase, because each trajectory must run on a real machine. World Model RL shows that internal simulation can carry the same learning signal at a fraction of the cost.

Before this work, scaling reinforcement‑learning for AutoResearch agents was bottlenecked by environment execution, which cannot be batched and therefore dominates compute as trajectories grow. Generation benefits from massive batching across many prompts, but every rollout still occupied an exclusive sandbox, inflating wall‑clock time regardless of model size. This asymmetry made post‑training the most expensive stage of research‑agent pipelines.

“the two corrections turn the permanent error floor of world model training into a contracting term and reduce the variance below either reward stream alone, and they cut the training compute by three to four times while matching or exceeding full real‑execution RL at two scales.” [1] The paper’s theory guarantees convergence improvement, and the empirical results confirm a consistent 3–4× speedup across tasks and agent scales without degrading performance.

On a 9 B Qwen‑3.5 model the method required only 349 GPU‑hours versus 1,174 for full execution—a 3.4× reduction—yet achieved 21.6 on MLE‑Dojo and 32.8 on DSBench, essentially on par with the baseline. The 4 B variant showed a similar 3.1× drop in compute (286 GPU‑hours) while preserving scores, demonstrating that the gains hold across different parameter regimes.

The approach still hinges on an imperfect world model; bias and noise must be mitigated with online debiasing and inverse‑variance denoising, and the evaluation stops at 9 B parameters and a limited suite of benchmarks. This suggests that further work is needed to verify whether the same compute savings translate to substantially larger agents or more diverse, stochastic environments.

Future AutoResearch pipelines should replace raw sandbox runs with World Model RL as the default post‑training step, meaning that cost estimates for training new agents can be divided by three without sacrificing performance.

References

Scaling Automatic Research Agents via World Models

Zero-training speech integration preserves vision-language ability

Papers Mache — Thu, 17 Sep 2026 05:00:00 +0000

Adding discrete speech tokens to a frozen vision‑language backbone instantly yields functional audio understanding without any gradient updates. “Training‑Free Omni (TFO), a plug-and‑play framework that converts a frozen VLM into a speech‑centric omni model without architectural modification, or multimodal re‑alignment.” [1]

Before TFO, omnidirectional models required dedicated audio encoders and costly joint training on image‑audio‑text triples, tightly coupling the new modality to a specific backbone and often degrading the original visual reasoning abilities.

On 56 benchmarks spanning 21 languages, TFO matches native omni models in audio‑visual understanding while improving average audio‑only performance across all five evaluated model settings. The system extracts confidence‑filtered Whisper transcripts, timestamps them, and feeds the discrete tokens through the VLM’s existing language interface, leaving the visual pathway untouched.

The approach still depends on accurate speech transcription; noisy or out‑of‑domain audio can limit performance, and tasks that benefit from richer acoustic cues may require dedicated encoders. The authors note that “richer acoustic representations remain essential” for certain scenarios, suggesting an open question whether pure transcript routing can cover all auditory phenomena.

Practitioners can now retrofit supported frozen VLMs (e.g., Qwen, Gemma, MiniCPM) with speech capability simply by installing the provided TFO adapters, preserving image‑text benchmarks such as visual grounding and medical QA while gaining multilingual spoken query support. This eliminates the need for expensive multimodal fine‑tuning and invites a rapid‑prototype workflow: drop in the plug‑in, run the existing vision suite, and immediately obtain a functional speech‑centric omni model.

References

Training-Free Speech-Centric Omni Understanding with Frozen VLMs

Unified multimodal model rivals specialized systems

Papers Mache — Wed, 16 Sep 2026 05:00:00 +0000

Unified models have been assumed to lag behind specialist pipelines in generation fidelity, but SenseNova‑U1.5 shatters that myth by reaching the highest open‑source GenEval score with a single 8 B backbone. The system dispenses with any vision encoder or variational auto‑encoder and instead learns directly from raw patches, allowing one set of parameters to process visual data such as images and video frames alike.

Before this work, multimodal deployments stitched together frozen image encoders (e.g., CLIP), separate diffusion decoders, and task‑specific VAEs, creating a patchwork of components that required individual tuning and maintenance. Those architectures achieved strong results in their own domain but suffered from inconsistencies when combined, prompting the community to treat unified modeling as an aspirational research goal rather than a production‑ready solution.

SenseNova‑U1.5 achieves 0.92 on GenEval, the best overall performance among open‑source models and overtaking larger baselines such as Qwen‑Image, while using a single encoder‑free backbone [1]. The authors attribute this to spatially coherent patch reconstruction, curated generation/editing data up to 4K resolution, and multi‑expert on‑policy distillation that injects specialist capabilities without separate encoders.

The study’s scope remains confined to visual generation; audio and video benchmarks are absent, and the paper acknowledges limited exposure to structured formats despite strong generalization [1]. This suggests that while the unified architecture can match specialists on vision tasks, its cross‑modal robustness and latency characteristics under real‑world workloads remain open questions.

Enterprises can replace fragmented vision‑language stacks with a single SenseNova‑U1.5 checkpoint, cutting maintenance overhead without sacrificing output quality. The logical next experiment is to evaluate the same model on speech‑to‑image and video‑editing suites to verify whether the unification claim truly extends beyond static images.

References

SenseNova-U1.5: Towards Native Unified Visual Intelligence

Steering vectors align LLMs with human values

Papers Mache — Tue, 15 Sep 2026 05:00:00 +0000

Linear directions extracted from large‑language‑model activation distributions map onto human‑value axes with measurable fidelity. The study demonstrates that these steering vectors preserve the full geometry of a theory‑driven value space, not merely isolated behavioral tweaks.

Before this work, safety constraints were typically imposed by heavyweight fine‑tuning pipelines such as RLHF or DPO, or by behavior‑centric steering methods like COLD‑Steer and BiPO that optimize single‑task performance without guaranteeing coherent semantic structure. Those approaches focus on reducing targeted harms while leaving the underlying value topology unexplored.

Distribution‑driven steering recovers the expected human‑value topology with a Spearman correlation up to 0.51, confirming alignment with Schwartz’s theory of basic values[1]. The authors compute this metric by projecting activation distributions onto linear directions and comparing the resulting pairwise similarity matrix against the theoretical circumplex; the peak correlation exceeds 0.5, which the authors treat as indicative of meaningful structure.

Across all evaluated metrics, only distribution‑driven methods exhibit strong geometric alignment, while behavior‑centric approaches show no statistically significant correlation despite comparable results on the benchmark (Table 4)[1]. This contrast indicates that shortcut‑based steering can hit target scores without embedding the intended value relationships, underscoring the uniqueness of linear, distribution‑derived vectors.

Geometric fidelity improves with model scale but consistently degrades once the base model is instruction‑tuned (Table 1)[1]. Larger models furnish richer activation manifolds that better capture the circumplex, yet the fine‑tuning process reshapes those manifolds enough to weaken the correspondence between steering directions and human values.

The paper’s own limitation is that the observed degradation after instruction tuning leaves a gap for practitioners who need both strong task performance and value geometry preservation. This suggests an open question: can we design instruction‑tuning regimes that retain—or even enhance—the latent alignment discovered by distribution‑driven steering? Moreover, while geometry correlates with human‑consistent transfer across values, it does not directly guarantee the absence of edge‑case harms, so downstream validation remains essential.

If these findings hold, safety pipelines should incorporate a lightweight inference‑time module that extracts and applies distribution‑driven steering vectors rather than resorting to full model fine‑tuning. Re‑running existing safety benchmarks with an added geometry metric (Spearman correlation against the Schwartz framework) will expose whether a deployment truly respects the intended value structure.

References

Steering Geometry: Validating Human Value Geometry in LLM Steering Space

AI/ML Research Digest — Sep 12, 2026

Papers Mache — Mon, 14 Sep 2026 05:00:00 +0000

Efficiency across multimodal and language models

Hybrid‑precision attention quantization halves the compute of transformer layers while keeping accuracy intact [1].

Latent next‑concept prediction reduces the number of training tokens to roughly 51 % and lifts downstream scores compared with conventional language models [2].

Chain‑of‑thought token compression lets a model skip redundant key‑value lookups, cutting memory use by up to 40 % during long reasoning chains without harming answer quality [3].

Safety via latent steering and self‑distillation

Linear directions extracted from activation distributions align model latents with human value topologies, offering a systematic way to steer outputs toward ethical constraints [4].

Automatically generated refusal data feed a fine‑grained self‑distillation loop that raises refusal rates on target domains; the same loop can increase over‑refusal on broader benchmarks, highlighting a trade‑off that must be managed [5].

Transformer backbones replace task‑specific pipelines

FreeFlow’s hierarchical windowed attention removes explicit correlation volumes yet reaches state‑of‑the‑art optical flow performance on Sintel and KITTI, simplifying the pipeline dramatically [6].

Other transformer‑only designs attain comparable results on depth estimation, surface normal prediction, and relighting without any handcrafted biases, proving that a single backbone can serve multiple vision tasks [7].

Highlighted papers

SenseNova‑U1.5 unified multimodal model – The encoder‑free architecture processes images, audio, and video with one parameter set and matches the performance of dedicated models on each modality [8].

World Model RL for LLM agents – Replacing costly environment steps with a learned world model trims wall‑clock training time by 3–4× while preserving long‑horizon task success [9].

NCP‑ArchPreview next‑token and concept prediction – Joint token‑and‑concept forecasting consumes about half the usual training tokens and yields better downstream metrics than standard language models [2].

Additional technical details

Hybrid‑precision attention quantization (HyQuant) mixes precision levels inside attention matrices, delivering near‑lossless accuracy and notable speedups, though the exact factor is not disclosed [1].

Training‑Free Omni injects speech tokens into frozen vision‑language backbones without any gradient updates; the resulting model handles speech while retaining image‑text capabilities [10].

Co‑evolving harnesses let weaker LLM agents receive expert corrections during on‑policy reinforcement learning, preserving alignment between agent and harness and accelerating convergence without resorting to imitation learning [11].

A deterministic token‑selection algorithm guarantees coverage across multiple camera views during inference, boosting zero‑shot 3D perception consistency in visual reasoning benchmarks [12].

DEV Community: Papers Mache

KV cache cut by ~45% with near‑same accuracy

References

Open‑vocab relation predictor triples recall

References

Recursive self‑improvement cuts token traffic 49%

References

35 B MoE runs under 3 GiB RAM

References

AI/ML Research Digest — Sep 19, 2026

Additional insights

References

Hybrid-precision attention reduces compute cost with minimal accuracy loss

References

Beacon queries cut KV memory by 40 %

References

World‑model RL accelerates LLM training by 3‑4

References

Zero-training speech integration preserves vision-language ability

References

Unified multimodal model rivals specialized systems

References

Steering vectors align LLMs with human values

References

AI/ML Research Digest — Sep 12, 2026

References

35 B MoE runs under 3 GiB RAM

Beacon queries cut KV memory by 40 %