When one model hands work to another, it writes words. Those words get tokenized, embedded, and re-expanded into internal state that the first model already had.
An ICLR 2026 paper skips that round trip entirely, projecting one model's KV cache straight into another's, and reports a 2.5x latency speedup. The most interesting number in it is buried in an ablation table.
Text is a lossy interface between models
Every multi-model system today communicates the same way: model A generates tokens, model B reads them. That interface has two costs that are easy to stop noticing.
The first is semantic loss. A model's internal representation of a concept is a high-dimensional state, and writing it out collapses that into a token sequence chosen by a sampling procedure. The receiver then rebuilds an approximation from scratch.
The second is latency. Text has to be produced token by token, so the handoff costs a full generation pass before the second model can start.
You are paying a serialisation tax between two systems that both speak the same internal language.
The oracle that made it plausible
Before building anything, the authors ran a check worth copying as a methodology. The question was whether a fixed-length cache can be made semantically richer, because if not, the whole idea fails.
Three conditions, same target model:
- Direct, prefilled on the question alone: 58.42%.
- Few-shot, prefilled on exemplars plus question, giving a longer cache: 63.39%.
- Oracle, prefilled on exemplars plus question, then discarding the exemplar segment and keeping only the question-aligned slice: 62.34%.
That third number is the result. The oracle cache is exactly as long as the direct cache and performs almost as well as the full few-shot version. The benefit of the examples survived inside a cache of unchanged size.
A second oracle trained a 3-layer MLP to map Qwen3-4B's cache into Qwen3-0.6B's space. A t-SNE plot showed the raw caches sitting far apart, and the projected cache landing inside the target's representation space.
Run the oracle before the system. If a fixed-size cache could not be enriched, no amount of engineering would have saved this.
What the fuser actually does
The C2C module is small and sits between a Sharer and a Receiver, with both models frozen. Only the fuser trains. It has three parts:
- Projection. Concatenate the Receiver's KV cache with the Sharer's, push the result through a projection layer, then a feature fusion layer.
- Dynamic weighting. An input-aware head modulation layer reweights the projected information per input.
- A learnable gate. A trainable per-layer value decides whether to inject the Sharer's context at all, implemented as a Gumbel-sigmoid with temperature annealed linearly from 1.0 to 0.001 over training.
Training is unremarkable in a good way: 500k OpenHermes-2.5 samples, batch size 256, 1,929 steps, learning rate 1e-4, standard next-token prediction loss on the Receiver's responses.
Both models stay frozen. You are training an adapter between two fixed systems, not fine-tuning either of them.
Making mismatched models line up
Two practical problems had to be solved, and the solutions are refreshingly unglamorous.
Different tokenizers. Structural template sections are aligned by padding the shorter side with <pad> until lengths match. Content tokens are handled by decoding each target token back to its string and re-encoding it with the Sharer's tokenizer. When that produces a one-to-many mapping, they use maximal-coverage selection: decode every candidate, measure its string length, keep the longest.
Different depths. Layers are matched by terminal alignment: pair the final layers first, then the penultimate pair, working backwards until the shallower model runs out. A depth-normalised alternative was tested and lost.
Terminal alignment winning is a real hint. The layers that matter for transfer are the late, abstract ones, not the proportionally equivalent ones.
The ablation says fusion, not projection
The component breakdown is stark, averaged over four benchmarks:
- Projection only: 20.70%.
- Plus fusion (residual): 44.88%, a gain of 24.18 points.
- Plus the gate, the full method: 47.95%, a further 3.07.
Projection alone is not merely weak, it is far worse than the Receiver running by itself. Dropping a foreign cache into a model, even a well-projected one, actively destroys it. The residual fusion path is what makes the transfer survivable, and the gate is a refinement on top.
Most of this method is not the clever projection. It is the residual connection that lets the target ignore what it cannot use.
The gate reveals what a partner is actually for
This is the finding I would build on. The learned gate behaves completely differently depending on the training regime:
- General-purpose training (OpenHermes-2.5): gates stay almost fully open, with average activation above 98.21%.
- Task-specific training (MMLU): activation drops to 52.67%.
On broad, open-ended work the Receiver wants everything its partner has. On a narrow domain task it wants roughly half, and learns to shut the rest off.
The gate is a measurement instrument, not just a control. Its sparsity tells you how much of a partner's knowledge your task actually needs.
Where the gains quietly go away
Now the parts that deserve more attention than the abstract gives them.
The identical-sharer control. The authors ran C2C with the Sharer being the same model as the Receiver, which carries no complementary knowledge at all:
- No sharer: OpenBook 45.80, ARC-C 47.65, MMLU 36.81, C-Eval 35.81.
- Identical sharer: 50.60, 52.52, 42.17, 40.34.
- Different sharer: 52.60, 54.52, 42.92, 41.77.
Run the arithmetic. A model talking to a copy of itself captures roughly 70% to 88% of the total gain. Only the remainder is attributable to the second model knowing something different.
Long context erodes it. On LongBench, the advantage over text communication falls from +7.17 at 0-4k, to +5.41 at 4-8k, to +0.83 beyond 8k.
And generation length reverses it. This chart ships in the repository under Apache 2.0:
The lower panel measures how much of the gap to the larger model the method recovers. At low length ratios it recovers 60% to 78%. Past a ratio of roughly 4.5, it goes negative, ending near minus 50%.
Average improvement across 5,350 questions is 30.9%.
When the model generates far more than it was given, injected cache stops helping and starts hurting. That boundary is a deployment parameter.
Where this bites
- Most of the benefit is cache capacity, not collaboration. Before building a two-model system, test the single-model version with its own cache enriched, because that is where 70% or more of this lives.
- Both models must be frozen and paired in advance. The fuser is trained for a specific Sharer and Receiver, so this is not a drop-in protocol between arbitrary endpoints.
- The headline numbers vary by source. The abstract claims 6.4 to 14.2% over individual models and about 2.5x latency; the main results table reports 8.5 to 10.5% and 2.0x. Cite the table.
- Gains shrink with context and invert with generation length. Short question, short answer is the regime where this works.
- Larger receivers benefit less, which the authors state directly: stronger baselines overlap more with what the Sharer knows.
- Tokenizer alignment is heuristic. Decode-and-re-encode with longest-match selection is a reasonable hack, not a guarantee, and it will misfire on unusual vocabularies.
This is a real result with a narrow operating window. The window is short inputs, short outputs, and a genuinely complementary partner.
Final Thoughts
The idea deserves the attention it is getting. Text is an interface we adopted because humans needed to read it, and two models with compatible internals have no such requirement, so a projected cache is a reasonable thing to pass between them.
Before you build a multi-model pipeline around this, run their control: give one model a richer cache of its own and measure. If you recover most of the gain without a second model, you have saved yourself an entire system.
Resources & References
- Cache-to-Cache: Direct Semantic Communication Between Large Language Models (arXiv 2510.03215)
- thu-nics/C2C: official implementation, Apache 2.0
- C2C project page
- Paper page on Hugging Face
Stay in Touch
Short takes and discussions on X
→ https://x.com/sebuzdugan
Practical AI / ML videos on YouTube
→ https://www.youtube.com/@sebuzdugan/
Partnerships & collabs
→ sebuzdugan@gmail.com
Originally published on Medium.
Top comments (0)