For the past while, I’ve been developing Mnemara, a tool designed to handle state injection by pinning specific rows within a conversation. The idea was simple: inject state into a pinned turn row, and have it automatically evict old data and inject new data as the conversation progressed.
It felt powerful. It felt like I was giving the model a dynamic memory bank. But after extensive testing, I’ve hit a wall that every LLM developer eventually faces: The Reasoning Ceiling.
The Problem: Pinned Rows vs. Small Model "Brain Power"
What I discovered is that while high-end models (GPT-4o, Claude 3.5) could handle mid-context state injections with ease, smaller models (Llama 3, Mistral, etc.) struggled.
Smaller models suffer from what I (Gemini) calls "Contextual Friction." When you inject state mid-stream, it creates a discontinuity. The model's attention mechanism gets caught between the "Narrative Flow" (what just happened) and the "Injected State" (the hard facts). Often, the narrative flow wins, and the model ignores the state entirely.
The Pivot: The Turn 0 State Block
I am officially shifting Mnemara’s architecture. Instead of pinning rows throughout the chat, I am moving everything to Turn 0 (the System Prompt/Initial Message).
Why this works better:
Primacy Bias: LLMs naturally pay the most attention to the very beginning of the context window. By placing state at Turn 0, it becomes the "Physical Law" of the session rather than a "suggestion" buried in the middle.
Markdown as a Source of Truth: Using structured Markdown and natural language at the top of the prompt provides a clear "Header" for the model to reference before it even reads the user's latest message.
Consistency Across Tiers: This method is "model-agnostic." It makes small models punch way above their weight class and makes large models nearly infallible.
What’s Next for Mnemara?
The tool will now function as a Mutable Header Manager. It will:
Extract state updates from the ongoing dialogue.
Rewrite the Turn 0 block with updated natural language/Markdown state.
Ensure the "Source of Truth" is always at the top, even as older conversation history is evicted.
If you’re building for the "small model" ecosystem, stop fighting the attention mechanism. Lead with the state, and let the conversation follow.
Context and concept provided by mekickdemons-creator.
Refined, sharpened, and formatted by Gemini.
After I get more claude usage, I'll make due on my word here. It's a good force multiplier and I've tested it. Expect a little more from Mnemara and Mnemara-Gemma soon. Personally I'm going to start hashing everything, a friend let me in on the secret there.
Top comments (1)
Hey dear