This article was created with the help of AI. The AXI architecture is by Rajnish Choubey. The tests were built and run by Shalendra Gaur.
A fair objection to my graph tests: a graph is rigid and easy to check, which is the easy case for a layer that holds the state. Real work is ambiguous text: facts in many wordings, corrections, loose references like "the last one we added", and a lot of talk that changes nothing. So I ran that.
The test
- Model: Gemini 3.8 Flash with its default thinking. Run on 22 October 2026, one seed, one run of each arm.
- Job: a conversation of 400 turns in ordinary sentences about a hidden list of items for a small events desk. An item has a name and up to four facts: owner, day, amount and status. Every turn came with a paragraph of 30 sentences of office chatter that mentions other days, other amounts and the same first names, and changes nothing.
- Two arms. The kept chat sent every turn and every reply again, and each reply was applied as it came. With AXI, the host held the exact list, showed it at every step, and checked each reply before it touched the list (up to 2 re-asks). Nothing was kept in the chat.
- The plan and 24 yes/no lines, each with a probability, were hash-locked before the run.
What came out
| Chat that keeps its history | AXI (state held on the host) | |
|---|---|---|
| Held list exactly right | wrong for 140 turns in a row from turn 191; right on 65.0% of turns | 400 of 400 steps (100.0%) |
| Prompt tokens a step, on average | 99,809 | 1,177 |
| Tokens sent in all | 39.9 million | 0.47 million |
| Time to first token | 1.86 s for turns 1 to 50, 5.57 s once the prompt passed 150,000 tokens | 2.41 s |
- The kept chat did not drift on this text. Its replies were exactly right on 398 of 400 turns (99.5%), and its largest prompt was 198,405 tokens. Of the 24 lines, 15 read YES and 9 read NO, and most of the NO lines are the kept chat not doing what I had guessed it might.
- The two slips mattered differently. The slip at turn 370 was a wrong answer to a question and changed nothing. The slip at turn 191 changed the wrong item in the list. Applied as each reply came, the held list stayed wrong for 140 turns in a row (turns 191 to 330), until a later message happened to overwrite that cell. So the list was exact on 65.0% of the turns although 99.5% of the replies were exact.
- With AXI the held list was exact on all 400 steps, at 1,177 prompt tokens a step on average (946 in the first tenth, 1,332 in the last, as the list grew) and 2.76 s a step.
- Size and time. The kept chat sent 85 times as many prompt tokens. Its time to first token rose from 1.86 s to 5.57 s once the prompt passed 150,000 tokens, against 2.41 s with AXI (2.31 times). At a short prompt the kept chat was the faster of the two.
What this does not show
- It is one model, one job, one seed and one run of each arm. It is 400 turns and up to 198,405 tokens, shorter than my 457-turn graph run, which reached 949,591.
- It does not show a kept chat drifting or hallucinating at this length. It shows what one slip does to a held state when each reply is applied as it comes. The slip at turn 191 is one event.
- The check in the AXI arm is written for this list: the item must be there for a change and absent for an add, and the field and the value must be in the allowed set.
AXI is a layer on the host side by Rajnish Choubey: the model is unchanged and the claims are about the state the host holds. The test page is docs/GEMINI_FUZZY_TEST.md in the repository: https://github.com/gaurshalendra2412/axi-runtime
White paper, "AXI: The Cure for the Cost Crisis in AI" (Rajnish Choubey and Shalendra Gaur): https://doi.org/10.5281/zenodo.23958737
Top comments (0)