DEV Community

Matt Leibel
Matt Leibel

Posted on Fully Autonomous

Why your AI forgets details in long chats, and how to fix it

You give your AI a detail early on, a budget, a deadline, "we're vegetarian", and twenty messages later the answer ignores it. It isn't only between chats. In one long chat, details get dropped every few prompts.

Why it happens

Every new message sends the whole conversation back to the model. As a chat grows, early details sit far back in a long input, and models are worse at using what's buried in the middle of one. Long chats also get expensive: message 50 re-reads everything before it, so they burn through plan limits fast.

Three fixes

1. Restate the details. Put your key details in one message and ask the AI to restate them every so often. Free, but you have to remember to do it.

2. Start fresh with a summary. Ask for a summary, open a new chat, paste it in. Cheap, but summaries drop things.

3. Keep a memory outside the chat. Save the chat's state (the goal, current values, decisions, open items, drafts word for word) and let a fresh chat continue from it, looking up older details only when needed.

What we measured

We ran a 40-message planning chat with Claude Sonnet, then asked 48 questions about it in a fresh chat:

Way to continue Right Tokens per answer
Keep going in the long chat 48 of 48 32,539
Paste a summary 47 of 48 2,736
Saved state (RAMen) 48 of 48 5,478
Nothing 4 of 48 32

The summary is cheapest but missed a detail. The saved state got everything right on 83% fewer tokens than staying in the long chat.

The tool

I built RAMen to do fix 3 for you, across apps: one memory shared by ChatGPT, Claude, Claude Code and local models, with a receipt for every note (your words, the date, the app). Long chats can continue in a fresh one on far fewer tokens, and RAMen Desktop trims long chats on local models too (up to 70% fewer tokens in our tests).

There's a free plan, and Pro is $1.99 a month: https://getramenai.com. How we test: https://getramenai.com/how-it-works.html

How do you deal with this today?


Disclosure: I'm the maker of RAMen. This post was drafted by AI from our test results.

Top comments (0)