The hard part of AI agent memory isn't retrieval, it's delivery

mathew_woo — Wed, 22 Jul 2026 12:58:56 +0000

I've been building WOS, a long-term memory API for AI agents, and I want to write down the one thing that reframed how I think about the problem.

When people say "give the AI memory," it sounds like the work should mostly be search: store what happened, pull the relevant bits back later. And retrieval on its own is close to solved. On LongMemEval, the retrieval step finds the right memory well over 99% of the time.

The Real Bottleneck

But end-to-end accuracy—actually answering the question correctly using that memory—sits a lot lower, around 85 to 90%.

So the model has the right memory in hand and still gets the answer wrong. The gap isn't in finding the memory. It's in delivering it back in a form the model can actually use, especially for questions that span many sessions or depend on time ("what did I say last month", "has this changed since").

That gap is basically what WOS is built around.

Architectural Trade-offs

A few things that follow from it:

Bring your own key: It's a retrieval API, not a model. You bring your own LLM key, and WOS only charges for the retrieval tokens. Your model, your data.
Privacy & Control: It works across languages, not just English. It doesn't train on your data, and you can wipe a user's entire memory in one call.
Tiered Performance: There's a lineup with different tradeoffs. The lightest tier (Tablet) does retrieval without calling a model at all, so it stays cheap and fast (~320ms). Heavier tiers add an LLM query layer for harder multi-session questions and score higher on the same benchmark.

One honest caveat: those tiers were measured with different reader models, so I wouldn't read the numbers as a clean apples-to-apples ranking. The papers say the same.

Benchmarks and write-ups are available at wontopos.com if you want the details.

Curious how others handle this. If you've given an agent long-term memory, where does it break for you—the retrieval step, or the part where the model has to use what it got back?

Anyone know prior research on rewriting a rough prompt per model?

mathew_woo — Wed, 22 Jul 2026 02:29:20 +0000

I've been kicking around an idea and want to read up before I build it.

The idea: you type a rough, half-formed prompt, an LLM works out what you actually mean, and then rewrites it into a specific, optimized prompt tuned for each target model (Claude, GPT, Gemini, etc.).

The Core Insight: The same instruction lands differently on different models. I don't want a single generic "cleaned-up" prompt—I want one explicitly tailored per model.

What I'm Looking For

Before I start building, I'd love to find prior work. Specifically, I'm looking for:

Automatic prompt optimization or prompt rewriting techniques.
Research on adapting a prompt to a specific target model's quirks and strengths rather than producing a generically better prompt.

Is this already well-studied and I'm just searching the wrong terms?

If you've run across papers, GitHub repositories, or blog posts tackling model-specific prompt adaptation, please drop them in the comments below! What keywords should I actually be searching for?

DEV Community: mathew_woo

The hard part of AI agent memory isn't retrieval, it's delivery

The Real Bottleneck

Architectural Trade-offs

Anyone know prior research on rewriting a rough prompt per model?

What I'm Looking For