Every "AI personalization" demo looks the same. Someone types a name and a company into a box, an LLM spits out three paragraphs of warm, specific-sounding prose, and the room nods. Then the email goes out and the prospect replies "who is this and how do you know I switched CRMs?" because the model didn't know that, it guessed, and it guessed right by accident.
We wrote about this problem at a high level in "How AI Is Quietly Rewiring Outbound", where the short version was: personalization isn't a prompting problem, it's a retrieval problem. This post is the long version, specifically the retrieval half of that pipeline.
*The Core Split: Generation vs. Grounding
*
There are two very different jobs hiding inside "write me a personalized email":
- Figure out what's true and relevant about this person right now.
- Turn those facts into a sentence a human would enjoy reading.
An LLM is genuinely good at job two. It is a liability at job one, because a language model's job is to produce plausible text, and "plausible" and "true" are not the same target. Ask it to name a mutual connection or a recent funding round with no grounding, and it will happen to be right sometimes and confidently wrong the rest of the time. Confidently wrong is worse than generic, because generic doesn't torch trust.
So the fix isn't a smarter prompt. It's refusing to let the model touch job one at all.
What actually goes in the retrieval layer
Before any text gets generated, we assemble a small, verified context bundle for the prospect. In practice that means pulling from a handful of signal categories:
- Recent public posts or interviews (what they're currently thinking about)
- Funding or hiring events (is the company in growth mode)
- Tech stack signals (what tools they've adopted or dropped)
- Role and title changes (are they new to this seat)
- Product or feature launches (something worth congratulating, if genuine)
Each of these gets embedded and stored as vectors, tagged with a source and a timestamp. At generation time, we retrieve only the handful of chunks relevant to this specific email, not the prospect's entire history. The LLM prompt then looks less like "write a cold email to Jane" and more like "write a cold email to Jane, using only these five verified facts, and cite which fact backs each claim."
That last part matters more than the retrieval step itself.
Freshness kills more personalization than bad writing does
A funding round from fourteen months ago isn't a signal anymore, it's a stale fact that makes the email read like it was written by someone who hasn't checked in for a year. This is where a lot of "AI personalization" tools quietly fail: they treat the vector store as a write-once archive instead of something that needs a decay function.
The practical fix is boring: every retrieved chunk carries a timestamp, and the retrieval layer applies a recency weight on top of similarity score. A highly relevant fact from ten months ago should often lose to a moderately relevant fact from ten days ago. Getting this ranking right is more of a tuning problem than an architecture problem, but skipping it entirely is how you end up congratulating someone on a role they left.
Entity resolution: the unglamorous failure mode
The other quiet failure mode is merging two different people who happen to share a name, or worse, merging a company with its acquirer and attributing the acquirer's funding news to the smaller company. None of this is an LLM problem. It's a data-matching problem that happens upstream, before anything gets embedded.
In practice this means every signal needs to carry more than a name string, a LinkedIn URL, a company domain, and a role at time-of-capture all help the resolution layer decide "same person, same company" with actual confidence rather than a fuzzy string match. Get this wrong and no amount of grounding-checking downstream saves you, because the model will faithfully and correctly summarize a fact that was never true of this person in the first place.
What this buys you?
Once retrieval is doing its job, the LLM's role shrinks to something much safer: reason over five to seven verified, timestamped, correctly-attributed facts and write one good paragraph. That's a task language models are actually reliable at. The failure modes that remain (a claim slipping through without a source, a tone that reads as too familiar) become the concern of a separate grounding-verification layer, not the retrieval layer.
Where this post stops?
This post is about getting the right facts in front of the model. It says nothing about how you decide whether this prospect is worth emailing at all, that's a scoring problem, and it says nothing about catching a hallucination that slips past retrieval, that's a verification problem. Both get their own post in this series.
Related
Top comments (0)