HyDE: The RAG Trick Where You Answer the Question BEFORE You Search

Here's a counterintuitive RAG trick: to find the right document, answer the question first — then search with the answer. It's called HyDE (Hypothetical Document Embeddings), and the wildest part is that the answer can be wrong and it still works.

👻 Interactive demo: https://dev48v.infy.uk/prompt/day9-hyde.html

This is Day 9 of my PromptFromZero series.

The problem: questions don't look like answers

Embeddings place text near other text that's worded similarly. But a question and its answer barely share words:

• Question: "Why won't my code build?"
• Answer doc: "A missing dependency in the lockfile is the most common cause of a failed compile…"

Almost no overlap. So the question's embedding lands in a different neighbourhood than the doc that answers it, and plain semantic search misses the best result.

HyDE's bet: generate the answer, embed THAT

Before searching, ask the LLM to write a plausible answer to the question. That hypothetical passage is long, specific, and answer-shaped — so its embedding lands right where real answers live.

const hypo = await llm(`Write a short passage that answers this question. Be specific.
Question: ${query}`);

const vec = await embed(hypo);   // ← embed the fake answer, not the question
const docs = await vectorSearch(vec, { k: 5 });

The surprising part: a wrong answer still helps

The hypothetical can contain factual mistakes and HyDE still improves retrieval. Why? You never show it to the user. It's only a search probe. What matters is that it's in the right style and topic, so it sits near the genuine documents in vector space. Shape beats accuracy here.

Answer from the REAL docs

You search with the fake answer's vector, get back genuine passages, then generate the final response grounded only in those real docs. The hypothetical is discarded:

return llm(`Answer using ONLY these:\n${docs}\nQuestion: ${query}`);

So you get better retrieval without the LLM's hallucinations leaking into the output.

Make it robust: ensemble the probes

A single guess can be a dud. Generate 3–5 hypotheticals, embed each, and average the vectors — like ensembling, it smooths out a bad probe:

const vec = mean(hypos.map(embed));

When to reach for it

HyDE shines when your queries are short and your documents are long, prose-heavy answers (support docs, wikis, research). It costs one extra LLM call per query — usually worth it when retrieval quality matters.

Play with the demo — pick a question and watch the true answer climb from the bottom of the list to the top once HyDE's hypothetical does the searching.