What nDCG sees that precision and recall miss

Two retrieval runs for the same query. Relevant document is D.

Run A:  [D, x, x, x, x, x, x, x]
Run B:  [x, x, x, x, x, x, x, D]

Precision@8 is identical: one relevant document out of eight, 0.125 both times. Recall@8 is identical: you found the one that existed, 1.0 both times. Hit rate is 1.0 both times.

Every set-based metric says these runs are the same. For a RAG system they are not.

Why position is most of the quality

Retrieval feeds a context window, and a context window is ordered and finite.

Take the top 4 chunks and run A includes D while run B does not. Recall@8 was 1.0 and your actual recall at the size you use is 0. The metric measured a depth you never read from.

Even when everything fits, position matters. Models attend unevenly across long contexts, and material buried in the middle of a large window gets used less reliably than material at the top. The practical effect today is that rank order is part of your quality, and a metric that ignores it ignores the part you can most easily improve.

What nDCG does

Discounted cumulative gain gives each hit a value that shrinks with position, using a logarithm so the penalty is steep near the top and gentle further down:

top.forEach((id, i) => {
  if (relevantSet.has(id)) dcg += 1 / Math.log2(i + 2);
});

Position 1 contributes 1 / log2(2) = 1.0. Position 2 contributes 1 / log2(3) ≈ 0.63. Position 8 contributes 1 / log2(9) ≈ 0.32.

That shape matches how people and models use ranked results. The gap between first and second is worth more than the gap between seventh and eighth.

The n is the normalisation. Raw DCG is not comparable across queries, because a query with five relevant documents can score higher than one with a single relevant document without being better retrieval. So you divide by the best achievable ordering for that query:

let idcg = 0;
const ideal = Math.min(relevant.length, k);
for (let i = 0; i < ideal; i++) idcg += 1 / Math.log2(i + 2);

return idcg === 0 ? 0 : dcg / idcg;

Every relevant document first, as many as could fit in k. The result is 1.0 for perfect ordering and comparable across queries with different numbers of relevant documents.

For the two runs above, A scores 1.0 and B scores about 0.32, which is the difference precision and recall could not see.

Which metric answers which question

They are not competing. A report should have all of them.

Metric	The question it answers
hit_rate	Did we find anything at all? The floor below which nothing else matters.
recall@k	Could the model possibly have got it right? It cannot cite what was never fetched.
precision@k	How much noise is in the context window? This is what predicts hallucination.
mrr	How far down is the first useful thing?
ndcg@k	Is the good material near the top, or merely present?

Read them in that order, and fix them in that order. Hit rate first, since ranking is irrelevant if you are not retrieving anything relevant. Then recall, which bounds everything downstream. Then nDCG, usually where the cheapest wins are, because reranking is a smaller change than reindexing.

Two traps in reporting them

Report k and use the k you actually read. recall@100 is a fine diagnostic and a bad headline if your prompt takes 5 chunks.

Do not average away a metric with missing inputs. A case with no relevance labels has no meaningful recall, and scoring it zero drags the average down for a labelling gap.

All five are in ragbench, computed deterministically with no model in the loop, so the same dataset and predictions give the same numbers on any machine. The nDCG test asserts exactly the property this article is about: a document found first scores higher than the same document found last, while precision and recall report both as identical.