- Book: AI That Reads
- The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
The first version of citations in every RAG app is the same. You
number the retrieved chunks, ask the model to cite by number, and
render the numbers as links.
const context = chunks
.map((c, i) => `[${i + 1}] ${c.text}`)
.join("\n\n");
// model returns: "Keys rotate every 90 days [2]."
const cited = chunks[Number(match[1]) - 1];
It works immediately, which is why it survives to production. It
breaks the first time anything upstream changes, and it breaks
silently: citation [2] still resolves, still renders, still links.
It just points at a different chunk than it did yesterday.
What the index is actually keyed to
[2] means "the second element of the array I built for this one
request." That array is a function of the query, the embedding model,
the chunker, the index, and the ranking weights.
Change your chunk size and every position shifts. Add a reranker and
the order changes by design. Re-embed with a new model and the whole
result set is different. Any of those is a routine improvement, and
each one silently repoints every stored citation.
If you persist the answer — and most products do, in a conversation
history — then [2] is now a pointer into an array that no longer
exists. It renders as a link to whatever occupies position two today.
The problem is not that the number is fragile. It is that the number
identifies a position in a transient list rather than a piece of
content.
Key to the content
export type SourceRef = {
docId: string; // stable across re-chunking
contentHash: string; // sha256 of the normalised chunk text
start: number; // char offset into the source document
end: number;
revision: string; // document version at retrieval time
};
Each field earns its place.
docId survives everything. Re-chunk, re-embed, re-rank — the
document is still the document.
contentHash identifies the exact text. If the chunk still exists
anywhere in the corpus, this finds it regardless of which chunk index
it landed in this time.
start/end locate the passage inside the document, so you can
highlight it even if the chunking changed and the hash no longer
matches any current chunk. This is the field that makes citations
degrade gracefully instead of vanishing.
revision records which version of the document was actually read.
Without it, a citation to a paragraph that has since been rewritten
points at text that never supported the claim.
The chunk id itself is deliberately absent. Chunk ids are an artifact
of the ingest run, and a re-ingest regenerates them.
Make an uncited claim not compile
Once refs are stable, the type system can enforce the rule that
matters: every factual sentence carries a source.
export type Cited<T> = { value: T; sources: [SourceRef, ...SourceRef[]] };
export type AnswerSpan =
| { kind: "prose"; text: string }
| { kind: "claim"; text: string; sources: [SourceRef, ...SourceRef[]] };
export type Answer = { spans: AnswerSpan[] };
The non-empty tuple type [SourceRef, ...SourceRef[]] is the whole
mechanism. A claim with sources: [] does not type-check. You
cannot construct one by accident, and you cannot forget to populate
it in a code path added later.
Splitting spans into prose and claim matters because not every
sentence is a factual assertion. "Here is how to rotate a key:" needs
no citation. "Keys expire after 90 days" does. A model that must cite
every sentence produces citation spam; a type that distinguishes the
two lets you require it exactly where it belongs.
Getting refs out of the model
The model cannot emit a contentHash — it never sees one, and asking
it to copy a hex string is asking for a transcription error.
Give it short opaque labels and resolve them yourself.
const labelled = candidates.map((c, i) => ({
label: `s${i + 1}`,
ref: c.ref,
text: c.text,
}));
const byLabel = new Map(labelled.map((l) => [l.label, l.ref]));
const context = labelled
.map((l) => `<source id="${l.label}">\n${l.text}\n</source>`)
.join("\n\n");
The label is scoped to this one request and never leaves it. On the
way back you translate:
const Raw = z.object({
spans: z.array(z.discriminatedUnion("kind", [
z.object({ kind: z.literal("prose"), text: z.string() }),
z.object({
kind: z.literal("claim"),
text: z.string(),
sources: z.array(z.string()).min(1),
}),
])),
});
export function resolve(raw: z.infer<typeof Raw>): Answer {
return {
spans: raw.spans.map((s) => {
if (s.kind === "prose") return s;
const refs = s.sources.map((label) => {
const ref = byLabel.get(label);
if (!ref) throw new UnknownSourceLabel(label);
return ref;
});
return { ...s, sources: refs as [SourceRef, ...SourceRef[]] };
}),
};
}
UnknownSourceLabel is worth throwing rather than dropping. A model
inventing s9 when you supplied six sources is a hallucinated
citation, and that is exactly the thing you built this to catch. Log
it as a distinct metric — the rate tells you something real about
your prompt.
Rendering a citation whose target moved
Storage is only half of it. The render path decides what a reader
sees when the source has changed since the answer was written.
export type Resolution =
| { status: "exact"; chunk: Chunk }
| { status: "moved"; doc: Doc; start: number; end: number }
| { status: "stale"; doc: Doc; revision: string }
| { status: "gone" };
export async function resolveRef(ref: SourceRef): Promise<Resolution> {
const byHash = await findChunkByHash(ref.contentHash);
if (byHash) return { status: "exact", chunk: byHash };
const doc = await findDoc(ref.docId);
if (!doc) return { status: "gone" };
if (doc.revision !== ref.revision) {
return { status: "stale", doc, revision: ref.revision };
}
return { status: "moved", doc, start: ref.start, end: ref.end };
}
Four outcomes, four renderings. exact is a normal link. moved
links to the document with the offsets highlighted — the chunking
changed but the text is where it was. stale links but marks the
citation as pointing to an older revision. gone renders as
non-clickable with a note that the source was removed.
A discriminated union here is doing real work: the UI cannot forget a
case, because a switch without a branch fails the exhaustiveness
check.
function render(r: Resolution) {
switch (r.status) {
case "exact": return <Link chunk={r.chunk} />;
case "moved": return <DocLink doc={r.doc} range={[r.start, r.end]} />;
case "stale": return <DocLink doc={r.doc} badge="older revision" />;
case "gone": return <span title="source removed">[source]</span>;
default: return assertNever(r);
}
}
The test that would have caught it
The bug this post is about never shows up in a unit test of the
retrieval function. It shows up across a re-ingest, which is why
nobody tests it.
it("keeps citations resolvable after re-chunking", async () => {
const answer = await ask("How often do API keys rotate?");
await reingest({ chunkSize: 600 }); // was 1200
for (const span of answer.spans) {
if (span.kind !== "claim") continue;
for (const ref of span.sources) {
const r = await resolveRef(ref);
expect(r.status).not.toBe("gone");
}
}
});
Answer a question, change the chunk size, re-ingest, and assert every
citation still resolves to something. With index-based citations this
fails immediately. With content-keyed refs it passes, which is the
entire point.
The rule
A citation is a reference to content, not to a slot in a list you
built for one request. Key it to the document and the text, record
where it was and which revision you read, and let the render path
decide how to degrade.
Then re-chunking becomes an ingest change instead of a data-integrity
event.
If this was useful
AI That Reads covers grounding
and citation properly — how to carry provenance through retrieval,
what to persist alongside an answer, and how to keep sources
resolvable as the corpus moves underneath you.
The full series is at
xgabriel.com/ai-in-typescript.
Top comments (0)