A retrieval index is a cache, and every other cache in your stack has an invalidation protocol. HTTP has ETag; your CDN has a purge API. An index has a cron job and a hope, and how wrong it is comes down to two numbers: how often its answers change, and how often you look.
// a document scanned every 1/r hours, answers changing Poisson at rate mu
S(x) = 1 - (1 - Math.exp(-x)) / x, x = mu / r
// NOT "half the refresh period" - that answers a different question
S agrees with a from-scratch numeric integral of its own definition to 2e-11 and with the simulation to 0.0028 across seven budgets. Sum it over a corpus and notice there are two ways to average. Your dashboard averages per document; your users average per query, and queries are Zipf. At the defaults the same index is 98.8% fresh and ships 3.04% stale answers, separated by the query distribution's inverse participation ratio - 12.5 at Zipf 1.0 over 300 documents, one line of SQL you have never run: https://dev48.infy.uk/ai/days/day70-rag-index-freshness.html
The second averaging trap is worth the whole page. Hold the corpus, the edits per hour, the budget and the bill fixed, and change only which documents get edited.
| edits follow popularity | stale answers / 1,000 | what the dashboard reports |
|---|---|---|
| independent of queries | 9.1 | 1.25% stale |
| tracking queries | 57.3 | 0.92% stale |
Six times worse, and the number you watch moves the wrong way while it happens. Popular documents are popular because they are about live things, and live things are what people edit. That correlation appears in no vector-store setting and no eval harness.
The policy ranking inverts on the same axis. Proportional-to-change-rate is the worst option at zero correlation, 12.7 against 9.1 for a flat cadence - Cho and Garcia-Molina's 2000 crawler result reproduced on a retrieval index - and the best at full correlation, 16.7 against that same cadence's 57.3. A corpus property decides the ordering of your options.
The recommendation I set out to make lost twice
Solve the allocation properly: a per-document Lagrangian over four modes, water-filled until the bill hits the budget. It wins its own objective, 8.0 against the heuristic's 9.9 predicted stale copies, and ships 17.7 stale answers per thousand against a one-line sqrt(w*lambda) rule's 10.5. Sixty-nine per cent worse, with a p99 error age of 199.6 hours against 5.6. The objective is separable per document; a fact with a copy in a second document is not, and 66.2% of what the optimum ships comes from duplicates.
The obvious repair lost too. Put a floor sweep under every webhook and it predicts the best number on the page, 5.8, and ships 32.7 - worse than the thing it was fixing, because the floor comes out of the same budget and starves the sweeps doing the work. What works is deleting the mode: sweeps only ships 8.3.
And a perfect webhook still ships 35.5 stale answers per thousand, because the event fires for the document that changed and the stale copy sits in a document that did not. Break ties on chunk recency in the reranker and the same configuration ships 0.0. It is the only lever here that costs nothing per hour; 94% of the embedding bill is fixed by your writers anyway.
Caught a real NaN bug mid-build, now pinned by an assertion. 111 verifier assertions, 33 in the page, five worlds of 3,000 queries, not one asset fetched off the page: https://dev48.infy.uk/aifromzero.php
Top comments (0)