The page says the property-management wiki assistant is returning two versions of the same move-out procedure. On-call sees nearly identical passages with different chunk IDs and the same source record ID. The least complex correction is a two-phase incremental reindex: identify changed records, delete every old chunk by stable source ID, then upsert the complete new chunk set.
TL;DR: make replacement, rather than patching, the unit of indexing. Carry a stable document_id through every chunk, reject stale work with a monotonic source version, and expose separate freshness and duplicate-result signals. This protects retrieval quality without putting re-embedding or cleanup on the query path.
The immediate action is to remove the stale version from serving. Then work backward: find why two generations became visible, add the signal that should have warned earlier, and set a threshold whose false positives will not train on-call to ignore it.
What should have paged before duplicate answers did?
A raw chunk count is weak evidence. A lease addendum can legitimately gain chunks, while an access procedure can lose them. The earlier signal is an invariant violation: for one stable document_id, more than one committed source version is visible. A second signal compares the source system's latest version with the latest searchable version.
The page should carry the document ID, expected version, visible versions, indexing job ID, and lag age. Avoid tenant names or document text in metric labels; keep those in access-controlled logs correlated by opaque IDs. For the near-duplicate detector, also track result sets containing two chunks with the same source ID. That retrieval-quality symptom can catch commit-visibility and filtering errors that worker-success counters miss.
One bad result merits investigation. It does not automatically merit a page.
How should an incremental reindex delete and upsert changed documents?
The change feed needs stable identity and a version. A content hash can skip unchanged work, but it is a poor primary identity because it changes with the content. A property wiki record should retain its document_id while its title, body, or metadata evolves.
This Go contract deletes all prior chunks and writes the complete new generation in one transaction. Deterministic chunk keys make a retry of the same version converge on the same rows.
package reindex
import (
"context"
"crypto/sha256"
"fmt"
)
type Chunk struct {
Text string
Embedding []float32
}
type Tx interface {
CurrentVersion(context.Context, string) (int64, error)
DeleteDocument(context.Context, string) error
UpsertChunk(context.Context, string, string, int64, Chunk) error
Commit() error
Rollback() error
}
type Store interface { Begin(context.Context) (Tx, error) }
func Replace(ctx context.Context, store Store, id string, version int64, chunks []Chunk) (err error) {
tx, err := store.Begin(ctx)
if err != nil { return err }
defer func() { if err != nil { _ = tx.Rollback() } }()
current, err := tx.CurrentVersion(ctx, id)
if err != nil { return err }
if version <= current { return tx.Rollback() }
if err = tx.DeleteDocument(ctx, id); err != nil { return err }
for position, chunk := range chunks {
sum := sha256.Sum256([]byte(fmt.Sprintf("%s:%d:%d", id, version, position)))
key := fmt.Sprintf("%x", sum)
if err = tx.UpsertChunk(ctx, key, id, version, chunk); err != nil { return err }
}
return tx.Commit()
}
There is a deliberate trade-off. Delete-then-upsert is easy to reason about only when readers cannot observe the gap. Postgres can hide both steps behind one commit. If a serving index lacks that boundary, write the new generation under a version, mark it ready, switch an active-version pointer, and remove the old generation later. Logical replacement stays atomic even when physical cleanup is asynchronous.
A source tombstone should delete by document_id and advance the recorded version. Otherwise, a delayed update can restore content intentionally removed. Monotonic versions are part of the correctness contract, not incidental queue metadata.
Instrument the gap, not merely worker failures
A worker can report success while a stale event overwrites newer content or retrieval reads both generations. Measure the states users depend on: the proportion of changed documents whose latest version becomes retrievable within the freshness objective, and the proportion of sampled queries containing no superseded generation. Keep query latency separate, because moving embedding work onto requests may improve freshness while damaging response time.
Capacity planning starts with arrival rate. Let C be changed documents per second at peak, K mean chunks per changed document, and E embedding throughput in chunks per second per worker. The lower bound is ceil(C*K/E) workers, before headroom, retries, and write limits. Measure these inputs on the real corpus; inspection reports and lease templates have different chunk distributions. No universal worker count is defensible.
Backlog age is more useful than queue depth when documents vary in size. Page on a threatened freshness objective, then use queue depth and stage latency to locate extraction, embedding, or commit pressure.
Buy, build, or preserve the boundary
The choice is about transaction semantics, on-call load, and exit cost.
| Concern | Managed index | Self-operated index | Required boundary |
|---|---|---|---|
| Atomic replacement | May expose bulk or namespace operations | Can use a storage transaction | Test that generations are never jointly visible |
| On-call load | More infrastructure is delegated | Platform owns capacity and recovery | Keep replay and reconciliation in application code |
| Lock-in | Native filters can spread | Operational coupling replaces API coupling | Preserve generic IDs, versions, and chunk records |
| Capacity | Service limits govern bursts | Worker and storage limits govern bursts | Test changed documents per second |
A managed service can reduce infrastructure duty, while self-operation can expose tighter transaction control; neither fact chooses a winner. Require the ability to replay an ordered change log into a clean index, compare generations, and switch implementations without rewriting identity rules. Otherwise the index has become the system of record by accident.
The buy-versus-build boundary also has to account for alert quality. An over-sensitive page is the closing failure mode. A semantic near-duplicate may be legitimate: two buildings can publish similar emergency procedures but differ in address or escalation contact. Automatic deletion based only on vector similarity risks erasing meaningful records. Similarity nominates candidates; source identity and version determine replacement.
Use a sustained-window breach for paging, calibrated from measured embedding and commit latency. Treat a duplicate-generation invariant violation more urgently than a mild backlog increase because the former can already affect answers. Record maintenance mode explicitly instead of weakening the invariant.
The operational rule is strict: replace by identity, reject stale versions, and alert on user-visible freshness or duplicate generations. A threshold firing on every harmless queue ripple has a real cost: engineers silence it, and the next useful alert arrives only after the assistant quotes an obsolete access instruction.
Further reading
References:
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks: https://arxiv.org/abs/2005.11401
Top comments (0)