Short answer: use a tenant-scoped cursor over a stable document snapshot, then chunk and embed each page idempotently; offset pagination is fine for a small backfill, but it becomes a correctness and index-cost trap as a logistics help center grows.
Measure first.
I build RAG features in Python and move them from notebooks into production. For a multi-tenant SaaS help center, the ingestion path is a data pipeline before it is an AI feature: select one tenant, read a bounded page, normalize the text, create chunks, and write vectors with provenance. The retrieval path later applies the same tenant boundary before ranking. Keeping those two paths explicit prevents a page token from quietly turning into cross-tenant data.
Our concrete workload is logistics. A tenant may publish carrier rules, customs notes, and warehouse SOPs while another publishes completely different material. The index has to absorb revisions without duplicating chunks, and the cost decision is mostly about how many vectors and embedding tokens we create.
How can retrieval architecture keep pagination tenant-safe in a SaaS help center?
Start with a snapshot, not a moving list. At the beginning of a run, record a high-water mark such as updated_at <= 2026-09-01T00:00:00Z (or an equivalent immutable revision). Every page query includes tenant_id, that mark, and a deterministic ordering by (updated_at, document_id). The cursor contains the last pair, not a human-visible page number. A new article arriving during the run waits for the next snapshot; an edit after the mark cannot make the current run skip an older row.
The tenant predicate belongs in the storage query and in the vector write key. It is not a filter added after similarity search. I use a compound identity like (tenant_id, document_id, revision, chunk_no), with the revision or content hash deciding whether a write is a no-op. That gives retries a safe target and makes deletion a deliberate operation: mark the old revision inactive, then remove or tombstone its chunks according to the index's consistency model.
A cursor is opaque to callers. Sign it or store it server-side, and bind it to the tenant, snapshot, sort order, and filter hash. If a client sends a cursor created for another tenant, reject it. This is a security property, not a pagination nicety.
The implementation: bounded pages, deterministic chunks, repeatable writes
Below is a small adapter-shaped example. The list_documents and upsert_vectors functions stand in for your database and vector index; their contracts are the important part. The example keeps all code in Python so I can run the same logic in a notebook, an eval harness, and a worker.
The worker has one deliberately boring invariant: it acknowledges a complete vector batch before moving the cursor. That makes a retry safe even if the process dies between requests.
from dataclasses import dataclass
from hashlib import sha256
from typing import Iterable
@dataclass(frozen=True)
class Document:
tenant_id: str
document_id: str
revision: str
updated_at: str
text: str
def list_documents(
tenant_id: str,
snapshot: str,
after: tuple[str, str] | None,
limit: int,
) -> list[Document]:
"""Return rows ordered by (updated_at, document_id)."""
raise NotImplementedError
def chunk_text(text: str, size: int = 420, overlap: int = 60) -> Iterable[str]:
words = text.split()
step = max(1, size - overlap)
for start in range(0, len(words), step):
chunk = " ".join(words[start : start + size]).strip()
if chunk:
yield chunk
if start + size >= len(words):
break
def vector_key(doc: Document, number: int, chunk: str) -> str:
digest = sha256(chunk.encode("utf-8")).hexdigest()[:16]
return f"{doc.tenant_id}:{doc.document_id}:{doc.revision}:{number}:{digest}"
def upsert_vectors(items: list[dict]) -> None:
"""Upsert by key; repeated calls produce the same records."""
raise NotImplementedError
def ingest(tenant_id: str, snapshot: str, page_size: int = 100) -> int:
cursor: tuple[str, str] | None = None
written = 0
while True:
page = list_documents(tenant_id, snapshot, cursor, page_size)
if not page:
return written
batch = []
for doc in page:
for number, chunk in enumerate(chunk_text(doc.text)):
batch.append({
"key": vector_key(doc, number, chunk),
"tenant_id": doc.tenant_id,
"document_id": doc.document_id,
"revision": doc.revision,
"chunk_no": number,
"text": chunk,
})
upsert_vectors(batch)
written += len(batch)
cursor = (page[-1].updated_at, page[-1].document_id)
# A worker stores this snapshot once, then resumes with the last cursor.
written = ingest(tenant_id="tenant_42", snapshot="2026-09-01T00:00:00Z")
print({"vectors_written": written})
The loop intentionally advances from the last returned row. A broken backend that returns rows in a different order will duplicate or skip work, so I assert ordering in the adapter tests. I also cap page_size; a giant page lowers request overhead but spikes memory, embedding concurrency, and retry blast radius. There is no universal best value. Measure it with your document mix.
Chunk size is a retrieval decision with a cost shadow. A 420-word window with 60 words of overlap is only a starting hypothesis for SOPs; tables, step lists, and legal clauses may need structure-aware boundaries. I preserve heading, source URL, locale, and ACL metadata beside each chunk. The generator then filters by tenant and authorization before similarity scoring, and the answer cites the document revision it actually saw.
Where does pagination fail, and how do I catch it?
Offset pagination looks simple: offset=page * limit. Under concurrent edits, an insertion before the offset shifts every later row, so one document can be processed twice while another is missed. Large offsets can also force a database to walk discarded rows. Cursor pagination avoids those specific problems only when the sort key is stable and unique; a timestamp alone is not unique enough.
Deletion is another sharp edge. If a tenant removes a customs policy, a fresh crawl that only upserts cannot make the old vector disappear. Keep a manifest of document revisions seen in the snapshot and reconcile anything previously active but absent from that manifest. In practice, I write the manifest row before starting page one, append the document revision and chunk count only after the vector batch is acknowledged, and close the manifest with a source-row total. A reconciliation worker then compares the closed manifest with the active index for that tenant and revision family, marks missing revisions inactive, and records how many keys changed. It never guesses from an empty page, because an empty page can mean either a completed cursor or a transient read problem in the adapter. Treat reconciliation as part of the run, with a dry-run count, an auditable tenant id, and an operator-visible run status. This extra bookkeeping costs a few rows of metadata, yet it is cheaper than serving a deleted warehouse procedure to a dispatcher who trusts the answer.
I once assumed a successful worker exit meant a complete index. It did not: a queue retry happened after the page cursor had been checkpointed, leaving a gap in the embedding batch. The repair was boring but effective: checkpoint the cursor only after the vector write acknowledges every item, and store a run ledger with page boundaries, counts, and checksums. A worker restart then replays one page safely instead of guessing.
Short checks help.
Keep them visible. For each run, compare source rows, chunk count, upsert acknowledgements, and active-vector count. Sample retrieval queries per tenant, including an empty-result case and a document edited at the snapshot boundary. My eval set includes expected source IDs, not just answer text, because a fluent answer grounded in the wrong tenant is a failure.
How should teams balance index cost, freshness, and answer quality?
Cost at scale is a multiplication, not a single embedding price: documents times chunks per document times revisions retained, plus query-time reranking and storage overhead. Reducing overlap from 60 to 20 words may cut vectors, yet it can remove the context that makes a shipping exception understandable. I would first measure recall on a labeled set, then test chunk and overlap variants under the same tenant distribution. Keep the cheapest configuration that stays above your factuality and citation thresholds.
Freshness deserves its own budget. A nightly full snapshot is easy to reason about; high-change tenants may need an incremental stream keyed by revision. Mixing the two without a reconciliation rule creates stale vectors. A practical compromise is a frequent incremental pass and a slower, complete audit that proves no active document was skipped.
The architecture is not suitable when a tenant has only a handful of static pages and no search latency requirement; a relational full-text index may be easier to operate. It is also a poor fit when ACLs change more often than content unless authorization metadata is versioned and tested as carefully as the text. Stick with a simpler store when your evaluation set cannot show a retrieval benefit.
I'm not sure a single chunking policy will survive every language and document format. Your mileage may vary. That uncertainty is exactly why I keep a small, versioned eval corpus and make the chunker a replaceable function rather than baking its assumptions into the database schema.
A release checklist that stays useful after launch
Before promoting a new ingestion worker, I verify that every query and write carries the tenant identity, cursors are bound to their snapshot and filter, and writes are idempotent on a revision-aware key. I replay a boundary edit, a deletion, a retry after checkpointing, and a tenant with zero documents. Metrics expose page latency, rows read, chunks produced, embedding tokens, upsert failures, and reconciliation counts without logging private article text by default.
Then I run retrieval evals by tenant, compare source-ID recall and citation accuracy, and inspect the tail of the cost distribution rather than only the average. A canary run on a few representative logistics tenants gives better evidence than a synthetic million-row benchmark. If the canary changes answer quality or cost beyond the agreed threshold, the worker stops before the full index is touched.
That is the decision rule I keep: a cursor is valuable because it makes a bounded snapshot observable and restartable, not because it is fashionable. Pagination, chunking, and authorization must be tested as one retrieval architecture.
Top comments (0)