Short answer: paginate a tenant-scoped, immutable retrieval snapshot, then rerank a bounded candidate set once and carry citation lineage through every page. That decision keeps page boundaries stable while the help-center index changes, which is the condition needed for grounded answers.
The unit of work is a retrieval run, not an HTTP page. Normalize the question, bind it to the authenticated tenant and access policy, retrieve candidates, rerank them, and assign a deterministic order before returning page one. A later request slices that same order. Re-running retrieval for every page feels fresh, but it creates gaps, duplicates, and citations that point at a different revision. This also gives the platform team one place to account for work: candidate retrieval, reranking, snapshot storage, and generation can each have a budget, and the run can be rejected before it consumes the rest of the latency SLO when the candidate cap or tenant quota is already exhausted. That is a capacity decision, not a pagination trick.
I treat a cursor like a distributed job handoff. It is state, not authority. A client can decode ordinary base64, so the token must be signed and its tenant, query fingerprint, policy version, snapshot ID, boundary, page size, and expiry must be checked server-side.
How should pagination shape retrieval architecture for a multi-tenant SaaS help center?
Use two orders. The snapshot order controls continuity; the answer order controls how citations are displayed. Never use a displayed citation number as a pagination key. Every item should retain document ID, document revision, chunk ID, and the exact source span supplied to generation. The Retrieval-Augmented Generation paper describes generation conditioned on retrieved passages; preserving that passage identity across pages is what makes the citation claim auditable.
Choose a stable tie-breaker such as chunk ID after the relevance score. If a reranker has any nondeterminism, persist the resulting ordered IDs for the cursor lifetime rather than calculating page two again. A 50-item page is a practical example, not a universal target; capacity planning should set the candidate cap from labeled relevance tests, memory headroom, and the latency budget for the retrieval SLO.
Small boundary, big blast radius.
Measure it.
The snapshot gives consistency, not unlimited freshness. A newly published article may wait until the next query, while current authorization still has to be applied on each continuation. The catch is that this design is not suitable when revoked content must disappear from an already-running answer immediately; in that case, recheck policy per page and accept that a page can shrink.
What signal exposes citation drift before users report it?
The useful alert names an invariant violation, not a vague “pagination failed.” Track overlap between pages, citations outside the authorized result set, revision mismatches, scope-rejected cursors, expired snapshots, and normal exhaustion as separate outcomes. An exhausted result set is healthy; combining it with errors makes an SLO dashboard lie.
Consider a trace for tenant acme-17. Page one ends at chunk billing:rev-8:0042, and page two starts with the same ID. The trace should show the query hash, policy version, snapshot revision, candidate count, page size, cursor age, returned-ID digest, and cited-ID digest. It does not need the customer's full question or document text in routine metrics. Put replayable detail in access-controlled traces with a retention policy that matches the data classification.
The first check is the cursor invariant: the next ID must be strictly after the stored boundary in snapshot order. If that holds, compare the cited revision with the retrieved revision. If both hold, inspect the answer assembly. This ordering keeps an indexing or authorization defect from being misdiagnosed as a model-quality problem.
A signed Go cursor keeps the boundary explicit
The token below is opaque to callers but explicit to the service. HMAC-SHA256 protects integrity; a key ID allows rotation. The decoder validates structural limits before the caller resolves a snapshot.
package cursor
import (
"crypto/hmac"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"errors"
"strings"
"time"
)
type Payload struct {
Version int `json:"v"`
KeyID string `json:"kid"`
TenantID string `json:"tenant"`
QueryHash string `json:"query_hash"`
Policy string `json:"policy"`
SnapshotID string `json:"snapshot"`
NextOffset int `json:"next_offset"`
PageSize int `json:"page_size"`
ExpiresAtUnix int64 `json:"expires_at"`
}
var (
ErrMalformed = errors.New("malformed cursor")
ErrSignature = errors.New("invalid cursor signature")
ErrExpired = errors.New("expired cursor")
)
func Encode(p Payload, key []byte) (string, error) {
body, err := json.Marshal(p)
if err != nil { return "", err }
mac := hmac.New(sha256.New, key)
_, _ = mac.Write(body)
enc := base64.RawURLEncoding
return enc.EncodeToString(body) + "." + enc.EncodeToString(mac.Sum(nil)), nil
}
func Decode(token string, key []byte, now time.Time) (Payload, error) {
var p Payload
parts := strings.Split(token, ".")
if len(parts) != 2 { return p, ErrMalformed }
enc := base64.RawURLEncoding
body, err := enc.DecodeString(parts[0])
if err != nil { return p, ErrMalformed }
sig, err := enc.DecodeString(parts[1])
if err != nil { return p, ErrMalformed }
mac := hmac.New(sha256.New, key)
_, _ = mac.Write(body)
if !hmac.Equal(sig, mac.Sum(nil)) { return p, ErrSignature }
if err := json.Unmarshal(body, &p); err != nil { return p, ErrMalformed }
if p.Version != 1 || p.PageSize < 1 || p.PageSize > 50 || p.NextOffset < 0 {
return p, ErrMalformed
}
if now.Unix() >= p.ExpiresAtUnix { return p, ErrExpired }
return p, nil
}
After decoding, compare TenantID, QueryHash, and Policy with values derived from the current request. Resolve SnapshotID, take the slice at NextOffset, filter it through current authorization, and sign the advanced offset. Because the snapshot is immutable, this offset is not an offset into a changing search index.
Continuation must be idempotent. A network retry can submit the same cursor twice, and both responses should describe the same slice while the snapshot is available. Do not consume a single-use token before the response is durable unless the protocol also has a request idempotency key and replay store.
Verification, rollout, and rollback are part of the design
Test malformed signatures, unknown versions, exact-boundary expiry, tenant and query mismatches, and page sizes 0, 1, 50, and 51. Property tests can generate unique ordered IDs, paginate with random legal sizes, and assert that concatenating pages reproduces the snapshot exactly once. An integration test should update the live index after page one and verify that page two still reads the original snapshot while applying current authorization.
Relevance tests belong beside cursor tests. Use a labeled set from the real help-center intents, evaluate retrieval and reranking before generation, and verify that each answer claim is supported by one cited span. Pagination is correct only when changing the boundary does not change that evidence contract.
Roll out with shadow traces, then a small tenant cohort, then broader traffic. Watch duplicate-result rate, cursor rejection by reason, empty continuation pages, snapshot lookup latency, and citation validity. A rollback should stop issuing the new cursor version while retaining its verification key until outstanding tokens expire.
Set alert thresholds against an actionable SLO. One cross-tenant result is an immediate containment event; one expired cursor usually is not. I'm not sure a single citation-coverage threshold transfers from account recovery to configuration help, because acceptable evidence differs by intent class. Per-intent labels and review are what would resolve that uncertainty.
The operational rule is simple: freeze the evidence order for a traversal, recheck authorization, and make every boundary observable. Stick with fresh per-page retrieval only when freshness is more important than continuity and citation reproducibility.
References
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks: https://arxiv.org/abs/2005.11401
- Go
crypto/hmacpackage documentation: https://pkg.go.dev/crypto/hmac - Go
encoding/base64package documentation: https://pkg.go.dev/encoding/base64
Top comments (0)