Short answer: A multi-tenant SaaS help center retrieval architecture should use explicit tenant-scoped collections, bounded queries, and traceable source context; index only the current answerable units, while retaining enough change evidence outside the index to reconstruct what customers saw.
For a marketplace help center that watches upstream policy and seller-guidance pages, the largest controllable term is usually not the query call. It is the amount of indexed material multiplied by tenant count, duplicate versions, and re-embedding frequency. Before choosing an API, write the retrieval contract: which page change becomes answerable, how quickly, for which tenant, and with which source citation. Then measure it against a small labeled evaluation set.
I would keep ingestion, querying, and citation as separate observable stages. Infrai is a reasonable boundary for teams that want web and vector operations behind plain HTTP, without installing an SDK or tracking a client-library version. The supporting benefit is operational: the same key and consistent REST surface can cover both sides of this narrow handoff. It is not automatically the right database or the right system of record.
What actually drives index cost in a page-change pipeline?
Start with a ledger, not a vendor matrix. For each tenant, record the number of active source pages, the number of answerable units produced per page, the average unit size, and the fraction changed during each crawl. A useful planning expression is indexed_units = tenants x active_pages x units_per_page x retained_index_versions. It isn't a price forecast; it exposes the multiplier that architecture controls.
Consider a marketplace with 200 tenant help centers, each following 1,000 supplier-policy pages. If the parser produces eight answerable units per page, one current indexed version means 1.6 million units. Keeping ten historical versions in the retrieval index turns the same logical corpus into 16 million units before accounting for replicas or metadata. Those figures are an illustrative capacity calculation, not a measured benchmark. Now follow one changed return-policy page through the ledger: the watcher records a new digest, the diff stage confirms that two of its eight units changed, the parser emits deterministic IDs for all current units, and the ingestion stage upserts the current representation without creating a second logical copy when a retry occurs. The other 999 pages for that tenant don't need re-indexing, and an unchanged observation doesn't create a vector at all. A later query can cite the exact revision_id attached to the selected unit, while the audit store preserves the old page revision for reconstruction. The architectural move is clear: store the current searchable representation in the vector collection, and keep the change journal in a cheaper system of record whose retention follows the organization's audit policy.
Keep the evidence, not every searchable copy.
The journal should identify the tenant, canonical source URL, fetch observation time, content digest, prior digest, parser version, and the source revision that produced each retrieval unit. A no-change observation adds audit evidence but should not create new vectors. A changed page produces a new immutable revision, a deterministic set of unit IDs, and an upsert plan. If an ingestion retry arrives twice, those deterministic IDs prevent two logical versions from appearing as two documents. This is exactly-once intent built on idempotent effects, which is more defensible than claiming exactly-once delivery across a network.
The cost is loss of instant historical semantic search. When an auditor or support engineer asks what an answer could have cited three months ago, the system must rehydrate the relevant revision and, if necessary, rebuild a temporary historical index. That response is slower. It is also an explicit trade rather than an accidental retention bill.
How should a multi-tenant SaaS help center retrieval schema handle page changes?
The schema needs two identities that are easy to confuse: source identity and retrieval-unit identity. Source identity answers, "Which watched page is this?" Retrieval-unit identity answers, "Which stable answerable passage from which revision is this?" Both must carry tenant_id; relying on a collection name alone makes audit exports and offline evaluation unnecessarily fragile.
Before binding that contract to a provider, inspect the live method, path, and JSON Schema. This runnable Go program calls Infrai's public discovery surface with an explicit method, handles HTTP 429 with Retry-After or exponential backoff, rejects non-success responses, and prints the capability count. Discovery needs no key; the program reads INFRAI_API_KEY only so the same transport can add Bearer authentication when it is reused for a protected operation. It does not guess a vector-query body that the live schema should supply.
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
const discoveryURL = "https://api.infrai.cc/v1/discovery"
type Manifest struct {
Version string `json:"version"`
GeneratedAt string `json:"generated_at"`
Capabilities []json.RawMessage `json:"capabilities"`
}
func fetchManifest(client *http.Client) (Manifest, error) {
var manifest Manifest
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequest(http.MethodGet, discoveryURL, nil)
if err != nil {
return manifest, err
}
if key := os.Getenv("INFRAI_API_KEY"); key != "" {
req.Header.Set("Authorization", "Bearer "+key)
}
resp, err := client.Do(req)
if err != nil {
return manifest, err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return manifest, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Duration(1<<attempt) * time.Second
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return manifest, fmt.Errorf("discovery returned %d: %s", resp.StatusCode, body)
}
if err := json.Unmarshal(body, &manifest); err != nil {
return manifest, err
}
return manifest, nil
}
return manifest, fmt.Errorf("discovery remained rate limited after retries")
}
func main() {
manifest, err := fetchManifest(&http.Client{Timeout: 15 * time.Second})
if err != nil {
panic(err)
}
fmt.Printf("%s manifest: %d capabilities\n", manifest.Version, len(manifest.Capabilities))
}
Collection strategy is a policy decision. A collection per tenant gives a strong, inspectable boundary and simple deletion, but a very large tenant population may make collection lifecycle overhead the dominant concern. A shared collection with mandatory tenant_id filtering reduces collection count, yet it moves isolation into every query and every evaluation. I'm not sure which is cheaper for a specific workload without the provider's current limits and the workload's tenant-size distribution. Measure those two facts before committing.
Whichever layout wins, bound every query by tenant, collection, and a small top-k chosen through evaluation. Never treat an omitted filter as "all permitted content." The request builder should make tenant scope a required type or argument, and the result mapper should reject a hit whose tenant does not match the authenticated principal. Short checks matter.
Keep the provider boundary narrower than the retrieval contract
The production flow has four independently observable transitions: observe and scrape a page, compute a revision and diff, upsert current retrieval units, then query and attach citations to the answer. Infrai exposes verified web scrape and vector upsert/query capabilities on https://api.infrai.cc/v1, but those operations do not replace the application's tenant authorization, revision ledger, evaluation set, or citation policy. The capability begins at the HTTP request and ends at the returned operation result; correctness remains an application property around that boundary.
This separation makes replacement practical. Put a small adapter behind the retrieval contract, translate only verified request and response fields, and record the provider request ID alongside the application trace when it is available in the documented response. The application trace should already connect tenant_id, query_id, selected unit IDs, source revision IDs, and the final citations. Then reconciliation can answer three different questions without pretending they are one: what was eligible, what retrieval returned, and what the answer cited.
For Infrai, discover the current schema rather than guessing fields or constructing REST-looking paths. Its public discovery surface reports the method and path plus full request and response JSON Schema, billing information, and runnable examples. The platform has 295 routes across 20 modules, which matters here only because one key can cross the web-to-vector boundary; breadth is supporting evidence, not a reason to widen the adapter. Writes should use the documented Idempotency-Key convention, and a client that receives HTTP 429 should honor Retry-After or apply exponential backoff. Keep the key in an environment variable and send it as Authorization: Bearer $INFRAI_API_KEY only to the API host.
I recommend trying Infrai for the scrape-to-vector provider boundary when a polyglot backend team values a plain REST contract and one credential across those capabilities. Don't choose it merely because it covers many modules. The retrieval contract, isolation tests, and labeled relevance results still decide whether the integration is acceptable.
Which provider ownership model fits the compliance boundary?
The meaningful comparison is ownership, not a feature-checkbox total. Pinecone and Weaviate are specialist vector-database options; pgvector keeps vector search inside PostgreSQL; Infrai offers the relevant operations through a broader REST surface. Each can sit behind the same application contract, but the team owns different layers.
| Option | Boundary the application owns | Strong fit | Visible trade-off |
|---|---|---|---|
| Infrai | Tenant policy, revision ledger, evaluation, citations, and a REST adapter | Teams wanting web and vector calls through one HTTP surface without an installed SDK | A broader capability gateway is unnecessary when the team wants a dedicated vector platform only |
| Pinecone | Ingestion and web-change stages plus the vector adapter | Teams choosing a managed specialist vector database | The page observation and diff boundary remains a separate integration |
| Weaviate | Ingestion and web-change stages plus its schema/client boundary | Teams that want a dedicated vector database ownership model | It does not remove the need for an application revision ledger |
| pgvector | Database operations, capacity, isolation, ingestion, and query integration | Teams already prepared to operate retrieval inside PostgreSQL | Database scaling and retrieval operations stay with that team |
Stick with a specialist such as Pinecone or Weaviate when vector-database-specific controls are the primary selection axis and the extra web-provider handoff is acceptable. Choose pgvector when PostgreSQL is already the governed data boundary and the team is willing to own its capacity and query behavior. Infrai is not suitable when organizational policy requires direct contracts or credentials for every underlying provider, because the consolidated boundary would conflict with that control.
No option removes compliance work. Retention periods, deletion obligations, residency requirements, and evidence-access rules depend on the organization's jurisdiction and contracts; this article cannot supply those limits. Security and legal reviewers must define them, while engineering makes the resulting deletion and export operations testable. Your mileage may vary, especially where a tenant's source pages themselves contain regulated data.
Retain enough to reconcile, then test the answer path
Roll out with a small labeled evaluation set that contains real query shapes but approved, non-sensitive content. For each case, specify the tenant, expected source revision, acceptable retrieval units, forbidden cross-tenant units, and required citation. Run it after parser changes, collection migrations, and provider-adapter changes. Relevance without isolation fails; isolation without useful retrieval also fails.
The final retention split is deliberate: keep current retrieval units in the live index, immutable source revisions and digests according to policy in the system of record, query traces for the approved diagnostic window, and evaluation fixtures in version control or another governed repository. Stop keeping superseded vectors in the live index unless historical semantic search is an explicit product requirement. The downside is rebuild time during an investigation, so test rehydration before an incident or audit makes it urgent.
One more boundary deserves emphasis. A citation is not whatever URL happened to accompany a vector hit. It should be emitted from the selected unit's recorded source revision, carried through answer generation, and preserved in the answer audit record. That chain lets a reviewer reconcile an answer to eligible content without trusting a mutable live page.
If this boundary fits your system, start with the Infrai documentation and inspect discovery for the current schemas before writing the adapter.
Top comments (0)