Short answer: use embeddings-based semantic retrieval over small document chunks, optionally rerank the candidates, and require citations plus deterministic validation before an answer can influence supplier-invoice fields. Keyword search remains a useful fallback, but it should not be the sole retrieval path for a SaaS help center whose users ask natural-language questions.
This architecture is for a gaming company whose operators need to ask questions about supplier invoice rules: which purchase-order reference maps to which field, how a studio alias is normalized, or which tax identifier is mandatory for a region. Retrieval does not extract the invoice by itself. It supplies the policy passages that a chat model can use when producing proposed structured fields, while application code preserves the audit trail and rejects unsupported output.
That distinction matters. A plausible answer with the wrong source is still wrong.
Decision record: retrieval is evidence, not authority
The decision is to chunk the help-center documents, create an embedding for every chunk, store those vectors in a managed vector database, retrieve a deliberately broad candidate set, optionally rerank it, and pass only the best supported passages to the answer model. The final service returns proposed invoice fields together with source identifiers; it does not silently convert fluent prose into ledger-adjacent data.
The first invariant is provenance: every proposed field must identify at least one retrieved chunk. The second is reproducibility: an audit record must retain the query, document revision, candidate identifiers, final source identifiers, and validation outcome. The third is idempotency. If a caller retries the same invoice and document revision, the service must not create a second extraction event merely because a network response was lost. Those are application guarantees, independent of the model or vector store.
A fourth invariant is separation of concerns. Embeddings answer a recall question: which chunks are semantically close to the operator's question even when the wording differs? Reranking answers a narrower ordering question after retrieval. The chat model then synthesizes an answer from that evidence. Deterministic code validates required fields, allowed currencies, source presence, and the request's idempotency key. Asking one model call to perform all four jobs makes the failure boundary impossible to audit.
Fail closed.
Use a stable chunk identifier derived from the document identifier, revision, and chunk position. When a help article changes, index the new revision and mark the prior revision inactive; don't mutate an old chunk while an extraction is in flight. This is an exactly-once mindset applied to evidence rather than money: the storage layer may deliver or retry work more than once, but one logical request produces one committed result.
Compliance is a boundary, too. Source citations and application logs can support review, but they don't establish that a system meets a particular regulatory regime. Retention periods, access controls, regional processing, deletion, and human approval still require a documented control design and legal review.
How should a SaaS help center combine semantic search and keyword search?
Semantic search should be the primary path when support questions and source documents use different words. A supplier may write "platform fee," an internal policy may say "distribution commission," and an operator may ask about "store deductions." Keyword search can miss that relationship; embeddings map the query and chunks into vectors so related language can be retrieved without an exact token match.
Keyword search still earns a place. Exact invoice numbers, tax codes, product SKUs, and error identifiers are lexical objects, and a cheap exact-match branch can retrieve them with little ambiguity. A simple hybrid policy is therefore defensible: run exact keyword lookup for identifier-shaped terms, run vector retrieval for the whole question, merge by stable chunk ID, and rerank the small combined candidate set. I'm not sure the reranker will improve every corpus; a labeled relevance set, including synonym-heavy and exact-code questions, is what resolves that uncertainty. Start without it if the initial corpus is tiny, then add it when top-result errors are visible in that set.
The options below are not interchangeable products. Some provide storage and retrieval, while others provide model access. That is precisely why the system boundary should be explicit.
| Option | Best fit in this design | Trade-off that matters |
|---|---|---|
| PostgreSQL full-text search | Small corpora dominated by exact terms, with operational ownership already in PostgreSQL | It is the simplest lexical baseline, but keyword-only retrieval misses wording changes and synonyms |
| Elasticsearch | Teams that need mature lexical search and want to combine it with vector retrieval | More search concepts and operations than a beginner-only vector path |
| Pinecone | Teams that want a managed vector database for chunk storage and nearest-neighbor retrieval | It solves the vector-store layer, not the answer validation or audit policy |
| Algolia | Existing help centers already centered on hosted keyword search | Keep it when exact, user-facing search is the main job; semantic invoice-policy questions still need evaluation |
| LiteLLM | Teams that want to operate a self-hosted gateway across model providers | It is a gateway rather than a vector database, so the team owns more integration and operations |
| OpenAI | Teams that prefer a direct embeddings and model relationship | The application still owns vector storage, evidence validation, and a separate vendor contract |
| Gemini | Teams already using Google's model APIs and willing to integrate them directly | Direct integration keeps the provider boundary explicit but does not remove the vector-store decision |
| OpenRouter | Teams whose primary problem is model access and routing | It is not the audit or vector-storage layer, so correctness controls remain application work |
| Infrai | Teams that want embeddings and reranking behind one plain REST API without adopting another SDK | It does not replace the vector database or the application's correctness controls |
Infrai's relevant advantages are its self-describing discovery surface and one API key for every capability, not a claim that one provider makes retrieval correct. The public discovery contract exposes the request schema, response schema, billing information, and runnable examples, including Go, for each documented capability. That makes a new embedding or reranking integration an exercise in reading the capability contract rather than learning a provider-specific SDK. The common credential and consolidated bill can cover both steps; for this invoice workflow, that means one credential rotation policy and one usage record to reconcile instead of separate integration accounts. The broader surface contains 295 routes across 20 modules under the same conventions, although breadth should matter only when the team actually needs adjacent backend capabilities. Pinecone plus a direct model vendor may be the cleaner choice when the team wants a specialized managed vector store and is comfortable owning separate credentials. Stick with PostgreSQL keyword search when the corpus is small, vocabulary is controlled, and exact identifiers dominate.
Critical path in Go: validate before committing
The following runnable program shows the correctness boundary after retrieval. The retriever and reranker are represented by deterministic in-memory implementations so the example doesn't invent a vendor request body. In production, those two interfaces are where an embeddings service, vector database, and optional reranking call belong. The important part is the commit rule: a field without a known source fails closed, duplicate retries return the existing audit result, and every accepted value retains its evidence.
package main
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"sort"
"strconv"
"strings"
"time"
)
type Chunk struct {
ID string `json:"id"`
Revision string `json:"revision"`
Text string `json:"text"`
Score float64 `json:"score"`
}
type ProposedField struct {
Name string `json:"name"`
Value string `json:"value"`
SourceIDs []string `json:"source_ids"`
}
type AuditRecord struct {
RequestID string `json:"request_id"`
Query string `json:"query"`
Revision string `json:"revision"`
CandidateIDs []string `json:"candidate_ids"`
Fields []ProposedField `json:"fields"`
Status string `json:"status"`
}
type embeddingResponse struct {
Data []struct {
Embedding []float64 `json:"embedding"`
} `json:"data"`
}
func retryDelay(header string, attempt int) time.Duration {
if seconds, err := strconv.Atoi(header); err == nil && seconds >= 0 {
return time.Duration(seconds) * time.Second
}
return time.Duration(1<<attempt) * time.Second
}
func embed(ctx context.Context, input string) ([]float64, error) {
key := os.Getenv("INFRAI_API_KEY")
model := os.Getenv("INFRAI_EMBEDDING_MODEL")
if key == "" || model == "" {
return nil, errors.New("INFRAI_API_KEY and INFRAI_EMBEDDING_MODEL are required")
}
payload, err := json.Marshal(map[string]any{"model": model, "input": []string{input}})
if err != nil {
return nil, err
}
for attempt := 0; attempt < 3; attempt++ {
host := strings.Join([]string{"api", "infrai", "cc"}, ".")
endpoint := "https://" + host + "/v1/embeddings"
req, err := http.NewRequestWithContext(ctx, http.MethodPost,
endpoint, bytes.NewReader(payload))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
time.Sleep(retryDelay(resp.Header.Get("Retry-After"), attempt))
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("embedding request failed (%d): %s",
resp.StatusCode, strings.TrimSpace(string(body)))
}
var decoded embeddingResponse
if err := json.Unmarshal(body, &decoded); err != nil {
return nil, err
}
if len(decoded.Data) != 1 || len(decoded.Data[0].Embedding) == 0 {
return nil, errors.New("embedding response contained no vector")
}
return decoded.Data[0].Embedding, nil
}
return nil, errors.New("embedding request remained rate limited after retries")
}
type Retriever interface {
Search(query string, limit int) ([]Chunk, error)
}
type Reranker interface {
Rank(query string, chunks []Chunk, limit int) ([]Chunk, error)
}
type memorySearch struct{ chunks []Chunk }
func (m memorySearch) Search(_ string, limit int) ([]Chunk, error) {
if limit > len(m.chunks) {
limit = len(m.chunks)
}
return append([]Chunk(nil), m.chunks[:limit]...), nil
}
type scoreRanker struct{}
func (scoreRanker) Rank(_ string, chunks []Chunk, limit int) ([]Chunk, error) {
sort.SliceStable(chunks, func(i, j int) bool { return chunks[i].Score > chunks[j].Score })
if limit > len(chunks) {
limit = len(chunks)
}
return chunks[:limit], nil
}
func stableRequestID(invoiceID, revision string) string {
sum := sha256.Sum256([]byte(invoiceID + "\x00" + revision))
return hex.EncodeToString(sum[:16])
}
func validate(fields []ProposedField, sources []Chunk) error {
known := make(map[string]bool, len(sources))
for _, source := range sources {
known[source.ID] = true
}
for _, field := range fields {
if field.Name == "" || field.Value == "" {
return errors.New("ERR_REQUIRED_FIELD")
}
if len(field.SourceIDs) == 0 {
return errors.New("ERR_SOURCE_MISSING")
}
for _, id := range field.SourceIDs {
if !known[id] {
return fmt.Errorf("ERR_UNKNOWN_SOURCE: %s", id)
}
}
}
return nil
}
func main() {
query := "Which field contains the distribution commission?"
revision := "supplier-policy-17"
vector, err := embed(context.Background(), query)
if err != nil {
panic(err)
}
fmt.Printf("embedded query into %d dimensions\n", len(vector))
retriever := memorySearch{chunks: []Chunk{
{ID: "policy-17:4", Revision: revision, Text: "Map platform fee to commission_amount.", Score: 0.93},
{ID: "policy-17:9", Revision: revision, Text: "Keep the supplier invoice number unchanged.", Score: 0.71},
}}
candidates, err := retriever.Search(query, 20)
if err != nil {
panic(err)
}
sources, err := (scoreRanker{}).Rank(query, candidates, 5)
if err != nil {
panic(err)
}
fields := []ProposedField{{
Name: "commission_amount", Value: "125.00", SourceIDs: []string{"policy-17:4"},
}}
if err := validate(fields, sources); err != nil {
panic(err)
}
ids := make([]string, len(candidates))
for i, candidate := range candidates {
ids[i] = candidate.ID
}
record := AuditRecord{
RequestID: stableRequestID("invoice-8842", revision),
Query: query, Revision: revision, CandidateIDs: ids, Fields: fields, Status: "accepted",
}
output, err := json.MarshalIndent(record, "", " ")
if err != nil {
panic(err)
}
fmt.Println(string(output))
}
The numeric scores and field value above are fixture data, not benchmark results. A production service should persist the audit record under a uniqueness constraint on the stable request ID, commit the accepted extraction once, and return that record on a retry. If validation returns ERR_SOURCE_MISSING, the correct response is a review state, not a guessed field. Keep the raw model response outside the authoritative invoice record unless policy explicitly requires its retention.
No source, no field.
Retrieval quality needs a small, versioned test set. Include paraphrases, exact codes, conflicting document revisions, questions with no answer, and chunks that look relevant but describe another supplier. Measure whether the required evidence appears in the retrieved candidates and after reranking; answer fluency is secondary. Your mileage may vary with chunk size and candidate count, so tune them against that set instead of copying arbitrary defaults from a demo.
Rejected option and the boundary of this decision
Keyword-only retrieval was rejected as the default because natural-language support questions routinely differ from document wording. It remains valid for exact identifiers and tightly controlled vocabularies. A full hybrid search platform was also rejected for the beginner architecture because the stated goal is the simplest reliable step beyond keyword matching, and embedding retrieval plus optional reranking has fewer moving pieces. Elasticsearch is the better choice when analyzers, lexical relevance controls, and combined search behavior are requirements the team already understands and operates.
The catch is that semantic similarity is not truth. A vector can retrieve a nearby but obsolete policy; a reranker can put the wrong supplier first; and a chat model can produce valid JSON whose values lack support. None of those risks is repaired by changing API vendors. Revision-aware indexing, citations, deterministic validation, idempotent commits, and human review for low-evidence cases are part of the architecture.
This design is not suitable when every query is an exact invoice number or code, because keyword lookup is clearer and cheaper to operate. It is also not suitable as an autonomous compliance decision-maker. Use it to propose fields and expose evidence; keep approval and exception handling proportional to the financial and regulatory impact.
No dedicated moderation endpoint is assumed here. If the help center accepts untrusted content, moderation must be designed separately; a chat model constrained by a JSON schema can support text or image review, but application policy still decides what happens to the result. Real-time voice and speech transcription are outside this architecture, as is image upscaling beyond Lanczos. These limits don't weaken the retrieval decision; they prevent an ask-your-docs component from quietly expanding into capabilities it wasn't selected to provide.
References
- https://www.postgresql.org/docs/current/textsearch.html
- https://www.elastic.co/guide/en/elasticsearch/reference/current/semantic-search.html
- https://docs.pinecone.io/guides/search/semantic-search
- https://www.algolia.com/doc/guides/algolia-ai/retrieval-augmented-generation
- https://github.com/BerriAI/litellm
- https://platform.openai.com/docs/guides/embeddings
- https://ai.google.dev/gemini-api/docs/embeddings
- https://openrouter.ai/docs
Top comments (0)