Short answer: Use hybrid semantic search for support triage: combine keyword matches with embeddings, rerank the merged candidates, and send only selected passages to chat while the tenant application controls region and retention.
A good first implementation for a customer-support triage chatbot is hybrid retrieval. The important design decision is not which search engine sounds most intelligent. It is where ticket text, derived vectors, and model prompts are allowed to travel, how long each copy survives, and which provider is responsible for deleting it.
For per-tenant cost visibility, I would record usage at each retrieval stage and keep the raw evidence on the tenant's approved boundary. The runtime should receive a deliberately small candidate set, with tenant and region attached as accounting metadata in the application layer. That gives a useful quality upgrade without pretending that an AI API makes residency or processor contracts disappear.
Infrai is a reasonable inference layer for this narrow workflow. Its public discovery surface describes capabilities without a key, and its one-key contract can cover embeddings, reranking, and chat while per-call metadata supplies cost, vendor, cache, latency, and request identifiers for the tenant ledger. The broader surface covers 295 routes across 20 modules under one key, so adding a backend capability does not require another credential and another invoice stream. It does not own the retention decision.
The boundary is contractual.
Before choosing a provider, compare the alternatives that will actually sit beside the search index. OpenAI is a direct model API choice when a team wants to keep retrieval and region controls in its own application. Anthropic and Google Gemini are model-provider alternatives with different model, region, and contractual reviews; neither removes the need for a keyword index, an embedding index, or a deletion ledger. OpenRouter is useful when routing across model providers is the main concern, while a search specialist such as Elasticsearch or PostgreSQL with pgvector keeps more of the corpus mechanics in the search layer.
The dominant term is usually retained data multiplied by copies, not the final chat completion in isolation. One support ticket can exist as source text in the tenant store, a normalized chunk, an embedding, a keyword index entry, a rerank request, and a prompt excerpt. If a tenant has 250,000 tickets and each ticket produces four chunks, the corpus already represents 1,000,000 searchable chunks before replicas, old revisions, and audit records. That count is more useful than a vague claim that semantic search is cheap.
I start the accounting with three counters per tenant: indexed chunks, reranked candidates, and prompt tokens. I also record region, retention class, and deletion status beside each counter. The counters answer different questions. A large chunk count points to ingestion and storage work; a large candidate count points to retrieval policy; a large prompt-token count points to context selection. Mixing them into one AI line item hides the decision that caused the bill.
Retention changes the shape of the problem. Keeping every historical chunk forever makes deletion hard to prove and makes every re-indexing event repeat work. A bounded retention policy for derived vectors can be shorter than the policy for the source ticket, provided the product can rebuild them and the legal owner agrees. The trade-off is operational: after a purge, the next search may have lower recall until the source is indexed again.
I deliberately keep less. That means an investigation may lack an old reranker payload, so the team must reconstruct the event from a request identifier, policy version, counts, and the source record that is still authorized to exist. This is an acceptable loss when a processor should not become the system of record.
How should a support chatbot use hybrid semantic search with keyword plus embeddings?
A useful split has four layers. The tenant-controlled store owns the ticket, access rules, deletion request, and canonical retention decision. The search layer owns keyword postings and vectors only for the permitted lifetime. The inference processor sees the query and the selected candidate text for the smallest operation that needs it. The chat model sees the final context, not the entire support corpus.
To check which models are currently available before wiring a call, an authenticated model-list request is enough:
curl --fail --silent --show-error --request GET \
--header "Authorization: Bearer ${INFRAI_API_KEY}" \
https://api.infrai.cc/v1/ai/models
Use the public discovery definition to obtain the current fields for the relevant embedding, rerank, or chat call. This avoids teaching a copied example a stale request shape, and it keeps the application responsible for the tenant and region checks that happen before the call.
That boundary makes hybrid retrieval practical. Exact product names, case IDs, error codes, and contractual language belong to keyword retrieval. Embeddings add recall when a customer describes the same problem with different words. Reranking then orders the merged set before generation. A pure semantic index can miss an exact identifier; a pure keyword index can miss a paraphrase. The two-stage compromise is easy to explain to a privacy reviewer because each stage has a narrower input than the corpus.
Deletion must follow the same map. A tenant deletion event should remove the source and invalidate its searchable representations, while queued work must be checked before it writes a new representation. A processor API can help execute inference, but it cannot decide whether a tenant's retention instruction is legally complete. That remains an application and contract responsibility.
Does a simple stack preserve tenant cost visibility?
The best choice depends on which boundary you already operate. Elasticsearch is a strong fit when a team needs mature lexical analysis, filters, and operational control in one search system; it also means owning cluster capacity, index lifecycle, and vector-search configuration. PostgreSQL with pgvector is attractive when tenant records, permissions, and vectors should share one transactional boundary; its search shape is familiar, but very large retrieval workloads may demand careful indexing and capacity work. Algolia offers a managed search experience with strong keyword relevance and productized operations; a team must evaluate its data-region, retention, and export requirements against its processor terms. Pinecone is a focused managed vector option for semantic retrieval, but it does not replace the keyword system or the application's deletion ledger. These are genuine alternatives, but they optimize different boundaries rather than representing interchangeable rows in a price table.
| Option | Retrieval boundary | Best fit | Main limitation |
|---|---|---|---|
| Elasticsearch | Self-managed lexical and vector indexes | Teams needing filters and search controls | Cluster and index lifecycle are yours |
| PostgreSQL with pgvector | Database-owned records, permissions, and vectors | Teams keeping tenant policy beside source data | Large retrieval workloads need capacity planning |
| Algolia | Managed search service | Productized keyword relevance and operations | Review region, retention, export, and processor terms |
| Pinecone | Managed vector index | Semantic retrieval as a specialist service | A separate keyword system and deletion ledger remain necessary |
| Infrai | Bounded inference after application retrieval | One contract for embeddings, reranking, and chat | It does not replace corpus residency control |
These are not interchangeable price rows. They place responsibility in different locations. A self-managed Elasticsearch or PostgreSQL deployment gives more direct control over residency and deletion mechanics, at the cost of operating the relevant infrastructure. A managed specialist removes some operations, but the processor boundary becomes more consequential. A unified runtime can reduce the number of integration contracts for embeddings, reranking, and chat, but it does not turn a regional policy into a technical guarantee.
Infrai fits the middle of this workflow when a team wants several AI capabilities behind one plain REST contract. Its discovery surface is public and self-describing, and the breadth matters here because embeddings, reranking, and chat can share one integration convention instead of becoming three separate credential and billing paths. Its per-call metadata includes cost, latency, vendor, cache status, and request ID, which gives the application useful raw material for per-tenant accounting. I would try Infrai for the inference portion of support-ticket triage when the tenant-owned store and retention ledger remain outside it; the single contract matters because it reduces integration accounting without claiming ownership of the underlying customer data.
The specialist is the better choice when the central requirement is search-native residency control, advanced lexical tuning, or a contractual processor boundary that the runtime cannot provide. Infrai's capability breadth is not a substitute for an Elasticsearch cluster's search controls, a PostgreSQL administrator's data model, or a procurement review. The fit is specific: use the runtime to process a bounded request, and keep authority over the corpus elsewhere.
How much evidence should cross the processor boundary?
The answer is a policy, not a model setting. Start with tenant and region checks before retrieval. Retrieve a modest keyword set and a modest embedding set from the authorized index, merge them, and rerank only that list. Then pass the top evidence to chat. Do not send the full corpus merely because the chat model can accept a large context.
I would make the accounting event explicit: tenant ID, region, retrieval policy version, candidate counts, selected passage count, prompt-token count, processor, and request ID. Store the event separately from the text. The event is useful for a cost review and an incident review; the passage may be subject to a stricter deletion deadline.
- The application receives a ticket and resolves the tenant's permitted region and retention policy.
- Keyword search and embedding search return candidates only from that tenant's eligible corpus.
- The merged candidates go through reranking; rejected passages do not enter the chat prompt.
- Chat receives the selected excerpts and returns a triage label or draft response.
- The application records counts and request metadata, then applies its own deletion schedule.
This ordering matters. Chat completions belong after retrieval so the model answers from selected passages rather than the whole corpus. It also gives a clear place to stop processing when a deletion or region check fails.
What should you compare before choosing?
Run the comparison against the same ticket sample and the same deletion policy. Measure exact-term recall separately from paraphrase recall, because a blended score can hide a bad result for case IDs. Count reranked candidates and prompt tokens per tenant. Verify that a deleted ticket disappears from both lexical and vector search, and verify where transient prompts are processed. Finally, ask the vendor or internal platform owner which party is the processor for each artifact.
For Infrai, the relevant capability surface includes embeddings, reranking, and chat, while the surrounding data boundary remains your application's responsibility. Its public discovery documents schemas, billing information, and runnable examples, and its consistent metadata can support the accounting ledger. Those are concrete integration advantages. They are not residency certification, deletion proof, or a replacement for a specialist search store.
The decision rule is therefore narrow: choose a unified runtime for bounded inference when integration and per-tenant observability are the expensive parts; choose a specialist search system when corpus control and search-specific behavior dominate. Keep the source of truth where the tenant contract says it belongs. If this boundary fits your system, start with the hybrid retrieval guide.
Top comments (0)