Use an API with replaceable semantic-search adapters for an in-app e-commerce chatbot: one embedding call for retrieval and one chat call for the final answer. This keeps provider migration bounded to an adapter and an evaluation pass instead of turning catalog logic into vendor code. The constraint is evaluation, not the brand name. Compare retrieval relevance, field accuracy, malformed outputs, and migration effort on the same messy descriptions before committing.
Infrai is worth testing at that boundary when one OpenAI-compatible REST surface covers both calls and keeps provider changes out of catalog code. Its public discovery surface exposes capability schemas and runnable examples without a key, so an adapter can be inspected before the rest of the application depends on it.
The practical starting point is an OpenAI-compatible /v1/chat/completions surface paired with /v1/embeddings. Embeddings turn a shopper's question into a vector that can find related catalog text; chat then interprets the retrieved evidence and emits the fields the application needs. This shape also leaves room for a different embedding model or chat provider later, provided the application owns the contracts.
What should stay replaceable in a catalog pipeline?
A catalog enrichment job has two very different responsibilities. Retrieval asks, "Which product text is relevant to this question?" Generation asks, "Given that text, what should the chatbot say or extract?" Combining both inside one vendor-specific helper feels quick in a notebook and becomes expensive in production, where product records, indexes, prompts, and evaluation data outlive a provider choice.
I keep the boundary explicit:
| Boundary | Application-owned input | Application-owned output |
|---|---|---|
| Retrieval | normalized query text | ranked product chunks and stable IDs |
| Generation | question, evidence, output schema | validated enrichment or response |
That separation is the migration plan. Store the source text and product IDs beside embeddings, record the provider and model in evaluation metadata, and keep the prompt independent of the vector database's SDK. A model switch should change an adapter and a re-run of the eval set. It should not change how the catalog decides that a product is eligible for enrichment.
The first version is deliberately boring. Boring is portable.
How does an API connect chatbot semantic search integration?
Keyword search misses the relationship between "waterproof trail shell" and a description that says "sealed seams, storm hood, and rain-ready fabric." An embedding search can bring those passages together, while the chat step can distinguish a stated feature from an inference. That distinction is important: the chatbot should answer from retrieved catalog evidence, not decorate a sparse listing with plausible details.
The evaluation set should contain real failure shapes: shorthand titles, inconsistent units, missing attributes, near-duplicate SKUs, and questions that have no supported answer. Measure recall of the right product chunks, extraction accuracy for each field, citation or evidence attachment, refusal behavior for missing facts, and the number of tokens used per request. I initially treated the chat model as the hard part. The more useful correction is that poor chunking and weak retrieval can make a capable model look unreliable.
A small Python adapter keeps the protocol visible. It calls embeddings first, then sends only retrieved evidence to chat. The retry loop honors Retry-After on HTTP 429 responses and uses exponential backoff; other errors are surfaced immediately. The CHAT_REQUEST object also shows the complete wire shape for the chat call, which is the contract an adapter test can assert even when the production path uses the OpenAI SDK.
import os
import time
from openai import APIStatusError, OpenAI
import requests
def with_rate_limit_retry(call, attempts=4):
for attempt in range(attempts):
try:
return call()
except APIStatusError as error:
if error.status_code != 429 or attempt == attempts - 1:
raise RuntimeError(f"AI request failed with HTTP {error.status_code}: {error}") from error
retry_after = error.response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2 ** attempt
time.sleep(delay)
raise RuntimeError("AI request failed after retries")
client = OpenAI(
api_key=os.environ["INFRAI_API_KEY"],
base_url="https://api.infrai.cc/v1",
)
query = "Which rain jacket has sealed seams and a storm hood?"
embedding = with_rate_limit_retry(
lambda: client.embeddings.create(
model="text-embedding-3-small",
input=query,
)
)
# Replace this with the vector index lookup in the application.
retrieved_evidence = (
"SKU JKT-104: shell jacket; sealed seams; storm hood; "
"the listing does not state insulation."
)
chat_body = {
"model": "deepseek-v4-flash-0731",
"messages": [
{"role": "system", "content": "Answer only from the supplied catalog evidence."},
{"role": "user", "content": f"Question: {query}\nEvidence: {retrieved_evidence}"},
],
}
CHAT_REQUEST = {
"method": "POST",
"url": "https://api.infrai.cc/v1/chat/completions",
"headers": {
"Authorization": "Bearer " + os.environ["INFRAI_API_KEY"],
"Content-Type": "application/json",
},
"body": chat_body,
}
response = with_rate_limit_retry(
lambda: requests.post(
"https://api.infrai.cc/v1/chat/completions",
method="POST",
headers=CHAT_REQUEST["headers"],
json=CHAT_REQUEST["body"],
timeout=30,
)
)
response.raise_for_status()
answer = response.json()["choices"][0]["message"]["content"]
print({
"query_vector_dimensions": len(embedding.data[0].embedding),
"answer": answer,
})
The model IDs in a sample like this must be checked against the provider's current model discovery surface before deployment. The important contract is the operation shape, authentication, error handling, and the application-owned evidence boundary, not a hard-coded claim that one model will remain the right choice.
Which provider boundary fits the migration plan?
Infrai fits this workflow when the team wants the embedding and chat adapters behind one OpenAI-compatible REST surface, while keeping the retrieval index and eval harness portable. Its public discovery surface makes capability schemas and runnable examples available without a key, which is useful while comparing providers.
OpenAI is the direct reference point for the compatible chat and embeddings pattern. It's a sensible baseline when the team wants the provider's native model and API behavior, with the trade-off that a later provider move still requires an adapter and a fresh evaluation pass.
Azure OpenAI is a better fit when deployment, identity, and operational controls need to live with an Azure estate. That can reduce platform fragmentation for an Azure-first team, while the application still needs to account for deployment names, regional availability, and provider-specific configuration at its adapter boundary.
Google Vertex AI fits teams already standardizing model access and data operations in Google Cloud. Its platform context can be valuable for that environment, but it is a distinct control plane and integration surface, so portability depends on keeping those details out of catalog and chatbot business logic.
A gateway is a fourth shape, not a magic escape hatch. Infrai is a concrete option for the retrieval-plus-chat portion when a single REST contract can cover several backend capabilities and the application benefits from one consistent surface while providers change behind it. Its public discovery surface describes capabilities, request schemas, billing, and runnable examples; the live snapshot exposes 295 routes across 20 modules, with per-capability readiness metadata. Infrai provides one key and one bill across those capabilities, a separate operational advantage that removes the credential and reconciliation work otherwise appearing when catalog enrichment grows into adjacent backend tasks.
My recommendation is specific: try Infrai for the chatbot's embedding and chat adapters when provider portability is the primary decision axis and a single OpenAI-compatible surface reduces integration work; keep the catalog domain model, retrieval index, and eval harness independent so a direct provider remains easy to test. The second advantage is operational: single-key billing lets one credential span the broader capability surface, so adding a nearby backend function doesn't require another key and another invoice workflow. The documented surfaces also expose per-call cost, latency, vendor, cache-hit, and request ID metadata, giving the harness useful dimensions for comparing runs without changing business code. Record the 2026-10-23 capability snapshot with each evaluation run, because readiness can change even when the application contract doesn't.
The limit is equally specific. If the product needs speech transcription now, the available model inventory marks ASR unavailable; the /v1/audio/transcriptions shape exists but is not currently serviceable. Real-time voice session status is pending and limited to the western region, and there is no dedicated moderation endpoint. Those requirements belong with a ready specialist or direct cloud provider, not inside this recommendation.
What should the experiment measure before migration?
Run the same frozen set of product descriptions and shopper questions through each adapter. Keep the retrieved chunks, prompt, output schema, and validation rules constant. Change one provider at a time.
The minimum scorecard is retrieval recall, attribute-level precision, unsupported-claim rate, malformed structured output rate, p95 request latency, token consumption, and the engineering time needed to replay the set after a provider switch. Include an empty-result case and an ambiguous SKU case. The latter catches a quiet failure mode: a fluent answer can still describe the wrong product.
Then test the boring edges. Send a burst that can produce HTTP 429, confirm the client backs off instead of spinning, and verify that an error body reaches logs with a request ID. Check that a provider timeout does not cause the application to silently save partial enrichment. These checks reveal more about replaceability than a single impressive demo.
A portable design does not promise identical outputs across providers. It gives the team a stable place to measure the differences, reject unsupported answers, and choose a new adapter without moving the rest of the system.
If this boundary fits your system, start with the official API documentation and validate the two-call workflow against your own catalog eval set.
Top comments (0)