Short answer: choose the embedding model that passes a labeled health-catalog retrieval test, then keep it replaceable behind a provider-neutral interface. Portability matters because a new vector dimension or normalization rule means a new index, not a harmless configuration change.
Before the boundary, catalog code calls a provider SDK and the index becomes coupled to its output. After it, the pipeline is source text -> embedding profile -> vector index -> retrieved evidence. The chatbot gets a reversible decision. The catalog team gets an audit trail.
How should a chatbot use embedding models for semantic search in RAG?
Start with the retrieval contract, not a model name. A messy description might say wireless ECG patch, 7-day wear, adult, while a clinician asks for ambulatory cardiac monitoring for one week. The system should retrieve the right product record and preserve its source wording. A similarity score is not a clinical claim.
Build a labeled set with abbreviations, spelling mistakes, units, brand-neutral descriptions, and exclusions such as pediatric versus adult use. Add hard negatives. A respiratory sensor is a dangerous miss when the query asks for cardiac monitoring. Measure recall at the number of chunks the chatbot actually reads, then inspect false positives. One score hides the failure that matters.
Keep lexical retrieval in the design. Exact identifiers, regulatory terms, and model numbers often belong in a keyword index. Hybrid retrieval can combine lexical and vector candidates before applying filters for region, catalog status, and intended population. Embeddings are a ranking signal, not a permission system.
Which portability trade-offs change the design?
Providers expose different dimensions, limits, languages, and tokenization behavior. An index configured for one dimension cannot accept another without rebuilding or projecting the data. Projection can reduce information and makes evaluation harder to interpret, so a fresh index is the clearer migration boundary.
Store an immutable profile beside every vector: model family, dimension, normalization rule, chunking revision, and timestamp. Never append vectors from two profiles just because both are arrays of numbers. They are not interchangeable coordinates. The trade-off is stark: a clean second index costs a migration, while a mixed index costs trust.
Three public provider families show why this boundary is useful. OpenAI documents embedding variants and dimensions. Cohere documents input-type options for search use cases. Voyage AI documents model-specific embedding endpoints and dimensions. These are factual interface differences, not a ranking. The application should own the contract; an adapter should own the request shape.
A minimal Node.js boundary for catalog enrichment
The adapter can call a hosted service, a local runtime, or a test double without changing catalog code.
type EmbeddingProfile = { id: string; dimensions: number; revision: string };
type Embedder = {
profile: EmbeddingProfile;
embed(texts: string[]): Promise<number[][]>;
};
async function enrichCatalog(texts: string[], e: Embedder) {
const vectors = await e.embed(texts);
if (vectors.length !== texts.length) throw new Error('Embedding count mismatch');
return texts.map((text, i) => {
const vector = vectors[i];
if (vector.length !== e.profile.dimensions) throw new Error('Vector dimension mismatch');
return { text, profileId: e.profile.id, vector };
});
}
Queue failures with bounded retries and an idempotency key derived from the record, normalized text, and profile revision. A retry should not create a second logical product. Record batch size, latency, retry count, provider status class, vector profile, and index-write outcome. Do not log raw health descriptions by default; they may contain sensitive or identifying information.
I would alert on empty-result rate, profile mismatches, and recall on a fixed canary set. A latency chart alone misses a completed request that retrieves the wrong record. That is the operational trap.
An adapter can hide useful provider features. Keep escape hatches at that boundary, where they are testable, rather than leaking provider request objects through every service. If a feature changes the retrieval contract, version the contract deliberately.
Rebuilding an index takes time. Use dual-read migration: write the new profile to a separate index, compare retrieval on the labeled set, then switch reads after the comparison passes. Keep the old index during rollback. A lexical fallback is reasonable when the vector index is unavailable, but label that path in telemetry so a quality regression is explainable.
The decision rule is narrow: select the candidate that meets the healthtech test set and operational constraints, then preserve the ability to replace it. A model is a component. The profile, evidence trail, filters, and retrieval checks are the system.
Keep the migration boring.
For a real cutover, the long part is comparison, not code: replay the labeled queries against both profiles, compare expected product IDs and hard negatives, inspect every disagreement involving population or regulatory language, and record the decision with the profile revision. That sequence gives reviewers something concrete to challenge and gives operators a rollback target if the new index looks healthy but retrieval quality changes.
The boundary is the product decision.
Retrieval-augmented generation should pass retrieved catalog evidence into the answer step. The generation step cannot repair a missing product attribute, and semantic search cannot grant authorization.
Top comments (0)