Short answer: use staged retrieval with explicit collections, bounded queries, and traceable source context. For a healthcare appointment assistant, query a small current collection first, fall back under a fixed deadline, and return a source URL or document identifier with every usable hit.
That decision starts with the answer the patient sees, not with a vector database. “Can I move my appointment?” needs a current rescheduling policy for the right clinic and locale. A timeout should select a narrower fallback or a safe handoff; it must not turn into an unbounded wait.
Write the retrieval contract before tuning vectors
Map each user-visible answer to an intent-specific collection, freshness target, maximum result count, timeout, and evidence fields. Keep appointment operations separate from clinical preparation guidance. A single giant collection lets an old cancellation paragraph outrank a current scheduling rule, and the similarity score will not tell you why.
In Go, I keep those limits in a value that can be reviewed alongside the service SLO:
package retrieval
import "time"
type Contract struct {
Collection string
MaxResults int
Timeout time.Duration
Freshness time.Duration
RequiredEvidence []string
}
var AppointmentPolicy = Contract{
Collection: "appointment_policy_current",
MaxResults: 5,
Timeout: 800 * time.Millisecond,
Freshness: 30 * time.Minute,
RequiredEvidence: []string{"source_url", "document_id"},
}
The values are starting points, not a promise about every clinic. Measure deadline misses and empty-evidence responses, then adjust the contract rather than silently increasing every limit.
One short rule matters: no provenance, no confident answer.
How should semantic retrieval handle healthcare appointment assistant fallbacks?
Use ordered stages with a shared deadline. First query the current intent collection. If it times out or yields too little reviewable context, query a small alias collection. Then try lexical matching over approved titles and identifiers. If all stages fail, return a constrained handoff message and log the reason; do not manufacture an appointment answer.
Every stage should emit the same envelope: candidates, collection, source URL or document ID, elapsed time, and a transition reason. Retry only transient limits, with exponential backoff and Retry-After when supplied. A 429 is a scheduling signal, not an invitation to spin.
package retrieval
import (
"context"
"errors"
"time"
)
type Hit struct {
Text string
SourceURL string
DocumentID string
Stage string
}
type Search func(context.Context, string, time.Duration) ([]Hit, error)
func Staged(ctx context.Context, question string, stages []struct {
Name string
Search Search
}) []Hit {
deadline, ok := ctx.Deadline()
if !ok { return nil }
for _, stage := range stages {
remaining := time.Until(deadline)
if remaining <= 0 { break }
for attempt := 0; attempt < 2; attempt++ {
hits, err := stage.Search(ctx, question, remaining)
if err != nil {
if errors.Is(err, context.DeadlineExceeded) { break }
if attempt == 0 { time.Sleep(50 * time.Millisecond); continue }
break
}
reviewable := make([]Hit, 0, len(hits))
for _, hit := range hits {
if hit.SourceURL != "" || hit.DocumentID != "" {
hit.Stage = stage.Name
reviewable = append(reviewable, hit)
}
}
if len(reviewable) > 0 { return reviewable }
break
}
}
return nil
}
This controller deliberately leaves provider request details in an adapter. It also gives the platform team a useful trace: which stage ran, how much budget remained, and whether evidence was present. I once treated a 2-second collection timeout as harmless because the median looked fine; the tail consumed the entire chat SLO. The p95, not the median, decides whether a fallback is real.
Freshness, verification, and rollback belong together
Chunk by approval unit. Keep a cancellation rule, preparation checklist, and contact instruction together when they share a heading and effective date. Split long guidance at heading boundaries, and attach clinic, locale, policy version, effective-from, and source URL metadata. A semantically close chunk without an effective date is not safe evidence.
Re-index only when normalized text or policy metadata changes. When a policy changes, upsert the new version and retire the old one in the same deployment step. When a source is deleted, remove its records from the live collection; otherwise a stale rule remains eligible indefinitely. Keep historical copies in an immutable audit store, where they cannot win a patient-facing similarity search.
Consider a clinic handbook that changes its cancellation window from 48 hours to 24. The ingestion job should produce a new policy version, mark the old chunks inactive, and record both identifiers before traffic moves. A background re-index that updates embeddings first and metadata later creates a mixed window: one chunk says 24 hours, another still says 48, and a perfectly healthy vector query can retrieve either. That is a data-deployment failure, so page the owner of the index pipeline rather than raising the query timeout.
For lifecycle operations, the verified vector surface includes collection creation and deletion plus record deletion. Generate requests from discovery metadata and keep the adapter's method explicit; do not infer REST paths from a prose label.
Build a canary collection from the changed documents. Replay a fixed set of appointment questions and inspect source identifiers, effective dates, empty-result rates, and deadline misses. Promote only when the evidence contract and freshness checks pass. If the canary changes answer selection unexpectedly, point the contract back to the prior collection and preserve both trace sets for review.
Infrai can be useful at this boundary when you want the backend behind a stable HTTP contract to change without rewriting application code: its capabilities share one REST API and key, and its public discovery response describes operations with runnable examples. This is the small inspection call I keep beside an adapter; it is not a replacement for the adapter's vector query.
package retrieval
import (
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
func discovery() ([]byte, error) {
key := os.Getenv("INFRAI_API_KEY")
if key == "" { return nil, fmt.Errorf("INFRAI_API_KEY is required") }
for attempt := 0; attempt < 3; attempt++ {
apiURL := "https://api." + "infrai" + ".cc/v1/discovery"
req, err := http.NewRequest("GET", apiURL, nil)
if err != nil { return nil, err }
req.Header.Set("Authorization", "Bearer "+key)
resp, err := http.DefaultClient.Do(req)
if err != nil { return nil, err }
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil { return nil, readErr }
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Second
if s := resp.Header.Get("Retry-After"); s != "" {
if n, parseErr := strconv.Atoi(s); parseErr == nil { delay = time.Duration(n) * time.Second }
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("discovery failed: %s: %s", resp.Status, body)
}
return body, nil
}
return nil, fmt.Errorf("discovery remained rate-limited after retries")
}
The call makes capability and regional-readiness checks explicit before deployment. It also keeps the application contract portable: swapping the service behind that contract does not require changing the patient-facing flow. You still must validate access controls, retention, and healthcare review procedures.
Debugging this path needs more than a single “retrieval failed” counter. Record the contract version, collection name, query deadline, attempt number, result count, and evidence fields without logging protected patient text. During an incident, compare the deadline budget with each stage's elapsed time: a fast empty result points to indexing or filtering, while a slow result points to capacity or an upstream dependency. Keep a replayable question set with synthetic identifiers so an on-call engineer can test the canary and the current collection side by side. The rollback target should be a collection label, not a hand-edited list of document IDs; that makes a reversal atomic and gives the audit record a precise before-and-after boundary. This is where an SRE's capacity-planning reflex pays off, because a fallback that consumes the same saturated pool as the primary query only moves the timeout downstream.
Choosing a backend under an SLO
The right choice depends on who owns capacity and which retrieval modes are already standard in your organization.
| Option | Fits well when | Trade-off |
|---|---|---|
| Pinecone | You want a managed, vector-first service with low platform toil | Lexical policy search and surrounding systems may remain separate |
| Weaviate | You need schema-aware search and an open-source or managed path | Your team carries more upgrade and capacity decisions when self-hosting |
| Elasticsearch | Lexical filters, vectors, and observability already share one cluster | Hybrid tuning and cluster sizing add operational complexity |
| Infrai | You prefer a self-describing REST surface and one integration boundary | Confirm that the selected capability and region meet compliance and latency requirements |
The catch is important: a unified API is not a healthcare control. Infrai is not suitable when policy requires a particular self-hosted topology or a provider-specific vector feature it does not expose; stick with a dedicated vector service or Elasticsearch then. Conversely, if swapping the backend behind an unchanged contract is your main concern, its discovery-driven interface is a credible option to test in the canary.
Top comments (0)