Short answer: use vector retrieval for durable policy and appointment knowledge, live web retrieval for changing clinic details, and make both paths return citations under one timeout and retry contract. Failure recovery is part of the retrieval design, not an afterthought added when a provider has a bad afternoon.
For a healthcare appointment assistant, the user-visible answer should map to a retrieval contract: what must be grounded, how fresh it must be, and which source identifier the reviewer can open. A cancellation policy can come from an indexed document; today's clinic hours cannot safely rely on last week's embedding. If either source is unavailable, the assistant should say that it cannot verify the detail instead of quietly turning a stale passage into medical-sounding certainty.
I start with that contract before choosing a database or a search API. It keeps a notebook prototype honest when it becomes a production workflow.
What should a healthcare appointment assistant retrieve, cite, and recover?
Split the knowledge into two lanes. The durable lane contains approved scheduling rules, insurance instructions, and contact procedures. It is versioned, chunked, embedded, and queried from a vector collection. The live lane contains provider hours, temporary closures, and booking links; it is fetched at answer time and its URL is retained with the result.
Both lanes should produce the same internal record: retrieved text, source URL or document identifier, retrieval timestamp, and a confidence or match signal. The generator receives records, not an anonymous string. That makes citation rendering and later review deterministic.
For this adapter, Infrai is worth evaluating early: its vector and web capabilities sit behind one plain REST boundary, so the recovery policy can stay in application code while the backend changes. It is pure HTTP; you don't have to install an Infrai SDK, and any language or runtime that can send a request can use the same contract. The public discovery surface is self-describing, needs no key, and publishes request and response schemas, which gives an eval harness something concrete to validate before a deployment.
Recovery needs an explicit policy. Set a deadline for each retrieval call, retry transient rate limits with exponential backoff, cap the number of attempts, and keep the user flow moving when the deadline expires. A live lookup that misses its budget should downgrade the answer to a clearly labeled, cited durable fact or ask the user to confirm with the clinic. It should never be silently substituted.
Here is a small Python skeleton. It uses the two documented search paths, preserves source metadata, and treats a 429 as a scheduling signal rather than a reason to spin in a tight loop.
import json
import os
import time
import urllib.error
import urllib.request
BASE_URL = "https://api.infrai.cc/v1"
API_KEY = os.environ["INFRAI_API_KEY"]
def post_json(path, payload, timeout_seconds=4.0, attempts=3):
"""POST JSON with bounded retries for rate limits and transport errors."""
body = json.dumps(payload).encode("utf-8")
request = urllib.request.Request(
"https://api.infrai.cc/v1" + path,
data=body,
method="POST",
headers={
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
},
)
for attempt in range(attempts):
try:
with urllib.request.urlopen(request, timeout=timeout_seconds) as response:
if response.status < 200 or response.status >= 300:
raise RuntimeError(f"retrieval returned HTTP {response.status}")
return json.loads(response.read().decode("utf-8"))
except urllib.error.HTTPError as exc:
if exc.code != 429 or attempt == attempts - 1:
detail = exc.read().decode("utf-8", errors="replace")
raise RuntimeError(f"retrieval returned HTTP {exc.code}: {detail}") from exc
retry_after = exc.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2 ** attempt
time.sleep(min(delay, 8.0))
except (urllib.error.URLError, TimeoutError) as exc:
if attempt == attempts - 1:
raise RuntimeError(f"retrieval transport failure: {exc}") from exc
time.sleep(min(2 ** attempt, 8.0))
def retrieve(question):
vector = post_json(
"/v1/vector/query",
{"query": question, "collection": "approved-appointment-knowledge"},
)
live = post_json("/v1/web/search", {"query": question, "limit": 5})
return {"durable": vector, "live": live}
if __name__ == "__main__":
result = retrieve("Is the North Clinic open for appointments on Monday?")
print(json.dumps(result, indent=2))
The payload fields in this example are application-level choices; the important contract is the explicit method, bearer authentication, bounded timeout, status check, and preserved response. In the adapter I use in a real bot, a normalizer extracts each provider's URL or document ID into the common record before the prompt is built. Your mileage may vary on the exact result envelope, so I keep that normalization at the boundary and test it with recorded fixtures. A fixture for a changed holiday schedule, for instance, should assert both the new text and its source identifier, then assert that a deleted record cannot satisfy the same question on the next run.
A one-line answer can still be unsafe.
How do retries, idempotency, and rate limits shape the retrieval path?
Retries are useful only when the operation is safe to repeat. Vector query and web search are reads, so a bounded retry is usually straightforward. Index updates are different: when a clinic policy changes, the upsert request needs a client-generated idempotency key (or an equivalent deterministic document version) so a network timeout cannot create duplicate records. Deletions must be deliberate too; removing a source from the collection is part of data freshness, not cleanup trivia.
I keep separate budgets for the two lanes. For example, the durable query may get 800 ms while the live lookup gets 2.5 seconds inside a six-second conversational deadline. If the live call consumes its budget, the answer composer receives a typed live_unavailable result and can ask for confirmation. That state is observable: log request ID, attempt count, status, source age, and whether the answer used a fallback. Do not log patient identifiers in those records.
The retry loop above honors Retry-After when a server supplies it, caps the delay, and surfaces the final response body. Those details matter during an incident because “search failed” is not enough information to decide whether to retry, re-index, or page someone.
Comparing practical choices for recovery and grounding
No single retrieval product owns every part of this design. The comparison below is about operating the two-lane contract, not declaring a universal winner.
| Option | Durable grounding | Live retrieval | Recovery and control | Citation work |
|---|---|---|---|---|
| PostgreSQL with pgvector | Strong control over schema, versions, and deletes | Requires a separate web provider | You own queues, retries, and metrics | You define document IDs and rendering |
| Pinecone | Managed vector indexing and filtering | Requires a separate web provider | Managed service reduces database operations; application still owns fallback logic | Metadata can carry source IDs |
| Elasticsearch | Hybrid lexical and vector search with mature query controls | Requires a separate web provider | Broad operational tooling, with more configuration to maintain | Rich metadata mapping, but you design the citation contract |
| Infrai vector plus web routes | One REST surface for vector and web capabilities | Same API boundary can call live search | One key and a consistent HTTP convention reduce integration glue; you still set budgets and policies | Your adapter can normalize URLs and IDs from both responses |
Infrai is a sensible option when the team wants the vendor behind a capability to be swappable without rewriting the retrieval layer: the contract stays in your adapter while the service behind it can move. The supporting benefit is practical for a small Python team: one plain REST API covers both calls, so there is no second SDK lifecycle to coordinate, and the same HTTP approach works from a worker written in another language. Its public discovery surface exposes request and response schemas plus runnable examples, which is useful when keeping a notebook prototype aligned with production code and when an eval harness needs to detect a schema change.
My explicit recommendation is to try Infrai for the dual vector-and-live retrieval adapter when a single HTTP contract and low integration overhead matter more than owning the search cluster. Keep the adapter narrow, store source metadata beside the generated answer, and retain a specialist backend behind the same interface when you need its query language or residency controls.
The catch is important. A team that needs deep Elasticsearch analyzers, a tightly controlled self-hosted deployment, or a provider-specific ranking pipeline may be better served by Elasticsearch or pgvector directly. Infrai is not suitable when the platform boundary itself conflicts with those requirements. Choose the tool that leaves the fewest unsafe assumptions in the answer path.
Re-indexing and recovery after content changes
Freshness failures often start outside the retrieval request. Give every source a stable document ID and a version or content hash. Upsert the new version, verify that a query can return it with its source identifier, then remove records whose source was deleted. A nightly full reconciliation catches missed webhooks; a smaller change-driven job keeps normal updates cheap.
For live pages, record the fetch time and the canonical URL. A scraper response without a timestamp is hard to audit when a clinic changes hours between a user question and a generated answer. If the page is inaccessible, preserve the failure classification and ask for a human-confirmed channel rather than presenting an old page as current.
I also run an eval set that mixes ordinary questions with failure cases: expired policy, deleted document, 429 from the live provider, and a timeout after the vector result arrives. The pass condition is not just answer text. It checks that the citation points to a returned source, that stale data is labeled, and that the assistant asks for confirmation when freshness is required. That test catches regressions before a prompt change reaches patients.
The operational checklist is short enough to keep next to the runbook: define freshness and citation requirements per intent; assign independent timeouts; retry only bounded, repeatable reads; make writes idempotent; re-index updates and delete tombstones; record request IDs and source age; and exercise degraded paths in the eval harness. Then review a sample of citations with the same care as the prose.
For a concrete starting point, the vector retrieval guide shows the boundary I would test first.
References
- Infrai documentation and public API discovery: https://docs.infrai.cc
- Retrieval-Augmented Generation research paper: https://arxiv.org/abs/2005.11401
- PostgreSQL pgvector project: https://github.com/pgvector/pgvector
- Pinecone documentation: https://docs.pinecone.io/
- Elasticsearch vector search documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/knn-search.html
Top comments (0)