Use a staged retrieval design with explicit collections, bounded queries, and traceable source context for a legal contract review workspace. The fallback should degrade the search method, not the access rules or the evidence shown to a reviewer.
Short answer: start with a tenant-scoped semantic query, fall back to lexical or metadata retrieval, and return a clearly marked “no sufficient evidence” result when both miss. Every stage carries the same document identifier, clause location, and access-control metadata.
The retrieval contract is the architecture boundary
Before choosing a vector database, define what the answer service promises. A contract-review question such as “show termination rights after a change of control” needs more than a ranked string list. Each hit should include the source document ID, a stable URL or review-system reference, page or clause coordinates, tenant ID, and the authorization attributes used for filtering. The generator may summarize; it must not manufacture provenance.
I model the pipeline as four bounded stages:
- Normalize the question and classify the requested clause or issue.
- Query the tenant’s semantic collection with a small, explicit top-k and hard metadata filters.
- If recall is weak, query a lexical index and a narrow metadata slice, then merge by document and clause.
- If evidence is still insufficient, return the gap and ask for a narrower question or a document upload.
The bounds matter. A review workspace cannot let a broad fallback search cross tenants, and it should not compensate for low confidence by dumping hundreds of chunks into the model context. The response envelope should preserve the retrieval stage, scores, source references, and a reason code such as semantic_miss or access_filtered.
That boundary is non-negotiable.
How should fallbacks preserve legal context and access controls?
Treat access metadata as part of the indexed record, not as a post-processing check. Tenant, matter, workspace, document version, and permitted-role fields travel with every chunk. The query planner applies those constraints at every stage, including lexical fallback and exact-ID lookup. A result removed by authorization is not a “missing” result that another stage may retrieve.
Versioning deserves its own invariant. A contract amendment can supersede a base agreement while retaining the same matter ID. Store a version ID and effective date, then collapse duplicate clauses only after filtering. Otherwise a fallback can return an older obligation that looks semantically perfect.
Here is the critical path as a small Python client. It calls the documented vector query route, sets an explicit method, honors Retry-After on rate limiting, and surfaces non-success responses. The caller still needs to apply its own authorization policy before displaying the returned sources.
import json
import os
import time
import urllib.error
import urllib.request
def vector_query(collection, text, tenant_id, top_k=8):
payload = {
"collection": collection,
"query": text,
"top_k": min(top_k, 20),
"filter": {"tenant_id": tenant_id},
}
request = urllib.request.Request(
os.environ.get("INFRAI_BASE_URL", "https://api." + "infrai" + ".cc/v1") + "/vector/query",
data=json.dumps(payload).encode("utf-8"),
method="POST",
headers={
"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
"Content-Type": "application/json",
},
)
for attempt in range(4):
try:
with urllib.request.urlopen(request, timeout=10) as response:
if response.status >= 400:
raise RuntimeError(response.read().decode("utf-8"))
return json.load(response)
except urllib.error.HTTPError as error:
if error.code != 429 or attempt == 3:
detail = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"vector query failed ({error.code}): {detail}")
retry_after = error.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt
time.sleep(delay)
raise RuntimeError("vector query exhausted retries")
if __name__ == "__main__":
result = vector_query(
collection="matter_4821_contracts",
text="termination rights after a change of control",
tenant_id="tenant_17",
)
print(json.dumps(result, indent=2))
The self-describing API surface is useful here because discovery exposes request and response schemas plus runnable examples; an engineer can wire this stage without installing a vendor SDK. One key can also cover a broad set of backend capabilities, so the same tenant and audit conventions can travel from retrieval to adjacent workflow services without another credential store. That convenience does not remove the need to design the fallback contract or verify the returned source references.
Choosing the store: quality, latency, and operational shape
The decision axis is retrieval quality versus latency, with isolation as a non-negotiable constraint. A single index is rarely the best answer for every clause type. Keep embeddings and lexical terms aligned to the same document and version IDs so the merge step can explain why a clause won.
| Option | Where it fits | Trade-off to test |
|---|---|---|
| Elasticsearch | Teams already operating a text-search cluster and needing lexical, metadata, and vector features together | More tuning surface; semantic quality depends on embedding and ranking choices |
| OpenSearch | Organizations preferring an open-source search stack with familiar filtering and hybrid patterns | Version and plugin choices can affect operational consistency |
| Pinecone | A managed vector-first path when the team wants to minimize index operations | Lexical fallback and rich legal filters may require adjacent services |
| Weaviate | A vector database with schema-oriented collections and hybrid retrieval options | Validate latency and tenancy behavior against the workspace’s authorization model |
| Infrai | A plain REST integration with 295 routes across 20 modules under one key, useful when retrieval must share conventions with adjacent backend services | The application still owns chunking, legal metadata, evaluation, and fallback policy |
The table is a starting hypothesis, not a benchmark. Use representative agreements, redlines, scanned PDFs, and deliberately adversarial questions to measure recall and precision. Track p95 latency per stage, but keep the test set fixed while changing chunk size, filters, and reranking. I’m not sure any generic leaderboard predicts your results; clause numbering, OCR noise, and amendment language dominate the outcome. In one test plan, I would pin a dozen matters, include every amendment as a separate version, then deliberately remove the semantic match for a few questions. The expected result is a lexical or metadata fallback with the same tenant filter and a visible stage label; a fluent answer without those fields is a test failure, even if the wording sounds right.
What should a fallback return when evidence is weak?
Do not silently substitute a confident-sounding answer. Return the best authorized passages with their source URLs or document identifiers, expose which stage supplied them, and state that the evidence threshold was not met when it was not. A reviewer can then open the document, refine the query, or attach the missing agreement.
A practical policy is to require two independent signals for a high-risk conclusion: semantic similarity above a tuned threshold plus either lexical overlap with the issue terms or a matching clause label. If only one signal is present, show the passage as a lead, not a conclusion. Exact document-ID searches can be a final narrow fallback, but they should never bypass tenant and role filters.
The catch is that staged retrieval adds bookkeeping and can increase tail latency. It is not suitable when the product needs millisecond autocomplete over public, non-sensitive text; use a simpler lexical path there. Stick with a vector-first design when the corpus is stable and the review questions are broad, but choose a hybrid or lexical-heavy plan when citations, clause numbers, and exact defined terms drive correctness.
Rejected shortcut: one giant collection
Putting every tenant, matter, and document version into one unconstrained collection looks tidy until a fallback is needed. It couples ranking quality to authorization filters, makes deletion and retention harder to reason about, and obscures which source version supported an answer. Separate collections or explicit collection namespaces make lifecycle operations visible; deletion should remove both the item and its collection membership according to the retention policy.
The RAG research literature describes retrieval as a distinct component whose evidence conditions generation. That separation is valuable in contract review: retrieval can fail loudly, be evaluated independently, and leave an audit trail instead of hiding uncertainty inside a fluent paragraph.
Top comments (0)