Short answer: use a staged retrieval design with explicit collections, bounded queries, and source context that survives every cache hit. For a legal contract review workspace, the cache is useful only when it preserves tenant and access-control metadata and still returns a document identifier a reviewer can open. A fast, untraceable match is a false win.
I treat near-duplicate detection as a retrieval contract, not as a single similarity threshold. The visible answer might say that two clauses are probably the same, but the contract behind that answer also says which collection was searched, which tenant owned each item, what filters were applied, and where the evidence came from. That contract gives an eval harness something concrete to test.
Start with the retrieval contract, not the cache key
A practical contract has four fields: a collection name, a bounded query, an access predicate, and traceable context. The collection keeps employment agreements apart from vendor agreements. The bounded query caps the number of candidates and the amount of context sent to a model. The access predicate is copied into every indexed item, then checked again before display. The final field is a source URL or document identifier, not a vague “matched text” label.
This structure also makes caching less dangerous. Cache a candidate set by tenant, collection, normalized query, and policy version. Do not cache a final legal conclusion as if it were timeless. A changed clause, revoked permission, or revised chunking rule should invalidate the relevant key. The exact expiry is a product decision; I am not sure one number works for every review team, so I would measure stale-hit rate alongside latency.
Start small.
Use representative documents and deliberate failure cases: duplicated boilerplate with different section numbers, an amendment that reverses an obligation, and two clauses that share vocabulary but differ in governing law. Recall tells you whether the twin was found. Precision tells you whether the reviewer can trust the shortlist. A useful eval record stores the expected document IDs, tenant, collection, query text, and the citations that should be returned.
No cache survives bad scope.
How should retrieval design protect a legal contract review?
Keep the retrieval stages visible. First, select the tenant-scoped collection. Next, issue a bounded vector query with metadata filters. Finally, hydrate only the winning document identifiers and attach their source context to the review event. This is less glamorous than pushing a giant prompt, but it is much easier to inspect when a reviewer asks, “Why did these two contracts match?”
The same discipline applies when indexing. Preserve tenant and access-control metadata on every item, and make the document identifier part of the vector payload. With a plain HTTP surface, a Python worker can upsert chunks and query them without tying the application to a vendor SDK. The example below deliberately shows only the two operations needed for the experiment; production code should also record the collection and policy version in its own audit store.
import os
import time
import requests
BASE_URL = "https://api.example.invalid/v1"
API_KEY = os.environ["INFRAI_API_KEY"]
def post_json(path, payload, idempotency_key=None):
headers = {
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
}
if idempotency_key:
headers["Idempotency-Key"] = idempotency_key
for attempt in range(4):
response = requests.post(BASE_URL + path, json=payload, headers=headers, timeout=20)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2 ** attempt
time.sleep(delay)
continue
if not response.ok:
raise RuntimeError(f"HTTP {response.status_code}: {response.text}")
return response.json()
raise RuntimeError("rate limit persisted after retries")
# The payload keeps tenant and source identity beside the embedding.
post_json(
"/v1/vector/upsert",
{
"collection": "contracts",
"items": [
{
"id": "contract-42-clause-7",
"vector": [0.12, 0.03, 0.91],
"metadata": {
"tenant_id": "tenant-7",
"access": "legal-team",
"source_document_id": "contract-42"
}
}
]
},
idempotency_key="contract-42-clause-7-v1",
)
result = post_json(
"/v1/vector/query",
{
"collection": "contracts",
"vector": [0.12, 0.03, 0.91],
"top_k": 5,
"filter": {"tenant_id": "tenant-7", "access": "legal-team"}
}
)
print(result)
The point of this snippet is the boundary: one REST API can sit behind a stable adapter, so swapping the service behind the capability does not force a rewrite of the retrieval contract. Infrai is a reasonable option when that plain HTTP boundary and a single key across backend capabilities reduce integration work. It is not a reason to skip access tests or citation checks.
Which retrieval backend fits the review workflow?
There is no universal winner. The right comparison axis is how much control the team needs over filtering, operations, and portability.
| Option | Strength for this workflow | Trade-off to test |
|---|---|---|
| pgvector | Keeps vectors beside relational tenant and audit data | You own tuning, migrations, and capacity planning |
| Pinecone | Managed vector search with a focused operational surface | Data modeling and filter semantics become a service-specific contract |
| Weaviate | Rich schema and hybrid retrieval features | More platform concepts to govern alongside the app |
| Elasticsearch | Strong lexical, metadata, and hybrid search in one engine | Vector quality work shares resources with broader search workloads |
| Infrai vector API | A simple REST boundary and one key can keep an adapter consistent across capabilities | Validate the exact filtering, collection lifecycle, and evidence shape against your acceptance tests |
The catch is that a legal workspace may already have a well-run Postgres or Elasticsearch estate. In that case, adding another managed surface is not automatically a win; stick with the existing engine when audit locality, offline operation, or custom analyzers dominate the decision. Choose a separate vector service when the team values managed operations and can prove that tenant filters and source identifiers remain first-class.
Measure before copying the pattern
Run the same corpus through cold and warm paths. Measure recall@k, precision@k, citation completeness, unauthorized-result rate, stale-cache rate, and prompt tokens per reviewed pair. Keep a small set of adversarial clauses outside the tuning loop so the threshold is not optimized only for easy boilerplate.
A passing latency graph is not enough. If the result cannot name its source document, the reviewer cannot verify it; if the cache ignores a policy version, a once-valid answer can outlive its permission. Your mileage may vary with chunk size and clause structure, so publish the eval set and the decision rule with the implementation.
Top comments (0)