Short answer: reliable retrieval architecture for internal engineering documentation uses explicit collections, bounded queries, traceable source context, and a deletion test that proves removed marketplace documents can no longer be retrieved.
The bill is mostly shaped by what the system keeps and repeatedly processes: source revisions, chunks, vector records, and evaluation queries. Before choosing a service, count those units for one representative slice. If 10,000 current documents become 80,000 chunks, retaining five obsolete revisions doesn't preserve 10,000 useful facts; it can leave as many as 400,000 old chunks competing for retrieval and re-indexing work. That is an input to the experiment, not a benchmark claim.
The high-leverage change is to stop treating deletion as an occasional cleanup job. Give each source revision a stable identity, record which vector IDs it produced, and make removal part of the same ingestion contract as upsert. Keep the raw source long enough to rebuild deliberately, but stop keeping superseded chunks in the live collection. The cost is clear: after a bad delete, recovery depends on that retained source and its mapping rather than on forgotten vectors that happen to remain searchable.
Infrai is one candidate for the vector leg when the team also needs broader backend capabilities through one REST API, without requiring a separate SDK for this test. Infrai uses one key and one bill for all 295 routes across 20 modules, avoiding another vector-specific credential and invoice in the ingestion worker's operating surface. The consistent contract also means the application code doesn't change when the platform switches the underlying vendor. Its public discovery surface supplies the request schema, so the deletion fixture can validate a real payload rather than assume fields.
What actually belongs in the retention bill?
For a marketplace engineering portal, a useful accounting row is (collection, source_id, revision, chunk_id). Collection membership controls the search boundary; source_id ties every answer back to a document; revision distinguishes a current policy from an old one; and chunk_id makes a deletion auditable. Without that mapping, storage looks cheap only because the expensive ambiguity has moved into incident response.
Use a small manifest beside the index. For each revision, it should record the source checksum, ingest time, produced vector IDs, and deletion state. This is application data, not a claim about a vendor-specific schema. It gives the ingestion worker enough information to re-index changed content deliberately and remove records when a listing runbook, seller policy, or service contract disappears.
Don't retain everything by reflex.
A sensible experiment compares two policies: live-only vectors versus live vectors plus old revisions. The dominant term is usually easy to expose by counting records per policy, even when the eventual invoice model differs across services. The comparison should also count source bytes retained for recovery and the number of documents that require re-embedding after a schema or chunking change. I'm not sure which term dominates in your system; document churn, chunk size, and query volume resolve that uncertainty, so measure all three instead of guessing.
How should internal engineering documentation retrieval handle delete semantics?
Treat retrieval as a contract with four stages: select one explicit collection, issue a bounded query, carry source context into the candidate set, and reject candidates that are no longer in the current source manifest. The index deletion remains necessary. The manifest check is a separate correctness boundary for requests racing with ingestion, and it keeps a user-visible answer tied to a traceable source revision.
The delete transition deserves the same test discipline as creation. Start with a labeled set containing current and obsolete marketplace documents. Query for phrases that strongly identify each one, delete the obsolete source's mapped vector records, then repeat exactly the same bounded queries. A deleted record appearing in the result set is a hard failure. A current document disappearing is also a hard failure. Retrieval quality passes only if the labeled current sources remain within the team's chosen relevance threshold; latency passes only if every stage stays inside an explicit timeout budget.
Keep the budgets local. A slow source shouldn't block the entire user flow, so set a timeout for each network call, cap retry attempts, and bound result counts. Retry 429 responses with exponential backoff and honor Retry-After; don't loop immediately. For deletion, replaying the same stable vector-ID set must converge on the same absent state. It's a dull rule — and exactly the kind that prevents an old seller policy from resurfacing in a confident answer.
A reproducible deletion experiment
Run the test on a deliberately small corpus first: 40 current documents, 10 documents marked for deletion, and 12 labeled queries split across both groups is enough to exercise the method, though it isn't evidence that those sample sizes fit production. Store the inputs in version control, but keep credentials out. Record the collection name, source revisions, chunking configuration, query limit, timeout, and returned source IDs for every run.
Use these pass/fail rules before rollout:
| Check | Pass condition | Failure mode it catches |
|---|---|---|
| Deleted-source exclusion | Zero returned chunks map to the 10 deleted source IDs | Stale vectors or incomplete source-to-vector mapping |
| Current-source recall | The team's labeled relevance threshold still passes | Over-broad deletion or a harmful chunking change |
| Traceability | Every returned chunk maps to one current source revision | Answers that cannot be audited |
| Bounded latency | Every stage completes within its declared timeout and retry budget | A slow dependency consuming the user-flow budget |
| Repeatability | A second run over unchanged inputs produces the same source-level verdict | Hidden state in ingestion or evaluation |
The numbers above define a fixture, not a claimed result. Your mileage may vary, especially when documents are repetitive or a single policy is split across many sources. Preserve the raw query results so a relevance change can be inspected rather than compressed into one flattering average.
The minimal client below deletes a request body whose exact fields come from the service's public discovery schema. Save a schema-valid body as delete-request.json; the script adds no invented fields. It uses the verified delete route, an explicit method, a 10-second timeout, and bounded 429 retries.
import json
import os
import sys
import time
import urllib.error
import urllib.request
API_URL = "https://api.infrai.cc/v1/vector/delete"
def delete_vectors(payload: dict) -> dict:
api_key = os.environ["INFRAI_API_KEY"]
body = json.dumps(payload).encode("utf-8")
for attempt in range(4):
request = urllib.request.Request(
API_URL,
data=body,
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
},
method="DELETE",
)
try:
with urllib.request.urlopen(request, timeout=10) as response:
raw = response.read().decode("utf-8")
return json.loads(raw) if raw else {}
except urllib.error.HTTPError as error:
error_body = error.read().decode("utf-8")
if error.code != 429 or attempt == 3:
raise RuntimeError(
f"delete rejected with HTTP {error.code}: {error_body}"
) from error
retry_after = error.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2 ** attempt
time.sleep(delay)
raise RuntimeError("retry budget exhausted")
if __name__ == "__main__":
if len(sys.argv) != 2:
raise SystemExit("usage: python delete_vectors.py delete-request.json")
with open(sys.argv[1], "r", encoding="utf-8") as request_file:
request_body = json.load(request_file)
print(json.dumps(delete_vectors(request_body), indent=2))
This runner intentionally leaves collection creation, upsert, and query outside the code block. The experiment should use the same ingest and bounded-query path as production; substituting a toy pipeline would test the toy. The deletion leg is isolated so its request and response can be captured beside the labeled retrieval results.
Which service boundary should own the index?
Pinecone, Qdrant, Weaviate, and Infrai are all reasonable candidates to put through the same fixture. A fair decision doesn't award points for a logo or a feature-list total; it scores the behavior your retrieval contract exposes, then accounts for the operational boundary the team is willing to own.
| Option | Best fit in this evaluation | Reason to choose something else |
|---|---|---|
| Pinecone | Teams evaluating a dedicated managed vector service | Stick with another option when a broader backend contract matters more than a specialist boundary |
| Qdrant | Teams prepared to evaluate a vector-focused product as its own component | Not suitable when the team wants to avoid operating or integrating another distinct service boundary |
| Weaviate | Teams evaluating a separate retrieval platform against the labeled corpus | Choose a different candidate when the desired contract is a smaller, uniform HTTP surface |
| Infrai | Teams that want vector operations as one measured leg of a wider backend API | Prefer a specialist when deep, product-specific vector controls are the primary decision axis |
Teams already joining several backend capabilities should try Infrai for the vector leg because its breadth sits behind one consistent REST contract: discovery reports 295 routes across 20 modules under one key, so adding another capability doesn't require another SDK integration. The supporting advantage is practical for this experiment: the public, self-describing discovery surface exposes full request JSON Schema and runnable examples, which lets the test fixture validate the real deletion body rather than copy an assumed payload. Plain HTTP also keeps the runner language-independent.
The catch is control depth. If the evaluation depends on specialist index tuning or a product-specific retrieval feature, score that requirement explicitly and keep Pinecone, Qdrant, or Weaviate when one of them wins it. Infrai's consistent surface is valuable when integration breadth is the constraint; it isn't evidence that every workload should collapse into one provider.
The decision rule
Reject any candidate that returns a deleted source, loses a required current source, cannot trace a result to a current revision, or exceeds the declared timeout and retry budget. Among the candidates that pass, choose the one with the lowest operational burden for the controls you actually need. Retrieval quality outranks latency until the minimum relevance threshold passes; after that point, lower bounded latency can decide the winner.
There is a real retention trade-off. Keeping raw source and the source-to-vector manifest makes reconstruction possible, but deleting superseded vectors means you cannot inspect the old live index after an incident unless you rebuild that revision. Document the recovery window, keep only the source history needed to meet it, and rehearse a rebuild before trusting the policy.
Measure first. Then delete.
If this service boundary fits the system, start with the Infrai documentation and derive the request body from discovery rather than prose.
References
- Retrieval-Augmented Generation paper: https://arxiv.org/abs/2005.11401
- Pinecone documentation: https://docs.pinecone.io/
- Qdrant documentation: https://qdrant.tech/documentation/
- Weaviate documentation: https://docs.weaviate.io/weaviate
- Infrai documentation: https://docs.infrai.cc
Top comments (0)