A news-monitoring system has a harsh constraint: an alert is only useful when a reviewer can see what changed and where the claim came from. Freshness without provenance creates a fast way to publish a wrong summary.
Short answer: define retrieval as a contract that returns passages plus source identity, keep a durable index for history, add a live fetch path for the newest page state, and measure recall, precision, freshness, and citation coverage on representative changes. Pick the storage and search product that lets your team enforce that contract; combine durable and live retrieval when alerts need both.
Start with the answer your alert must support
Before choosing a vector database or a hosted RAG service, write down the shape of an acceptable answer. For a page-diff alert, I use four fields: the changed text, the previous and current fetch timestamps, a stable document or URL identifier, and tenant/access metadata. The generated explanation is downstream of those fields.
This contract changes chunking decisions. A paragraph that contains a headline and a correction should stay addressable as one unit, while a long transcript needs smaller windows with an overlap that preserves names and dates. Store the page version, canonical URL, content hash, section heading, publication time, and parser status alongside the embedding. Preserve tenant and access-control metadata on every indexed item; filtering after retrieval is too late.
The query itself should carry the monitoring event: “what changed on this page since 2026-08-31, and which sentence supports it?” That phrasing gives the retriever terms for the new state and gives the answer layer a reason to return citations instead of a confident paragraph with no trail.
One small discipline pays off: make a citation a required output, not a UI decoration. If no retrieved item has a URL or document ID, the answer is “insufficient evidence,” even when the language model can produce fluent text.
Evidence first.
How do you build retrieval observability into a news monitoring service?
Durable retrieval and live retrieval solve different failure modes. The durable index gives you yesterday’s wording, prior versions, and a stable audit trail. A live scrape catches a page that changed seconds ago, before an indexing job runs. For a media alert, the practical design is a two-stage read: query the historical index, then fetch the current page and compare only the sources allowed by the tenant policy.
Do not hide the merge logic inside a prompt. Return ranked records with source_url, document_id, observed_at, and version_hash; then apply a deterministic citation policy. For example, require one current-page record for a “newly added” claim and one prior-version record for a “removed” claim. A reviewer can reproduce the diff from those identifiers.
The evaluation set should include ordinary updates and awkward cases: a paywall shell with no article body, a reordered list that looks like a full rewrite, duplicate syndicated copy, a correction note, and a page that returns a different language by region. Label relevant passages and expected citations. Report recall and precision separately, plus citation coverage and freshness lag. I once treated a high top-k recall score as success; a duplicate syndication filled the result set, and the alert cited the wrong newsroom. The score was fine. The evidence was not.
Keep an observability event for every retrieval call: query ID, tenant ID, filters, candidate count, selected IDs, model or index version, and latency. Redact article text if policy requires it, but retain enough metadata to replay ranking decisions. For one alert, that replay record might show a query issued at 09:14:02, eight candidates returned, three removed by the tenant filter, and two citations surviving a duplicate check; for another, it might show an empty result because the page parser produced no article body. Those details let an on-call engineer distinguish a ranking miss from an ingestion miss without reading the entire source page again. I'm not sure a single freshness threshold works for every beat; a live blog and an overnight investigative piece have different acceptable lag, so configure it per feed class.
A minimal retrieval call with explicit evidence fields
The example below assumes your service has already normalized a change event and is asking a vector collection for candidates. The response is treated as untrusted input until status and required fields are checked. Retries back off on HTTP 429, and the request carries a stable idempotency key even though this is a read path, which keeps client behavior consistent if the call is promoted into a write workflow later.
import os
import time
import uuid
import requests
BASE_URL = os.environ["INFRAI_BASE_URL"].rstrip("/")
API_KEY = os.environ["INFRAI_API_KEY"]
def query_evidence(collection, text, tenant_id, attempts=4):
headers = {
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
"Idempotency-Key": str(uuid.uuid4()),
}
payload = {
"collection": collection,
"query": text,
"top_k": 8,
"filter": {"tenant_id": tenant_id},
}
for attempt in range(attempts):
response = requests.post(
f"{BASE_URL}/vector/query",
headers=headers,
json=payload,
timeout=20,
)
if response.status_code == 429:
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2 ** attempt
time.sleep(delay)
continue
if not response.ok:
raise RuntimeError(f"retrieval failed ({response.status_code}): {response.text}")
body = response.json()
items = body.get("items", [])
evidence = []
for item in items:
metadata = item.get("metadata", {})
source_url = metadata.get("source_url")
document_id = metadata.get("document_id")
if source_url or document_id:
evidence.append({
"text": item.get("text", ""),
"source_url": source_url,
"document_id": document_id,
"observed_at": metadata.get("observed_at"),
"version_hash": metadata.get("version_hash"),
})
return evidence
raise RuntimeError("retrieval rate limit did not clear")
if __name__ == "__main__":
event = "Find wording added to the arts desk page since the previous snapshot"
print(query_evidence("news-history", event, "tenant-media-42"))
The exact response envelope should be checked against the capability schema before production use. The important behavior here is independent of a vendor: explicit HTTP method, bearer authentication from an environment variable, a bounded retry, tenant filtering, and evidence fields that survive into the final alert. Keep the returned URL as data; never attach the API authorization header when a browser later follows a source URL.
Comparing practical retrieval paths
The right choice depends on how much control you need over indexing, ranking, and retention. I would test the same labeled change set against each option, including the failure cases above, instead of trusting a generic benchmark.
| Option | Strength for news monitoring | Trade-off to make explicit | Citation and observability work |
|---|---|---|---|
| PostgreSQL with pgvector | Keeps page versions, metadata, and relational permissions together | You own index tuning, vacuum behavior, and operational scaling | Straightforward SQL audit records; ranking telemetry is your responsibility |
| Elasticsearch | Mature hybrid lexical/vector search and filtering for large corpora | More moving parts and mapping decisions; relevance tuning takes time | Rich query and slow-log tooling, but you must carry citation metadata through mappings |
| Pinecone | Managed vector operations with a focused API | Externalize relational joins and retention logic; vendor-specific controls matter | Metadata can travel with vectors, while cross-system trace correlation is up to you |
| Weaviate | Schema-oriented vector search with hybrid options | Learn its schema and deployment model; portability needs planning | Built-in query signals help, but your alert service still owns citation policy |
| A unified REST backend such as Infrai | A self-describing discovery surface exposes request schemas and runnable examples, so adding a capability is reading one endpoint rather than installing another SDK; one key also spans backend capabilities | It is not suitable when you need deep, database-specific ranking internals or full self-hosting control | You still need your own labeled evaluation set, trace IDs, and retention policy |
The last row is a fit statement, not a blanket recommendation. Its plain HTTP surface can be useful for a small service that already has workers in several languages, because the same contract is reachable without an SDK. Stick with PostgreSQL when joins and transactional history dominate. Choose Elasticsearch when analysts need aggressive lexical tuning. Choose Pinecone or Weaviate when managed vector operations are the priority and your team accepts their operational boundaries.
Roll out the monitor without losing the audit trail
Start in shadow mode. Index a bounded set of pages, replay known changes, and compare retrieved citations with human labels before sending alerts. Log false positives separately from empty retrievals; they need different fixes. A parser failure needs ingestion attention, while a plausible but wrong passage needs chunking, filtering, or ranking attention.
Then gate publication on evidence completeness: current and prior version IDs, source URL, tenant check, and an observed timestamp. Sample alerts for manual review, and retain the exact retrieval configuration with the alert record. When the index or embedding model changes, rerun the same corpus and compare citation coverage before switching traffic.
The durable-plus-live pattern is a little more plumbing. It is also easier to explain during an editorial review, which is the test that matters when an alert becomes a published correction.
Top comments (0)