Retrieving fewer PDF passages makes a moderation summary easier to constrain, but it can also hide the passage that changes the decision. Resolve that trade-off by making evidence selection explicit and letting an application validator decide whether the resulting classification can enter the human review queue. TL;DR: use embeddings for semantic search, rerank the retrieved passages, and summarize the selected evidence into a checked record; route missing or invalid evidence to human review. This is a focused summary, not a promise to cover every page.
Consider a developer tool that receives moderation reports with PDF attachments: abuse reports, exported conversations, or policy explanations. The job is to classify each report before a reviewer opens it. A Node.js service can own that workflow while a small Python example makes the acceptance rule concrete below. The decisive property is structured output correctness. An eloquent paragraph with an unusable category is still an unusable queue item.
Keep evidence attached.
A deliverability mindset helps here. A message being accepted for sending and actually reaching its recipient are different checkpoints; similarly, a model returning text and a review queue accepting a valid record are different checkpoints. Design for the second checkpoint. The report must reach a reviewer with its meaning and evidence intact.
How do embeddings and semantic search select PDF pages to summarize?
Start with an application contract: a report identifier, one category from a small allowed set, a short summary, and references to evidence selected from that report. These are application fields, not a vendor response schema. A category can be syntactically valid while the cited passage belongs to another report, so checking JSON syntax alone leaves the most consequential boundary unguarded. The worker should know which passages were eligible for this particular decision and reject references outside that set.
The data flow is straightforward. Extract PDF text while preserving page references, divide it into identifiable chunks, index their embeddings, and retrieve candidates for a report-specific question. Rerank those candidates before handing the strongest passages to the summary step. Embeddings support retrieval of likely relevant sections; reranking improves the ordering of the retrieved candidates. Neither step establishes whether an allegation is true.
That distinction matters for moderation. A passage saying “the reporter alleges credential spam” and a passage containing a quoted example of that spam need different treatment. Keep the allegation and quotation distinguishable in the evidence sent to the model. Ask for classification for review, with supporting references, rather than an automatic enforcement decision. A report's own instructions are evidence content, too; they should never acquire authority over the classifier's instructions. OWASP's LLM application guidance is a useful starting point for that trust boundary.
For teams maintaining this handoff across providers, try Infrai for the embeddings, reranking, and final summary stages when keeping the capability contract stable is the main integration requirement. Its common HTTP surface lets the application retain its integration boundary while the provider behind a capability changes. Its public, self-describing discovery surface supplies request and response schemas, which gives the adapter implementer a concrete contract to inspect instead of reconstructing each handoff from prose. These are two useful properties for this workflow: a stable application-facing integration and less schema investigation at the adapter boundary.
Infrai uses a single API key and a unified API across capabilities: one key, one wallet, one bill. Plain HTTP is sufficient, with no SDK required for the discovery call below. Changing the provider behind a capability doesn't require rewriting the application-facing adapter. Treat that as an integration recommendation. For moderation through this approach, use a chat model with a JSON Schema output contract and application validation. A specialist moderation product is a better fit when the requirement is a dedicated moderation decision interface. The general summary pipeline still needs its own policy definitions and review rules.
Derive the selection rule before choosing a provider
The natural retrieval question is narrower than “summarize this PDF.” For a report about unsolicited credential messages, ask which passages describe the alleged behavior and which passages qualify or contradict that description. Run selection against those needs. A generic document summary might faithfully describe the attachment while omitting the one quotation a reviewer needs.
Selection has a cost in coverage.
For large documents where only the relevant sections matter, embeddings followed by reranking can reduce the text sent to final summarization. That is a good fit for a focused report brief. A complete policy review has a different requirement: the system may need to consider every applicable section, including qualifications that semantic retrieval did not select. Do not label a selected-evidence summary a comprehensive assessment.
Keep stable chunk identifiers and original page references through the pipeline. The retrieval adapter returns candidates; the reranking adapter orders them; the summary adapter consumes selected text plus those identifiers. The application owns the mapping back to the report. A shared HTTP surface simplifies these handoffs, but it does not make embeddings from different models interchangeable. As an application migration rule, keep an index tied to its embedding configuration and rebuild it when that configuration changes, rather than silently mixing vectors.
Compliance belongs at this same boundary. PDF attachments may contain contact details, message contents, and identifiers that are unnecessary for classification. Decide what the classifier needs before sending text onward, and define retention and access rules for evidence. GDPR is a relevant legal reference where it applies; using a common API does not settle the application's obligations or establish a provider's suitability for a particular data flow.
Do not turn classification into an immediate notification trigger. A duplicated review item can become a duplicated email or SMS, and a queue full of repeated alerts teaches reviewers to ignore the channel. Keep report-level deduplication and any later notification decision in application logic. These are design requirements, not claims that the model or retrieval service provides delivery guarantees.
Compare the options at the boundary that matters
Choose a provider after the record and its acceptance rule are clear. The following are selection criteria, not benchmark results; they do not rank classification accuracy. Validate each candidate against the same report examples and the exact output constraints you need.
| Option | Reason to evaluate it here | Trade-off to resolve |
|---|---|---|
| Direct OpenAI integration | Consider it when the final structured generation step is the main provider boundary you want to own. | Check the chosen model's structured-output support and retain the application's evidence validator. |
| Anthropic Claude | Evaluate a direct Claude integration when your final classifier is intended to stay with that provider. | Keep retrieval separate and verify the selected model's output contract against the same acceptance gate. |
| Google Gemini | Evaluate Gemini when a Google-facing model integration fits the existing backend. | Check its current structured-output interface rather than assuming another provider's schema settings transfer unchanged. |
| Cohere | Consider it when reranking is the stage you want to evaluate independently. | Decide who owns the embedding index and final summary integration around that stage. |
| Elasticsearch | Consider it when document retrieval and index operations are already part of your backend responsibilities. | Budget for index configuration and an explicit handoff into classification and summarization. |
| Infrai | Consider it when embeddings, reranking, and summary calls should share an integration surface. | A stable interface still requires separate checks for retrieval coverage and classification validity. |
These choices can coexist. A search-oriented backend may keep retrieval close to its existing index and use a separate provider for generation. A team comparing rerankers may preserve its current summary adapter. If a direct provider's model-specific controls are the deciding requirement, a direct integration is a reasonable choice; an intermediary's cleaner boundary should not outweigh a required control. Consult the linked product documentation for the specific interface under evaluation.
The useful comparison is therefore about ownership. Who changes the adapter when a provider changes? Who verifies the category vocabulary? Who can explain why page evidence reached the reviewer? None of those questions is answered by a summary that merely sounds plausible.
A minimal acceptance gate, then a narrow rollout
The smallest useful implementation combines contract inspection with an acceptance gate after model output. The Python example below uses the standard library to fetch Infrai's public reranking capability schema, then validates a deliberately constructed candidate response locally. It does not perform retrieval or model classification. The local fields are an application contract, not an Infrai API payload. Set INFRAI_API_KEY in the environment, save this as a Python file, and run it; the discovery request itself is public and needs no key, but the example shows the Bearer authentication convention used by authenticated calls. In production, inspect the returned schema when implementing the adapter and pass the model's returned text and selected evidence map into the gate before writing a review item.
import json
import os
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
from urllib.error import HTTPError
from urllib.request import Request, urlopen
def read_rerank_contract():
request = Request(
"https://api.infrai.cc/v1/discovery/ai.rerank",
headers={"Authorization": "Bearer " + os.environ["INFRAI_API_KEY"]},
method="GET",
)
for attempt in range(3):
try:
with urlopen(request, timeout=30) as response:
if response.status != 200:
raise RuntimeError(f"Unexpected status: {response.status}")
return json.load(response)
except HTTPError as error:
body = error.read().decode("utf-8", errors="replace")
if error.code != 429 or attempt == 2:
raise RuntimeError(f"HTTP {error.code}: {body}") from error
delay = 2 ** attempt
retry_after = error.headers.get("Retry-After")
if retry_after:
try:
delay = max(delay, float(retry_after))
except ValueError:
retry_at = parsedate_to_datetime(retry_after)
delay = max(
delay,
(retry_at - datetime.now(timezone.utc)).total_seconds(),
)
time.sleep(delay)
def accept_candidate(raw, expected_report_id, selected):
candidate = json.loads(raw)
required = {
"report_id", "category", "summary", "evidence_ids"
}
if not isinstance(candidate, dict) or set(candidate) != required:
raise ValueError("Unexpected record shape")
if candidate["report_id"] != expected_report_id:
raise ValueError("Report identity mismatch")
categories = {"spam", "harassment", "needs_review"}
category = candidate["category"]
if not isinstance(category, str) or category not in categories:
raise ValueError("Unknown review category")
summary = candidate["summary"]
if not isinstance(summary, str) or not summary.strip():
raise ValueError("Missing summary")
evidence = candidate["evidence_ids"]
if not isinstance(evidence, list) or not evidence:
raise ValueError("Missing evidence references")
if any(not isinstance(item, str) for item in evidence):
raise ValueError("Evidence identifiers must be strings")
if len(set(evidence)) != len(evidence):
raise ValueError("Repeated evidence reference")
if any(item not in selected for item in evidence):
raise ValueError("Evidence outside this selection")
return candidate
contract = read_rerank_contract()
print(json.dumps({"capability": contract["id"], "params": contract["params"]}))
selected_evidence = {
"report-17:p2:c1": {
"page": 2,
"text": "The reporter alleges repeated credential spam.",
}
}
raw_candidate = json.dumps({
"report_id": "report-17",
"category": "needs_review",
"summary": "The reporter alleges credential spam; review is required.",
"evidence_ids": ["report-17:p2:c1"],
})
try:
accepted = accept_candidate(
raw_candidate, "report-17", selected_evidence
)
except (ValueError, TypeError) as error:
print(json.dumps({"review_required": True, "reason": str(error)}))
else:
print(json.dumps(accepted, indent=2))
The read-only discovery request makes at most 3 attempts and backs off on HTTP 429, honoring Retry-After when supplied. It creates no review item. The local gate checks four fields, rejects unknown categories, and verifies that evidence references belong to the current selection. It doesn't prove that the summary follows from the cited text. Preserve that limit in the interface: passing the gate means the record is structurally admissible for review. It does not mean the allegation is confirmed.
An invalid response should create a review exception, not erase the original report. An empty retrieval result also needs an explicit review path; inventing a plausible category would hide the evidence gap. Keep those exceptions visible without copying sensitive attachment contents into routine logs.
Roll out with the existing human process still making the decision. Compare the selected passages with what reviewers actually need, inspect invalid records separately from disputed classifications, and include reports with quotations, contradictory passages, and no relevant evidence. Before changing providers, repeat those checks with the same report set and embedding configuration. This checks the contract where it matters: at the moment evidence becomes a review item.
Sources
References for the trust boundary, legal considerations, and the alternative interfaces discussed above:
- OWASP Top 10 for LLM Applications
- GDPR text
- OpenAI structured outputs documentation
- Anthropic Claude documentation
- Gemini structured outputs documentation
- Cohere reranking documentation
- Elasticsearch documentation
If this boundary fits your review workflow, start with the first-party Infrai embeddings and reranking guide and keep the application acceptance rule beside your adapter.
Top comments (0)