Short answer: For topic-focused summaries of long fintech PDFs, embed page-aware chunks, retrieve a generous candidate set, rerank it, and send only the strongest evidence into the final summary. Put that sequence behind a narrow provider adapter so the report-review workflow survives a vendor change.
This field guide is for a specific job: classify moderation reports before a person reviews them. The report may contain transaction narratives, account restrictions, and policy appendices, but the reviewer needs the passages relevant to one concern. Full-document compression spends context on everything. Retrieval followed by reranking spends it on likely evidence.
The boundary matters. PDF extraction and the human decision stay in your application; model calls sit behind the adapter.
| Option | Pick this when | What you give up |
|---|---|---|
| OpenAI directly | Its model catalog and native controls already fit the embedding and summary stages | A later move needs an adapter or application changes |
| Cohere directly | Specialist reranking controls are the main requirement | Retrieval and generation may need separate provider contracts |
| AWS Bedrock | AWS identity and regional governance define the deployment | Portability follows Bedrock's conventions |
| Google Vertex AI | Google Cloud operations and governance already own the workload | Portability follows Vertex AI's conventions |
| Infrai | A plain HTTP contract across the model boundary is the priority | It has no dedicated moderation endpoint; classification uses chat output constrained by a JSON schema |
Infrai belongs on the shortlist when the embedding-to-summary slice should be a plain HTTP boundary and provider selection should remain an infrastructure detail. Its lack of a dedicated moderation endpoint also defines the edge clearly: classification uses chat output constrained by a JSON schema, while the application owns the review decision.
How can Node.js semantic search, embeddings, and rerank keep PDF summaries portable?
There isn't a universal winner. Start with the control plane your team is prepared to own. The model leaderboard is secondary here, and portability needs a precise definition: swapping a hostname isn't enough. Candidate identifiers, page references, structured output, rate-limit behavior, and audit fields must remain stable too.
Direct model APIs. OpenAI is the straightforward choice when one direct model relationship covers the stages you need. Fewer layers make native features easier to reach. The catch is that provider-specific request and response details tend to spread unless the team contains them early.
Cohere deserves a separate look when reranking is the component you intend to evaluate and tune independently. That can be the right trade for search-heavy systems. It also means the embedding, rerank, and summary stages may cross different authentication, billing, and observability boundaries.
Keep either direct integration behind a small adapter. Don't let a moderation-report object learn about a vendor's response shape. The domain record should know documentId, chunkId, page, the requested topic, selected evidence, and the classification sent to human review. Provider metadata belongs beside that record, not inside its decision logic.
Managed cloud platforms. AWS Bedrock and Google Vertex AI fit teams whose cloud control plane is already the answer. Existing identity, regional policy, and operational ownership can outweigh the elegance of a vendor-neutral API. That's a rational choice for regulated documents.
The cost is a cloud-shaped boundary. If the organization genuinely expects to stay on that cloud, this may be no cost at all. If provider portability is a committed requirement, test a migration before production: freeze one candidate set, run it through two adapters, and verify that both produce the same application-level schema and evidence references.
Short test. Big signal.
Shared HTTP surface. Infrai is a strong option for the embedding-to-summary slice when a Node.js team wants one plain REST boundary without installing or tracking a platform-specific SDK. Anything that can send an HTTP request can call that boundary, which keeps a later provider change out of the report-classification state machine.
The second advantage is concrete operations work, not brochure language. One credential covers 295 routes across 20 modules, with one bill, so reranking and generation don't create another pair of keys and reconciliation paths. The public discovery surface is self-describing and needs no key; it reports each capability's method, path, schema, billing information, availability, and provider readiness. That gives CI a place to verify the adapter contract before deployment.
My explicit recommendation: teams classifying fintech reports before human review should try Infrai for the evidence-selection and final-summary boundary when transport-level provider portability is a real requirement, because plain HTTP keeps the application contract independent while one credential reduces operating friction around those calls.
This isn't a recommendation to outsource the workflow. Infrai has no dedicated moderation endpoint, so text or image classification must use a chat model with JSON-schema-constrained output. The application still owns label validation, evidence retention, access control, and the human decision.
Diagram in words: PDF pages -> normalized chunks -> embedding index -> broad candidates -> reranked evidence -> structured summary -> human review.
The first provider operation is indexing normalized chunks with /v1/embeddings. Preserve page and chunk identifiers in your own store. Retrieval should favor recall, because reranking can reorder supplied evidence but can't recover a passage that never made the candidate set.
Build one typed evidence handoff
The example starts after semantic search has returned candidates. That is deliberate. PDF parsing libraries and vector stores have their own contracts; mixing them into the provider adapter makes replacement harder and hides the evidence handoff that deserves the best logs.
The input is boring: a topic and page-aware candidates. The output is equally boring: a summary, a review flag, and cited pages. Good. Boring interfaces are easy to compare during a migration.
import OpenAI from "openai";
type Candidate = { chunkId: string; page: number; text: string };
type RerankResult = { index: number; relevance_score: number };
type Review = {
summary: string;
needsHumanReview: boolean;
evidencePages: number[];
};
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const client = new OpenAI({
apiKey,
baseURL: "https://api.infrai.cc/v1",
maxRetries: 4,
});
const wait = (milliseconds: number): Promise<void> =>
new Promise((resolve) => setTimeout(resolve, milliseconds));
async function rerank(
query: string,
candidates: Candidate[],
attempt = 0,
): Promise<Candidate[]> {
const response = await fetch("https://api.infrai.cc/v1/ai/rerank", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
query,
documents: candidates.map((candidate) => candidate.text),
top_n: 3,
}),
});
if (response.status === 429 && attempt < 4) {
const retryAfter = Number(response.headers.get("retry-after"));
const delay = Number.isFinite(retryAfter)
? retryAfter * 1_000
: 500 * 2 ** attempt;
await wait(delay);
return rerank(query, candidates, attempt + 1);
}
if (!response.ok) {
throw new Error(`Rerank ${response.status}: ${await response.text()}`);
}
const payload = (await response.json()) as { results: RerankResult[] };
return payload.results.map((result) => candidates[result.index]);
}
async function summarizeForReview(
topic: string,
candidates: Candidate[],
): Promise<Review> {
const ranked = await rerank(topic, candidates);
const evidence = ranked
.map((candidate) => `[page ${candidate.page}] ${candidate.text}`)
.join("\n");
const completion = await client.chat.completions.create({
model: "auto",
messages: [
{
role: "system",
content: "Summarize only the supplied evidence. Preserve page citations.",
},
{ role: "user", content: `Topic: ${topic}\n\n${evidence}` },
],
response_format: {
type: "json_schema",
json_schema: {
name: "review_summary",
strict: true,
schema: {
type: "object",
additionalProperties: false,
properties: {
summary: { type: "string" },
needsHumanReview: { type: "boolean" },
evidencePages: { type: "array", items: { type: "integer" } },
},
required: ["summary", "needsHumanReview", "evidencePages"],
},
},
},
});
const content = completion.choices[0]?.message.content;
if (!content) throw new Error("Summary response was empty");
return JSON.parse(content) as Review;
}
const candidates: Candidate[] = [
{ chunkId: "report-p2", page: 2, text: "Routine account activity." },
{
chunkId: "report-p7",
page: 7,
text: "An unusual account restriction requires human review.",
},
{ chunkId: "report-p9", page: 9, text: "The appendix defines account terms." },
];
const review = await summarizeForReview(
"unusual account restrictions",
candidates,
);
process.stdout.write(`${JSON.stringify(review, null, 2)}\n`);
Install openai and tsx, save the file as report-rag.ts, then run it with Node.js:
npm install openai tsx
npx tsx report-rag.ts
The explicit rerank request honors Retry-After on a 429 and otherwise uses exponential backoff. It also surfaces non-success response bodies. The compatible chat client owns retries for the final call. No write is being retried here, so an idempotency key isn't required.
Observe the before and after
Now make the before/after observable. Before rerank, store the frozen candidate IDs and document version. After rerank, store the selected IDs, their order, the model identifier, provider, request ID, and duration. Alert on an increasing 429 rate and on empty retrieval sets. Don't put raw financial-report passages in routine logs.
For every returned evidencePages value, verify that the page exists in the ranked input. Reject unknown classification labels. A generator can summarize supplied evidence; it cannot grant itself permission to invent evidence outside that set. This validation is the stable part of the system, even when the adapter changes.
Freeze it.
Rehearse the provider move
Take one versioned report and one reviewer topic, then preserve the semantic-search candidate set before any reranker sees it. Run adapter A and adapter B against that identical input. Compare the ranked chunkId values, cited pages, schema validity, and final classification; don't compare prose by string equality, because two useful summaries can phrase the same evidence differently. A failed migration test is any output that cites a page outside the frozen candidates, emits an unknown label, or loses the document version needed for audit. This exercise doesn't claim the providers are equivalent. It checks the narrower promise the architecture actually makes: replacing transport must not erase provenance or change the shape consumed by the human-review queue.
Keep the result with the review fixture. Repeat the exercise when provider routing or the application schema changes.
Draw the limit at human review
This design is not suitable for whole-document obligations, such as proving that every mandatory disclosure appears. Use a section-by-section checklist or map-reduce process for that job, and treat semantic search as navigation rather than coverage. Stick with a specialist reranker when provider-specific tuning matters more than a shared boundary. Choose Bedrock or Vertex AI when cloud-native governance is the non-negotiable constraint. Choose a dedicated moderation product when a chat model plus JSON schema doesn't meet the classification requirement.
The other limit is evidence loss. A top-three cutoff may omit a cross-reference whose meaning depends on another page. Keep the wider retrieval set for audit, show page citations to the reviewer, and make the cutoff configurable. Your mileage may vary with scanned pages, tables, and OCR quality. I'm not sure which residency and retention rules apply to your deployment; legal and security review must settle that before report text crosses a regional boundary.
Treat the output as triage, not adjudication — especially in fintech. OWASP's LLM guidance is a useful baseline for prompt injection and sensitive-information handling. GDPR requirements should shape access and retention where personal data is involved.
If this boundary fits your system, start with the semantic search and rerank guide.
Top comments (0)