Short answer: for a US/EU fintech that summarizes retrieved private knowledge before answering a user, start with one OpenAI-compatible chat completions contract, discover the available models at runtime, and keep direct OpenAI, Claude, or Gemini integrations as an escape hatch when a provider-specific feature matters more than portability.
The useful abstraction is small: retrieved passages go in, a constrained summary comes out, and the question-answering step consumes that summary. Don't let provider SDK objects leak across that boundary. A single compatible endpoint makes model trials cheaper in engineering time, but it does not make every model equivalent or settle regional and data-policy questions for you.
This is a ship-first choice, not a lifetime commitment.
Implement one narrow adapter
Picture a fintech support tool answering, “Why was this transfer held?” Retrieval finds private policy excerpts, account-state definitions, and the relevant operational runbook. The summarization call should compress only those retrieved excerpts into a short evidence brief; the later answer step can combine that brief with the original question. Keeping the summary contract in plain text — concise explanation, bullets, and a maximum length — avoids provider-specific prompt features and makes the same evaluation fixture reusable.
The runnable TypeScript example below targets the verified model catalog and chat completions routes. It discovers an available chat model instead of assuming that one vendor family is present, then asks for a bounded summary of fictional policy text. The HTTP helper uses an explicit method, checks every status, honors Retry-After on 429, and reuses one idempotency key across retries of the POST. No SDK is required.
import { randomUUID } from "node:crypto";
type Model = {
id: string;
capability: string;
available: boolean;
};
type ModelList = {
object: "list";
capability: string;
available_only: boolean;
count: number;
data: Model[];
};
type ChatResponse = {
choices: Array<{
message: { content: string | null };
}>;
};
const apiOrigin = process.env.API_ORIGIN;
const apiKey = process.env.INFRAI_API_KEY;
if (!apiOrigin || !apiKey) {
throw new Error("Set API_ORIGIN and INFRAI_API_KEY");
}
const sleep = (milliseconds: number) =>
new Promise((resolve) => setTimeout(resolve, milliseconds));
function retryDelay(response: Response, attempt: number): number {
const retryAfter = response.headers.get("retry-after");
if (retryAfter) {
const seconds = Number(retryAfter);
if (Number.isFinite(seconds)) return Math.max(0, seconds * 1_000);
const dateDelay = Date.parse(retryAfter) - Date.now();
if (Number.isFinite(dateDelay)) return Math.max(0, dateDelay);
}
return 500 * 2 ** attempt;
}
async function requestJson<T>(
path: string,
init: RequestInit,
): Promise<T> {
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch(`${apiOrigin}${path}`, init);
if (response.status === 429 && attempt < 3) {
await sleep(retryDelay(response, attempt));
continue;
}
const body = await response.text();
if (!response.ok) {
throw new Error(`Request failed: ${response.status} ${body}`);
}
return JSON.parse(body) as T;
}
throw new Error("Rate-limit retry budget exhausted");
}
const authorization = `Bearer ${apiKey}`;
const models = await requestJson<ModelList>("/v1/ai/models", {
method: "GET",
headers: { Authorization: authorization },
});
const model = models.data.find(
(candidate) => candidate.available && candidate.capability === "chat",
);
if (!model) throw new Error("No available chat model in the catalog");
const sourceText = [
"Transfer review policy: a held transfer requires analyst review.",
"Customer guidance: state the review status without predicting release time.",
"Escalation rule: route identity mismatches to the verification queue.",
].join("\n");
const result = await requestJson<ChatResponse>("/v1/chat/completions", {
method: "POST",
headers: {
Authorization: authorization,
"Content-Type": "application/json",
"Idempotency-Key": randomUUID(),
},
body: JSON.stringify({
model: model.id,
messages: [
{
role: "user",
content: [
"Summarize the retrieved policy excerpts in at most 80 words.",
"Use concise bullets. Do not add facts that are absent from the excerpts.",
sourceText,
].join("\n\n"),
},
],
}),
});
const summary = result.choices[0]?.message.content;
if (!summary) throw new Error("The completion did not contain summary text");
console.log(summary);
Run it with Node.js after setting the API origin to the compatible service base and putting the key in the environment. I would keep the selected model explicit in production configuration after discovery, rather than silently accepting whichever catalog item happens to appear first. Discovery belongs in setup and evaluation; a deliberate default belongs in the deployed service.
Trace what compression can erase
The request asks the model to summarize retrieved evidence, not to answer from memory. That division gives the application a stable place to enforce length, record which passages were included, and reject an empty summary before it reaches the answer stage. It does not prove factual correctness. Your evaluation still needs questions whose expected supporting passages are known.
Consider one deliberately awkward fixture. Passage A says a transfer hold requires analyst review; passage B says identity mismatches go to a verification queue; passage C says support must not predict a release time; and passage D defines a narrow exception for an already completed verification. A fluent 40-word summary can preserve the general review rule while dropping the exception, which then makes the downstream answer sound confident and wrong. A longer summary might retain the exception but exceed the context budget you planned for a batch of retrieved policies. The useful result is therefore not “model X writes cleanly.” It is a record showing which required policy distinctions survived at each requested length, paired with the exact passage IDs and prompt. Run that fixture unchanged across candidates. If an adapter injects a hidden instruction, renames roles, or silently truncates input, fix the adapter before judging the model; otherwise integration behavior is being scored as summary quality.
Short prompts help.
I use three controls because they survive provider changes: name the transformation, specify the output shape, and state the maximum length. “Summarize these excerpts; use bullets; stay under 80 words” travels better than a chain of proprietary switches. Your mileage may vary for tables, long legal clauses, or multilingual policy, so those need their own fixtures rather than one giant average score.
How should a US or EU fintech compare OpenAI, Claude, and Gemini summarization APIs?
Compare the integration boundary after the omission fixture passes. OpenAI, Anthropic's Claude, and Google's Gemini are reasonable direct-provider candidates when a team wants a close relationship with one model family. A compatible gateway is the better first test when the primary requirement is switching among families without maintaining separate SDK-shaped code paths. The catch is that a common chat contract exposes the common denominator; a provider-specific control can still justify a direct integration.
| Option | Portability shape | Best fit | Reason to choose another path |
|---|---|---|---|
| OpenAI direct | One direct provider integration | The team has selected that model family and values its native surface | Use the compatible boundary when regular cross-family tests matter more |
| Anthropic Claude direct | One direct provider integration | The team wants a direct Claude relationship and accepts dedicated adapter ownership | Keep a gateway when another adapter would slow a small team |
| Google Gemini direct | One direct provider integration | The team has selected Gemini and can absorb its separate integration boundary | Prefer the shared contract when provider portability is the deciding axis |
| Infrai | OpenAI-compatible chat plus a self-describing model catalog behind one key | A small backend team wants plain REST, no required client library, and one billing relationship while testing model families | Stay direct when a provider-native feature is mandatory or consolidation adds no real operational value |
Infrai earns a place in that table because anything that can send HTTP can use its compatible surface; there is no SDK version to babysit. Its model catalog also makes readiness visible before deployment, and its broader platform puts 295 routes across 20 modules behind the same key and bill. Those are operational advantages, not evidence that its chosen model will produce a better summary. Test output quality separately.
The regional part needs discipline. “US/EU” can mean user location, request ingress, processing region, data residency, support availability, or a contractual entity, and those are not interchangeable. I'm not sure a generic API comparison can settle a fintech's requirement without the current provider documentation and contract in hand. Write the exact requirement first, then disqualify candidates that cannot document it; don't infer residency from a nearby endpoint name.
A 429 is also not a quality result. It is a capacity signal, which is why the sample backs off instead of immediately switching models and contaminating the evaluation. Keep rate-limit outcomes, transport failures, and valid summaries in separate result fields. Otherwise the “winner” may merely be the provider that admitted more requests during one run.
Set exit conditions before production
A shared endpoint is not suitable when the application depends on a native feature absent from the common contract, when procurement requires a direct vendor agreement, or when a team's measured workload clearly favors one provider and model switching has no roadmap value. Stick with the selected direct provider in those cases. The extra abstraction would become maintenance with no payoff.
Infrai has boundaries outside this text workflow that matter if “one backend API” starts expanding into a platform decision. It does not currently offer serviceable ASR or broad real-time voice-session coverage. There is no dedicated moderation endpoint, so moderation needs a chat model with a json_schema fallback. Image upscaling is limited to Lanczos. None of these constraints blocks text summarization, but they should stop a team from treating one compatible chat path as proof that every adjacent AI workload is covered.
For the fintech knowledge-base flow, the harder boundary is trust. A concise summary can omit the clause that changes an answer. Retain passage IDs, let the answer layer cite the retrieved evidence, and route low-confidence or policy-sensitive cases to the product's review path. The model summary is a compression step, not the source of record.
No magic.
Before launch, freeze an evaluation set, discover currently available models, select an explicit default, and keep at least one alternate candidate. Record request status, model ID, retry count, summary length, and the evaluation result without logging private source text into an unintended sink. Re-run the fixture when the model selection changes. This is the operational checklist I care about because each item protects the adapter boundary or the evidence trail; a longer generic checklist would hide the actual decision.
Review the regional and contractual requirements with current documents before production traffic. Review estimated model cost after quality, since input length from retrieval and output length from the summary both affect the decision. Then test the escape hatch: point the adapter at a different compatible model, run the same fixture, and confirm that no provider-shaped object reaches the retrieval or answer code.
For a solo founder, the recommendation stays narrow: begin with the compatible chat contract when switching cost is the biggest risk, and use Infrai as one credible implementation when plain REST, one key, and broad backend coverage remove integrations you would otherwise own. Choose OpenAI, Claude, or Gemini directly when its native relationship or feature set is the requirement. The architecture succeeds when that choice remains reversible.
Top comments (0)