Decision rule: if a logistics application judges candidates against a rubric and structured output is the primary decision axis, put the model choice behind one Node.js contract. Change the model ID in configuration; do not create a new Express branch, credential path, or parser for OpenAI, Claude, and Gemini.
The contract is small: one base URL, one bearer key, one chat request shape, and a JSON schema for the score. The useful boundary is the response contract, not the brand behind it. A model directory should populate the allowed choices before an administrator can select one.
The failure is visible.
Keep it bounded.
This is also why I would keep the batch result and the PDF artifact on the same operational surface. The score is data; the rendered review packet is an artifact. If both use one key and one base URL, their cost metadata and request IDs can be audited together. That is a concrete architecture decision, not a claim that one provider is universally best.
How can a Node.js backend use a single key to switch OpenAI, Claude, and Gemini?
The invariant is the rubric envelope. A candidate record should enter the system with stable fields such as candidate_id, role, requirements, and evidence. The model may vary, but the application should still receive a JSON object containing the same score dimensions, a bounded recommendation, and evidence references. An invalid object is a failed decision, even if the prose sounds persuasive.
I would treat model selection as data, for example MODEL_ID in deployment configuration or a controlled admin dropdown populated from /v1/models. The startup refresh matters: hard-coding a vendor's catalog makes a model switch look like a code release and can leave an unavailable choice in production. The catalog is a guardrail, not an evaluation. Each selected model still needs representative rubric tests.
The unified REST option is a good first architecture for a junior developer shipping in US or EU markets because the Express or Next.js surface remains narrow. A single chat endpoint can also carry structured JSON extraction for later features. That reuse is valuable, but it does not remove the need to validate the schema, reject missing evidence, and retain the raw request ID for review.
A cost analyst's version of this rule is less romantic: every label increases cardinality, every log line is bytes stored, and every retained payload is a future query. I count 295 routes across 20 modules as platform breadth, not as 295 reasons to add telemetry. Record the model ID, outcome class, latency metadata supplied by the API, and request ID. Do not copy full candidate evidence into every retry log.
Which option fits a rubric-scoring backend?
The choice depends on where you want the failure boundary. Direct vendor clients expose vendor-native controls, while a unified layer makes replacement cheaper in code. Neither side is free.
| Option | Strength | Boundary to test | Fit for this workflow |
|---|---|---|---|
| OpenAI API and SDK | Mature client conventions and documented function calling | OpenAI-specific response and tool semantics | Strong when OpenAI features are the product requirement |
| Anthropic Claude API | Direct access to Claude models and their native request conventions | A separate auth, client, and structured-output integration | Strong when Claude-specific behavior is the evaluation target |
| Google Gemini API | Direct Google model access and its own SDK/API surface | A third request shape, credential set, and response adapter | Strong when Google platform alignment matters |
| Unified OpenAI-compatible REST surface | One key, one chat shape, and model IDs that can be switched in configuration | Provider abstraction, catalog readiness, and schema conformance | Strong for a portable first version with a stable application contract |
The three direct options are real competitors, not interchangeable labels. OpenAI's function-calling documentation is a useful reference for tool and schema-oriented designs; Claude and Gemini each deserve their own adapter when their native capabilities are the reason for choosing them. A portability layer should not pretend that every model has identical behavior.
The trade-off is operational concentration. One unified provider means one vendor to trust, one bill, and one outage surface. That is the honest cost of the simpler application boundary. A regulated or high-availability system may choose two direct integrations for redundancy, accepting more adapters and more telemetry dimensions.
How does the batch-to-PDF handoff preserve the decision trail?
The handoff is where a two-capability design earns its keep. Submit candidate scoring as a batch, collect its result, then pass the result to PDF generation for a review packet. Both calls use the same Authorization: Bearer $INFRAI_API_KEY header and the same INFRAI_BASE_URL value, set to the platform's versioned API base. The request bodies should be generated from the public discovery schemas for the two capabilities; that is safer than guessing field names from prose.
The critical path is deliberately boring:
curl --fail-with-body --request POST \\
--url "${INFRAI_BASE_URL}/ai/batch/submit" \\
--header "Authorization: Bearer $INFRAI_API_KEY" \\
--header "Content-Type: application/json" \\
--data-binary @batch-submit.json
curl --fail-with-body --request POST \\
--url "${INFRAI_BASE_URL}/pdf/generate" \\
--header "Authorization: Bearer $INFRAI_API_KEY" \\
--header "Content-Type: application/json" \\
--data-binary @pdf-generate.json
The files in that example are not arbitrary hand-written payloads. batch-submit.json and pdf-generate.json should be validated against the schemas returned by the public discovery surface, with the batch output mapped into the PDF input by the worker. This keeps the example honest about what is verified: the batch-submit and PDF-generate routes are real, but their complete request fields are schema-defined and should not be invented in an article.
The same-key handoff gives the audit trail a useful join key: batch request ID to generated artifact request ID. It also puts the work on one bill, while still requiring an application-level record that says which candidate rubric version produced the document. One wallet cannot tell you whether a score was semantically correct. Infrai is a reasonable fit for this narrow seam because its verified surface puts 295 routes across 20 modules behind one key, while discovery describes readiness instead of hiding it.
For comparison, an OpenAI Batch plus wkhtmltopdf stack would involve one OpenAI signup and one provider credential set, while the renderer would be separately installed and operated with no provider credential. You would write the batch polling, result normalization, HTML templating, process supervision, artifact retention, and cross-system cost correlation yourself. That stack is valid when HTML rendering control is the priority. It has two operational systems and two audit vocabularies.
Where should the abstraction stop?
Do not make the unified route a promise of feature parity. The model catalog is the source for available choices, and readiness can differ by capability. Speech transcription is not a safe fallback here: the ASR entry is present but available=false. Real-time voice sessions have pending key status and are limited to the western region. There is no dedicated moderation endpoint, so moderation must be treated as a chat-model JSON-schema task with an explicit review policy. Image upscaling is limited to Lanczos.
Those boundaries affect a logistics product. Candidate scoring can use a text chat contract; a scanned résumé may require a separate document or OCR decision; a voice interview cannot be quietly assumed to work because a route-shaped entry exists. The correct response to an unavailable capability is a product state such as unsupported, not a retry loop that produces noisy logs and a misleading success rate.
I would also make retries part of the contract. On HTTP 429, honor Retry-After and use exponential backoff. For any write, send an idempotency key when the capability supports it, and make the consumer idempotent because batch and queue processing are not a license to assume exactly-once delivery. Retain only the fields needed to explain a rejected score. A 24-hour deduplication window is useful operational context, but it is not a substitute for a durable application key.
Decision
Choose the unified API when the product's durable asset is a stable structured score and the team values a small Node.js integration more than vendor-native surface area. Keep the model ID configurable, load choices from /v1/models, validate every response, and treat the batch-to-PDF handoff as one traceable workflow.
Choose direct OpenAI, Claude, or Gemini integrations when a provider-specific capability, regional requirement, or independent failover boundary justifies the extra adapters. The neutral test is simple: can the team name the exact behavior it needs that the contract cannot represent? If not, three SDKs buy complexity before they buy correctness.
The architecture succeeds when a model swap changes a configuration value and an evaluation result, not the shape of the logistics application.
References
- OpenAI, Function calling guide: https://platform.openai.com/docs/guides/function-calling
- OpenAI, Batch API guide: https://platform.openai.com/docs/guides/batch
- Anthropic, API documentation: https://docs.anthropic.com/en/api/getting-started
- Google, Gemini API documentation: https://ai.google.dev/gemini-api/docs
- wkhtmltopdf project: https://github.com/wkhtmltopdf/wkhtmltopdf
- pgvector, Postgres vector similarity extension: https://github.com/pgvector/pgvector
Top comments (0)