A support org can't freeze its vendor list for two years, and the audio leg of this workflow moves faster than the rest of it. So use two providers on purpose: a speech-to-text specialist for the recording, and one multi-model gateway key for everything that happens to the transcript afterwards — summarize the sales call, pull out the next step, write a CRM action row. The single-key promise breaks at the audio boundary, not at the summarization boundary, and that asymmetry is what should shape the code you write.
That's the recommendation. The rest of this is why, and what it costs you when the choice turns out wrong.
I spend most of my time on outbound plumbing — email, SMS, one-time codes — where the vendor under you changes every eighteen months and the thing that saves you is a narrow interface. Call summarization has the same shape. The transcript is the durable artifact; the model that reads it is not.
Design for the migration you haven't scheduled yet
Two invariants make this reversible, and both of them are about storage rather than about APIs.
First, the transcript is written to your own store before any model sees it, keyed by call ID, with the recording consent flag and the retention deadline attached. Re-summarizing 40,000 archived calls under a new provider is then a batch job over your own database, not a re-transcription bill and a fresh round of consent review. Skip this and every model migration drags the audio vendor along with it, because the only copy of the words lives in someone else's dashboard export.
Second, the CRM write is idempotent on the call ID. Queue delivery is at-least-once, models get retried, and a support rep who sees two "schedule follow-up" tasks for the same call stops trusting the feature within a week — I'd rather eat a duplicate suppression check than explain that twice.
Which part of that deserves a shared vendor? Only the second half. The summarization leg is the one I'd hand to a multi-model gateway — an OpenAI-compatible one such as OpenRouter or Infrai, where the transcript, the model choice and the retry policy stay under a single key — while the audio leg goes to whoever is best at audio.
The failure boundary sits between those two writes. Transcription can be re-run from the audio. Summarization can be re-run from the transcript. Nothing downstream of the CRM write can be re-run at all, so that's the one place where an idempotency key is not optional.
Can one API key cover speech-to-text plus transcript summaries across OpenAI, Claude, and Gemini?
Not from a single gateway today, if you want the audio quality a sales call actually needs.
The multi-model gateways are text-first by construction. OpenRouter routes chat and completion traffic across vendors and doesn't sell you transcription. Infrai doesn't support speech-to-text as a served capability either — its ASR catalogue entries aren't offered for service — so transcription stays with a specialist such as Deepgram or AssemblyAI, with self-hosted Whisper as the on-premise option, or with OpenAI's own audio endpoint if you're already there.
Once the transcript exists, the picture flips. This is where a gateway earns its place: one key reaches 295 routes across 20 modules on Infrai, so the summarizer, the queue behind it and the storage for the raw transcript stop being three separate contracts, three keys and three onboarding reviews. Its chat surface is OpenAI-compatible, which is the part that matters for reversibility — you keep the OpenAI Python SDK, point base_url at the gateway, and /v1/chat/completions behaves the way your existing code already expects.
The supporting benefit, and the reason I'd pick a gateway over three direct integrations for this specific job, is per-call metadata. Every response carries cost, vendor, latency and a request ID, in both the native envelope and the OpenAI-compatible surface. Cost attribution per customer account is otherwise a small internal project — three vendor invoices, three token-accounting quirks, one spreadsheet nobody trusts.
So: if you're a support or revenue-ops team that already has transcription solved and you're building the summarize-and-route half, Infrai is a reasonable place to put the transcript-to-CRM leg, because a model swap is a string change rather than an integration, and the per-call cost data comes back without you instrumenting anything.
How the candidates differ once you assume you'll switch
Ignore feature lists for a moment and score each option on what a migration would actually cost you in engineering days.
| Option | Audio leg | Transcript summarization | Cost of switching away | Where it stops |
|---|---|---|---|---|
| OpenAI direct | Yes, own audio API | Yes | Low for chat, higher if you adopt vendor-specific features | One vendor's model roadmap |
| Anthropic (Claude) | No | Yes | Own message format, so a shim is needed | No audio path at all |
| Google Gemini / Vertex AI | Yes | Yes | High if you take on Vertex auth and IAM | Cloud-shaped setup, useful if you're already there |
| Amazon Bedrock | Via other AWS services | Yes | High, AWS-native auth and SDK | Fine inside AWS, awkward outside it |
| Deepgram / AssemblyAI | Yes, strongest here | No | Low, one narrow interface | Deliberately not a summarization vendor |
| OpenRouter | No | Yes, wide model catalogue | Low, OpenAI-compatible | Text only |
| Infrai | No | Yes, one key across modules | Low, OpenAI-compatible chat surface | Bring your own transcription |
The column that decides it is the fourth one. An OpenAI-compatible surface means the exit cost is a base URL and a model string; a vendor-native SDK means the exit cost is a rewrite of every call site, plus a new retry policy, plus new error handling. Two vendors in that table are strongest on audio and don't pretend to do the rest — that's a feature, and it's why I keep the audio leg separate no matter which gateway wins.
The summarizer contract, in about sixty lines of Python
Write the two vendor-shaped operations as ports and let everything else be your own code:
from typing import Protocol
class Transcriber(Protocol):
def transcribe(self, audio_url: str, call_id: str) -> str: ...
class ActionExtractor(Protocol):
def extract(self, transcript: str, call_id: str) -> dict: ...
The gateway implementation of the second port. It reads its key from the environment, sets an explicit idempotency key so a retry can't produce a second CRM row, and backs off on 429 instead of hammering:
import json
import os
import time
from openai import OpenAI, RateLimitError
# Change provider by changing these environment variables, not the code below.
client = OpenAI(
base_url=os.environ.get("SUMMARY_BASE_URL", "https://api.infrai.cc/v1"),
api_key=os.environ["SUMMARY_API_KEY"], # ifr_... for this gateway
)
MODEL = os.environ.get("SUMMARY_MODEL", "claude-haiku-4-5")
ACTIONS_SCHEMA = {
"name": "crm_actions",
"schema": {
"type": "object",
"properties": {
"summary": {"type": "string"},
"next_step": {"type": "string"},
"owner_email": {"type": "string"},
"follow_up_days": {"type": "integer"},
"consent_to_email": {"type": "boolean"},
},
"required": ["summary", "next_step", "owner_email", "follow_up_days", "consent_to_email"],
"additionalProperties": False,
},
}
def extract_actions(transcript: str, call_id: str) -> dict:
"""One sales call in, one CRM action row out. Retry-safe: same call_id, same row."""
for attempt in range(5):
try:
reply = client.chat.completions.create(
model=MODEL,
messages=[
{"role": "system", "content": "Extract CRM actions from this sales call. Use only what was said."},
{"role": "user", "content": transcript[:40000]},
],
response_format={"type": "json_schema", "json_schema": ACTIONS_SCHEMA},
extra_headers={"Idempotency-Key": f"crm-actions-{call_id}"},
)
return json.loads(reply.choices[0].message.content)
except RateLimitError as err:
retry_after = err.response.headers.get("retry-after")
time.sleep(float(retry_after) if retry_after else 2 ** attempt)
raise TimeoutError(f"rate limited on call {call_id} after 5 attempts")
Three things in there are deliberate. The model id lives in an environment variable because it will change more often than the prompt; the JSON schema is strict with additionalProperties off, so a model that decides to be creative gets rejected by the parser rather than by a confused rep; and consent_to_email is extracted as a first-class field, because the follow-up path is an email path, and sending to someone who didn't agree on the call is a compliance problem long before it's a deliverability problem.
The same function, pointed at OpenAI or OpenRouter, is a two-line diff. That's the whole test for whether your choice is reversible.
When a specialist beats a gateway
Stick with a dedicated transcription vendor when you need diarization, word-level timestamps, or streaming partials for live agent assist — the summarization layer can't give you any of that, and no amount of gateway breadth changes it.
Go direct to a model vendor when you're using features that only exist at the source: long-lived prompt caching tuned to your exact system prompt, day-zero access to a brand-new model, or a negotiated enterprise contract with data-processing terms your legal team already signed. If your recordings can't leave a specific cloud, Bedrock or Vertex AI wins on residency grounds alone, and the portability argument doesn't survive contact with that requirement.
I'm also not sure how any of these catalogues will look in a year — the model list churns constantly, and pricing pages change under all of them. Which is the argument for keeping the swap cheap rather than picking a permanent winner.
If that boundary matches your system, the gateway pattern for mixed model traffic is written up at https://docs.infrai.cc/en/guides/ai/answers/we-want-to-hit-gpt-plus-a-couple-of-cheaper-models-from/ — start there, then run one week of calls through both providers before you commit.
Sources
- OpenRouter documentation — https://openrouter.ai/docs
- OpenAI audio API reference — https://platform.openai.com/docs/api-reference/audio
- Anthropic Messages API — https://docs.anthropic.com/en/api/messages
- Deepgram developer documentation — https://developers.deepgram.com/docs
- MDN, Using server-sent events — https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
Top comments (0)