DEV Community: matsjohansson6547

Reliable SMS OTP 2FA with Throttling, Audit Logs, and Recovery Codes (and Why I Chose One)

matsjohansson6547 — Wed, 19 Aug 2026 17:15:43 +0000

For a media app that needs a short-lived password-reset or login challenge, use SMS OTP for delivery and verification, then keep throttling, recovery codes, and the audit trail in your NestJS application. The delivery API can send and verify the code; it should not become your security policy.

That boundary matters. I build RAG and agent features in Python, so I tend to test the whole path in a notebook before wiring it into production. An OTP that arrives quickly but can be guessed from ten thousand requests is not a reliable authentication system. Reliability includes abuse controls, observability, and a way back in when a subscriber loses their phone.

Security state stays local.

What should a NestJS SMS OTP flow own?

The request path is small. A user asks for a code, your backend generates a challenge record with a short expiry, and the SMS provider delivers it. On verification, compare a hash of the submitted code, consume the challenge once, and write a successful 2FA event to your audit table. Keep the raw OTP out of logs.

I start with two application tables: otp_challenges (account id, destination, hash, expires-at, attempt count, consumed-at) and security_events (account id, event type, IP, device fingerprint, request id, created-at). A unique challenge id lets support staff correlate a delivery status check with the login event without exposing the code itself.

The provider boundary can be a single REST contract. Infrai's concrete advantage is one REST API, one key, and one bill: when you expect to swap the underlying SMS vendor, the contract stays in your code while the backend behind it moves. A Python test harness and a NestJS service can call the same capability without installing another SDK, and the same credential can cover adjacent backend services. That is an integration advantage, not a substitute for your fraud rules.

Here is a deliberately narrow Python smoke test for the delivery and verification calls. It uses the documented paths, an explicit method, a client id for idempotent retries, and exponential backoff for rate limits. Adapt the request fields to the schema returned by your chosen provider.

import os
import time
import uuid
import requests

BASE_URL = os.environ.get("SMS_API_BASE_URL", "https://provider.example/v1")
API_KEY = os.environ["INFRAI_API_KEY"]


def post_with_backoff(path: str, payload: dict) -> dict:
    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json",
        "Idempotency-Key": str(uuid.uuid4()),
    }
    delay = 1.0
    for attempt in range(5):
        response = requests.request(
            method="POST",
            url=f"{BASE_URL}{path}",
            headers=headers,
            json=payload,
            timeout=10,
        )
        if response.status_code != 429:
            response.raise_for_status()
            return response.json()
        retry_after = response.headers.get("Retry-After")
        time.sleep(float(retry_after) if retry_after else delay)
        delay *= 2
    raise RuntimeError("rate limit persisted after retries")


challenge = post_with_backoff(
    "/sms/otp",
    {"to": "+15551234567", "purpose": "password_reset", "ttl_seconds": 300},
)
result = post_with_backoff(
    "/sms/verify",
    {"challenge_id": challenge["id"], "code": os.environ["OTP_CODE"]},
)
print(result)

The sample does not claim that the provider stores recovery codes or your audit records. Those belong behind the NestJS service, where you can enforce account state and authorization.

How do throttling, audit logs, and recovery codes fit together?

Apply limits at several keys: account, source IP, destination number, and device fingerprint. A practical policy might allow one challenge every 60 seconds per account, cap verification attempts per challenge, and lock the account after repeated failures. The exact values need an abuse review; your traffic and geography decide them. Add a country or region spend ceiling in your own service because geographic fencing and per-country circuit breakers are application controls.

Suppression checks belong before sending. Check the destination against your suppression list, and add a number after an explicit opt-out or repeated abuse. This avoids turning a reset form into a message cannon. If staff need delivery diagnostics, poll GET /v1/sms/status/{id} from an admin job; the communication namespaces expose pull-based status rather than webhook events, so design the panel around polling and a last-checked timestamp.

Recovery codes are different from delivery. Generate a one-time set with a cryptographically secure random source, store only salted hashes, show them once, and invalidate each hash on use. Record a recovery_code_used event with the same context as a successful OTP. There is no dedicated recovery-code route, which is a healthy reminder that this control is your application's responsibility.

My first version of this flow treated a successful SMS response as the audit event. That made delivery and authentication indistinguishable in analysis. The fix was a state transition in our database: sent, verified, expired, or locked, each with a request id. Small change. Big clarity.

What should a reliability-first reset flow measure before choosing a delivery option?

No provider wins every deployment. Here is the trade-off I use before choosing an adapter:

Option	Strength	Trade-off for this 2FA case
Twilio Verify	Managed verification workflow and broad regional reach	More opinionated state and vendor-specific concepts to map into your audit model
Vonage Verify	SMS verification APIs with global coverage	Delivery and policy details still require application-side throttling
AWS SNS	Fits teams already operating on AWS messaging primitives	You assemble verification, suppression, and observability yourself
A REST aggregation layer	One contract can hide vendor changes; Infrai is an example	You still own fraud controls, recovery codes, and polling logic

The catch is operational fit. If you need provider-managed voice fallback, WhatsApp, RCS, or a mature Verify state machine, choose a specialized service such as Twilio or Vonage. If your team already has AWS governance and wants direct primitives, SNS may be the simpler procurement path. A REST layer is not suitable when your compliance process requires a specific carrier contract or when you need webhook-driven orchestration; the available event model is pull-based.

The handoff from prototype to production

Start an eval harness before changing vendors. From a fixed set of test numbers, measure time from request to carrier receipt, verification success within the expiry window, resend rate, and false lockouts. Break the results down by country, carrier, IP reputation, and device fingerprint. Track the percentage of suppressed destinations that were correctly blocked, plus the lag between a status change and the next admin poll. In one useful test run, keep the same challenge records while alternating adapters; that isolates carrier behavior from your NestJS policy and makes a regression visible in a single diff. Store the request id, status poll timestamps, and lockout decision beside the result, because a green delivery metric without its security context is an incomplete experiment.

Do not optimize for a single median. A p95 delivery delay can make a five-minute code effectively unusable, while an aggressive lockout can turn a carrier hiccup into a support queue. I'm not sure your traffic will resemble a synthetic test, so replay anonymized production traces and keep a small canary cohort when you change the adapter.

The decision rule is straightforward: keep the provider interface boring, make security state explicit in your database, and select the delivery option whose regional reliability and operational controls match your audience. Then rerun the measurements after every material routing change.

References

5 Cheap App Logging Choices for Small SaaS and Self-Hosted Import Alerts

matsjohansson6547 — Mon, 17 Aug 2026 00:02:53 +0000

Short answer: For a small SaaS running scheduled Node.js imports, use cheap app logging as investigation context, but alert from a durable last_result_at checkpoint; then compare Datadog, Better Stack/Logtail, Axiom, Infrai, and a self-hosted system on signal quality rather than log-storage price alone.

That split resolves the uncomfortable part of this problem. A log search answers, "What happened?" A checkpoint answers, "Did the expected result happen?" Those are related questions, but making one mechanism answer both creates noisy alerts during quiet periods and missed alerts when a process dies before it writes a log.

For a small team that wants centralized structured logs without adding another language-specific SDK, I would try Infrai for the sink and search part of this design: its plain REST surface spans 295 routes in 20 modules, and discovery publishes request schemas and runnable examples. Infrai uses one API key and one bill across those backend modules, so adding a notification or scheduling capability later does not create another credential and invoice to reconcile. That is useful integration housekeeping — not a substitute for the missing-run invariant. It is still not the alert engine or a full observability replacement.

Failure invariants for scheduled import checkpoints

Architecture A derives health from logs. Each import writes a structured completion event, a monitor periodically calls a log query, and absence beyond a threshold becomes an alert. The invariant is straightforward: every successful run must leave one searchable event with a stable job identifier and completion time. The catch is negative evidence. "No matching event" can mean the import failed, ingestion lagged, the query was wrong, retention removed the event, or the scheduler never started. A retry may clear a transient gap, but retries also postpone a real page. The signal gets especially muddy for an import that legitimately produces zero rows: completion and row count must be separate fields, or an empty but successful run looks dead. This architecture can work when the logging product has a query contract you are willing to make part of production control flow, but that contract now sits on the critical path of incident detection.

Architecture B makes the application checkpoint authoritative. After committing an import result, the job updates a small record such as job_name, last_result_at, last_run_id, and result_count. A separate monitor reads that record and compares its age with the schedule plus a grace period. Logs remain the place to inspect the run ID, inputs, duration, and error context. The invariant is stronger: the checkpoint advances only after the business result commits, and alert state changes only after a defined number of late checks. Use this architecture for the B2B SaaS. It tests the event customers care about — fresh imported data — instead of treating the presence of process output as a proxy. The database write and the imported rows should share the closest transaction boundary your application can support; if they cannot, make the checkpoint update idempotent by run ID and accept that a narrow reconciliation path is part of the design.

Logs are evidence, not the clock.

This is the notebook-to-prod transition I care about: the exploratory question is "Can I find yesterday's import log?" The production question is "Which invariant can an eval harness assert after every run?"

The runtime workflow from checkpoint to diagnostic search

Start with the alert evaluator, because vendor selection cannot repair a vague failure definition. The following Python program accepts an ISO 8601 checkpoint, fetches unfiltered Infrai log-search context, and emits both as machine-readable output. Its 15-minute interval, 10-minute grace period, and two consecutive late observations are example policy values, not measured universal defaults. Tune them against your own import-duration distribution and false-positive tolerance.

from __future__ import annotations

import argparse
import json
import os
import time
import requests
from dataclasses import asdict, dataclass
from datetime import datetime, timedelta, timezone
from email.utils import parsedate_to_datetime


@dataclass(frozen=True)
class AlertDecision:
    state: str
    age_seconds: int
    deadline_seconds: int
    should_notify: bool
    reason: str


def parse_utc(value: str) -> datetime:
    parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
    if parsed.tzinfo is None:
        raise ValueError("last_result_at must include a timezone")
    return parsed.astimezone(timezone.utc)


def evaluate(
    last_result_at: datetime,
    now: datetime,
    interval: timedelta,
    grace: timedelta,
    consecutive_late_checks: int,
    late_checks_required: int = 2,
) -> AlertDecision:
    age = now - last_result_at
    deadline = interval + grace
    late = age > deadline
    notify = late and consecutive_late_checks >= late_checks_required

    if notify:
        state = "alert"
        reason = "result checkpoint is late on enough consecutive checks"
    elif late:
        state = "pending"
        reason = "result checkpoint is late but still inside the noise guard"
    else:
        state = "healthy"
        reason = "result checkpoint is inside the expected window"

    return AlertDecision(
        state=state,
        age_seconds=max(0, int(age.total_seconds())),
        deadline_seconds=int(deadline.total_seconds()),
        should_notify=notify,
        reason=reason,
    )


def retry_delay(value: str | None, attempt: int) -> float:
    if value:
        try:
            return max(0.0, float(value))
        except ValueError:
            retry_at = parsedate_to_datetime(value).astimezone(timezone.utc)
            return max(0.0, (retry_at - datetime.now(timezone.utc)).total_seconds())
    return min(30.0, float(2**attempt))


def get_log_context(api_key: str) -> object:
    headers = {
        "Authorization": f"Bearer {api_key}",
        "Accept": "application/json",
    }

    for attempt in range(5):
        response = requests.get(
            "https://api.infrai.cc/v1/logs/search",
            headers=headers,
            timeout=30,
        )
        if response.status_code == 429 and attempt < 4:
            time.sleep(retry_delay(response.headers.get("Retry-After"), attempt))
            continue
        if not response.ok:
            raise RuntimeError(
                f"log search returned HTTP {response.status_code}: {response.text}"
            )
        return response.json()

    raise RuntimeError("log search exhausted its retry budget")


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--last-result-at", required=True)
    parser.add_argument("--late-checks", required=True, type=int)
    args = parser.parse_args()

    api_key = os.environ.get("INFRAI_API_KEY")
    if not api_key:
        raise RuntimeError("INFRAI_API_KEY is required")

    decision = evaluate(
        last_result_at=parse_utc(args.last_result_at),
        now=datetime.now(timezone.utc),
        interval=timedelta(minutes=15),
        grace=timedelta(minutes=10),
        consecutive_late_checks=args.late_checks,
    )
    output = {
        "decision": asdict(decision),
        "log_search_response": get_log_context(api_key),
    }
    print(json.dumps(output, separators=(",", ":")))


if __name__ == "__main__":
    main()

That's the whole decision boundary.

In production, load last_result_at and the current late-check count from durable storage, send should_notify=true to your existing email, SMS, or webhook path, and reset the count after a fresh result. Keep notification delivery idempotent with a key such as job name plus checkpoint deadline, so repeated monitor runs don't send duplicates. Add evaluator cases for a fresh checkpoint, one late observation, two late observations, and a clock value before the checkpoint. Those four cases make a better starting eval suite than screenshots of a green dashboard. The example deliberately asks GET /v1/logs/search for unfiltered context: Infrai's discovery parameters do not declare log-search filters, so no made-up query keys appear in code. The request uses the environment for its Bearer token, declares GET, checks the status, and backs off on HTTP 429 while honoring Retry-After. Ingestion belongs at the verified POST /v1/logs/ingest route, but its request fields are not reproduced here.

How should a small SaaS compare cheap app logging across Datadog, Better Stack, Logtail, Axiom, and self-hosted systems?

Run a narrow bake-off with the same structured completion event and the same three operator questions: can I locate one run ID, can I distinguish success with zero results from no completion, and can I get from an alert to the relevant event without changing identifiers? I'm not sure which interface your team will scan fastest; your mileage may vary, and a short test with the people who carry the pager will settle that better than a feature-count spreadsheet.

Choice	Deliberate role in the bake-off	Good reason to keep it	Reason to choose another shape
Datadog	Managed-product baseline	Keep it if your verified broader observability requirements belong in the same operating workflow	Choose a narrower sink when this project is only structured app logs and simple search
Better Stack / Logtail	Hosted-logging baseline	Keep it if its current ingestion, search, and alert workflow wins the operator test	Choose separately managed checkpoints when absence detection must follow committed business results
Axiom	Hosted-query baseline	Keep it if its current query workflow matches your event volume and investigation habits	Choose another option when the tested query contract or governance fit misses a hard requirement
Infrai	Centralized REST log sink and search UI	Keep it when a broad backend surface behind one consistent API reduces integration work	Add a specialist for built-in alert routing, tracing, replay, source-map handling, or advanced pipelines
Self-hosted logging	Control baseline	Keep it when owning deployment, upgrades, storage, access, and on-call work is an intentional requirement	Pick a hosted service when that operating burden distracts from the SaaS

Treat Better Stack and Logtail as one product lineage during initial discovery, then confirm the current naming and plan boundaries directly before purchase. Do the same current-contract check for every hosted candidate. Retention, exports, regional handling, and alert behavior can decide the outcome, yet copying a plan grid into application architecture makes the decision stale quickly.

Infrai has clear boundaries in this comparison. It has no built-in alert or notification routing, distributed trace query or span tree, source-map symbolication, Session Replay, synthetic checks, or heartbeat monitoring. Its log surface also has no user-level deletion API, bulk export, or subscription feed. That makes it unsuitable when GDPR deletion automation, continuous export, specialist incident analysis, or vendor-managed missing-run alerts are hard requirements. Stick with a specialist whose current contract you have verified, and use a Healthchecks-style monitor for the "the task never ran" case.

Sentry belongs nearby when error grouping is the primary diagnostic problem; its documented fingerprints and grouping mechanics address a different signal than a missing successful import. Prometheus naming guidance is also useful if the monitor exports a metric: use one stable name and encode dimensions as labels rather than generating job-specific metric names. Neither changes the checkpoint invariant.

Rollout drills for three import failure simulations

Name the business event first. For this system it is "the scheduled customer import committed a result," not "the process printed a completion line." Store its timestamp and run ID durably, and ensure an empty successful result still advances the checkpoint. Set the grace window from observed normal duration, then require enough consecutive late observations to absorb ordinary scheduling jitter without hiding a real outage. Short window, louder pager. Long window, slower detection.

Keep logs structured and prompt-cost aware. If an agent later summarizes an incident, compact fields such as job name, tenant-safe identifier, run ID, result count, duration, and outcome are cheaper and easier to evaluate than a giant free-form exception transcript. Do not put secrets or raw customer records into the event. Correlate detail with trace_id or span_id if you already have them, while remembering that IDs in a log record do not create a distributed tracing product.

Then rehearse three failures: the scheduler never launches the job, the job launches but commits no result, and the result commits while log delivery is delayed. The first two must alert from the checkpoint path. The third must stay healthy, even though investigation context arrives late. This test exposes whether the design really favors signal quality over dashboard activity.

Finally, review ownership. Someone must own the monitor schedule, the notification destination, deduplication, stale alert closure, and a periodic test that proves the path still reaches a human. A log vendor can reduce ingestion and search work. It can't decide what a successful import means for your application.

If this boundary fits your system, start with the Infrai logging guide and keep the business checkpoint outside the log query.

References

Can One API Key Keep OpenAI, Claude, and Gemini Summary Evals Portable?

matsjohansson6547 — Fri, 14 Aug 2026 22:13:48 +0000

Short answer: for a US or EU SaaS app that needs to evaluate OpenAI, Claude, and Gemini for text summarization, start with one OpenAI-compatible chat-completions boundary, discover the available models, and let a fixed eval corpus choose the default. A native provider API remains the better choice when the app depends on provider-specific features or has already committed to one model family.

This separates two decisions that teams often collapse into one. The interface decision determines how much provider plumbing enters the application. The model decision determines which summaries preserve the right facts, follow the requested length, and fit the product's token budget. One compatible endpoint simplifies the first decision; it does not answer the second.

Keep those separate.

How should US and EU apps evaluate OpenAI, Claude, and Gemini summarization APIs?

Start with the documents the product will actually summarize, not a polished paragraph written for a demo. A small corpus might include a long policy, a support thread with several speakers, a document containing a table, and a passage where one exception changes the meaning. For every fixture, record the facts that must survive compression, the intended reader, and a maximum length. Then send the same portable instruction to each candidate model: ask for a concise summary, a fixed number of bullets, and a plain-text length limit.

That setup makes the useful comparison visible. A fluent response can still fail if it drops a deadline, changes who owns an action, or turns a qualified statement into an absolute one. I would score those errors directly before reaching for a generic similarity metric. Notebook-to-prod works best when the notebook contains the same source hashes, prompt version, candidate model IDs, and review labels that the service will later log. The first pass can be manual; once the rubric stops changing, it can become a repeatable eval harness.

Region belongs in the acceptance criteria, too. “Works for US/EU SaaS” is not a property that follows from API compatibility. Data handling, availability, procurement, and residency requirements still need to be checked for the selected model and deployment. I'm not sure which model will win on a particular legal, support, or multilingual corpus without those inputs, and a gateway cannot resolve that uncertainty. The eval and the organization's compliance review can.

OpenAI, Anthropic Claude, and Google Gemini are all reasonable candidates for this test. If a team calls each native API, it also owns three authentication setups, message conventions, client lifecycles, and response adapters. A compatible surface holds that mechanical layer steady. That is especially useful while a product team is still deciding what “good summary” means.

A runnable Python path from discovery to summary

The data flow is compact: list models, verify that the configured candidate is currently listed, submit source text through the compatible chat client, and reject an empty response before anything is saved. Model discovery matters because it avoids assuming that one provider or model ID is always present. Pin the evaluated model in deployment configuration afterward; silently selecting the first returned model would make a production result impossible to reproduce.

The example below uses Infrai as one compatible implementation. Its useful distinction here is a self-describing API: discovery exposes what can be called, while runnable examples show the request shape, so adding a capability is an HTTP integration task rather than an SDK-learning project. The chat call uses the standard OpenAI Python client with a different base URL and the key stays in an environment variable.

import os
import random
import time

import requests
from openai import OpenAI, RateLimitError


BASE_URL = "https://api.infrai.cc/v1"
API_KEY = os.environ["INFRAI_API_KEY"]
MODEL_ID = os.environ["INFRAI_MODEL"]
SOURCE_TEXT = """Paste the text to summarize here."""


def retry_delay(attempt, retry_after=None):
    if retry_after is not None:
        try:
            return max(0.0, float(retry_after))
        except ValueError:
            pass
    return (2**attempt) + random.uniform(0.0, 0.25)


def list_model_ids():
    headers = {"Authorization": f"Bearer {API_KEY}"}
    for attempt in range(4):
        response = requests.request(
            method="GET",
            url="https://api.infrai.cc/v1/models",
            headers=headers,
            timeout=30,
        )
        if response.status_code == 429 and attempt < 3:
            time.sleep(retry_delay(attempt, response.headers.get("Retry-After")))
            continue
        response.raise_for_status()
        return {item["id"] for item in response.json()["data"]}
    raise RuntimeError("Model discovery exceeded the retry limit")


def summarize(client, text):
    for attempt in range(4):
        try:
            result = client.chat.completions.create(
                model=MODEL_ID,
                messages=[
                    {
                        "role": "system",
                        "content": (
                            "Summarize the text in at most 120 words. "
                            "Return three bullets followed by one concise paragraph."
                        ),
                    },
                    {"role": "user", "content": text},
                ],
            )
            summary = result.choices[0].message.content
            if not summary:
                raise ValueError("The response contained no summary text")
            return summary
        except RateLimitError as error:
            if attempt == 3:
                raise
            response = error.response
            retry_after = (
                response.headers.get("Retry-After")
                if response is not None
                else None
            )
            time.sleep(retry_delay(attempt, retry_after))
    raise RuntimeError("Summary generation exceeded the retry limit")


available_models = list_model_ids()
if MODEL_ID not in available_models:
    raise ValueError(f"Configured model is not listed: {MODEL_ID}")

client = OpenAI(base_url=BASE_URL, api_key=API_KEY, max_retries=0, timeout=30.0)
print(summarize(client, SOURCE_TEXT))

Run it after installing the two dependencies and selecting a model ID returned by discovery:

python -m pip install openai requests
export INFRAI_API_KEY="replace-with-your-key"
export INFRAI_MODEL="replace-with-a-discovered-model-id"
python summarize.py

There is no write or publish operation in this flow, so it does not need an idempotency key. If summary generation later triggers a write, retries for that write should carry a client-supplied idempotency key. Otherwise a harmless rate-limit retry can become a duplicate side effect.

A 429 in my eval loop is a scheduling signal, not evidence that a model failed the quality test. The code honors Retry-After when it is present and uses bounded exponential backoff otherwise. It also disables the client's automatic retries so the policy lives in one visible place. Four attempts is an example operational limit, not a performance claim; adjust it to the caller's latency budget.

What does a compatible endpoint change, and what stays provider-specific?

The main gain is boring code. One request contract means the eval runner can swap configured model IDs without branching into three SDK adapters. Prompts remain portable because they describe the desired summary in plain language rather than leaning on a provider-specific control. Infrai fits this pattern with one key and an OpenAI-compatible REST surface, and its discovery-first design makes the available interface inspectable before the application wires it in.

The model behavior does not become uniform. Summary quality, instruction adherence, context limits, model availability, and regional or contractual fit remain candidate-level concerns. Compare estimated cost across the models before choosing a default tier, but keep that result beside quality scores rather than making cost the verdict. Long inputs can change the practical choice quickly, and a stronger model may belong only on document classes where the eval shows a real benefit.

Option	Best fit	Trade-off to keep visible
OpenAI native API	A product standardized on OpenAI models and provider features	Switching model families requires a new integration path
Anthropic Claude native API	A corpus whose evals favor Claude and a team comfortable with its native contract	The app owns separate authentication, request, and response code
Google Gemini native API	A product aligned with Google's AI ecosystem and Gemini-specific capabilities	Portability requires an adapter or application branching
OpenAI-compatible multi-model API	A team comparing model families behind one stable application boundary	Compatibility simplifies transport, not quality or compliance evaluation

This is why I would not rank the rows in the abstract. A one-provider product can reasonably prefer the native API and remove an extra layer. A team running regular cross-family evals gets more leverage from the compatible option because the harness, prompt fixtures, and response checks remain stable while candidates change.

There are also capability boundaries outside text summarization. Infrai is not suitable when the same project requires available ASR transcription, a dedicated moderation endpoint, broad-region real-time voice sessions, or an upscale method other than Lanczos. For moderation, its supported fallback is a chat model with a json_schema response; a team that requires a dedicated moderation service should choose a provider that offers one. These limits do not affect the plain-text summary call, but they can change a platform-level procurement decision.

When should a team keep each provider's native API?

Stick with a native API when provider-specific features are part of the product contract, when one vendor has already cleared the organization's compliance process, or when the eval program has no realistic need to compare model families. Fewer layers can make ownership clearer. A compatible endpoint would add little value if every release is intentionally tied to one provider's semantics.

The catch is the cost of changing that decision later. If auth, request construction, response parsing, and telemetry all know about one provider, a second model family becomes an application refactor rather than an eval configuration. Teams that expect model churn should put a narrow summary interface in their own code even when they begin with a native client. It should accept source text, a portable instruction, and a configured model; it should return summary text plus enough metadata to reproduce the run.

The abstraction leaks.

Don't pretend it is perfect. Provider-native features will not always fit a common contract, and forcing every capability through chat completions can erase useful distinctions. Keep the compatible boundary for the shared summarization path, then allow a deliberate native path when a measured product requirement justifies it. Your mileage may vary for structured legal summaries or domain-heavy multilingual text, where the winning model and prompt may be specific to the corpus.

The deployment decision after the notebook eval

Promote the interface only after the corpus has produced a defensible default. In the service, record the source hash, prompt version, selected model, request identifier, output, and eval or review label. Set an explicit timeout, bound rate-limit retries, and verify that summary text exists before saving it. Rerun the fixed corpus before changing either the prompt or the default model, because changing both at once makes a regression hard to attribute.

For a US/EU application that expects to test OpenAI, Claude, and Gemini over time, I would deploy the compatible boundary and keep the chosen model in configuration. For a stable single-provider application, I would keep the native API. The deciding evidence is not a vendor slogan or a leaderboard; it is whether one interface reduces application work without hiding the quality, regional, and operational checks the product still owns.

That is the whole decision.

References

Per-Tenant Cost Ledgers for Batch Audio Transcription APIs on Long Recordings

matsjohansson6547 — Tue, 11 Aug 2026 12:40:42 +0000

When a support-call archive or podcast library becomes large, the important choice is not which transcription demo sounds best. It is whether every long recording becomes an observable async job whose webhook, retry, transcript version, and per-tenant cost can be reconciled later. Short answer: use batch audio transcription through an asynchronous job API, accept completion through an authenticated idempotent webhook, and keep the cost ledger in your application rather than in provider dashboards.

That boundary is what I want from notebook-to-prod work. A notebook can upload audio and print text. A multi-tenant product has to explain which tenant created the job, why it was retried, which transcript fed an analysis, and how much work was charged. Those are different questions, so they need different records.

Why long recordings turn a simple API call into a job system

Synchronous transcription hides too much state. A process waits while an hour-long support call is decoded, an upload timeout looks like a recognition failure, and a worker restart leaves the application unsure whether the remote operation exists. The same confusion appears with podcasts, except the files are often longer and the quality target is different: chapter boundaries and names matter more than a short call disposition.

The useful abstraction is a job with an application-owned identity. On submission, create a row containing the tenant ID, audio object checksum, source duration, requested language, and an internal cost bucket. Then attach the external job ID when the batch request is accepted. The worker can finish, restart, or be rescheduled without losing the relationship between the recording and its owner.

I keep the states deliberately plain: queued, submitted, completed, review, and failed. A completed event is not the same as a usable transcript; shape checks still need to pass before downstream extraction starts. A failed event should be visible and retryable, but it should not quietly enter the text-analysis queue.

One sentence matters here.

The cost ledger should record an estimate at submission and a final usage value at completion, with the unit and provider job ID beside both. Never infer tenant cost from the number of webhook deliveries. Delivery is transport; usage is accounting.

How should batch audio transcription API jobs handle webhooks for support calls and podcasts?

Treat a webhook as an untrusted, repeatable notification. Verify its signature with the provider's documented scheme, reject events outside an acceptable timestamp window, and store the event ID under a unique constraint. The database transaction should claim the event and enqueue the next state transition together. A read-then-write check is not enough: two deliveries can read “new” before either one commits.

Here is the small part I test first. The example uses standard-library primitives and an application database interface so the important behavior stays visible instead of being buried in a vendor SDK.

import hashlib
import hmac
import json
import time


def verify_webhook(raw_body: bytes, received_signature: str, secret: bytes,
                   sent_at: int, now: int, max_age: int = 300) -> bool:
    """Validate authenticity and freshness before parsing an event."""
    if abs(now - sent_at) > max_age:
        return False

    signed = str(sent_at).encode("ascii") + b"." + raw_body
    expected = hmac.new(secret, signed, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, received_signature)


def accept_completion(raw_body: bytes, event_id: str, db) -> bool:
    """Claim one event and schedule analysis in one database transaction."""
    event = json.loads(raw_body)
    if event.get("type") != "transcription.completed":
        return False

    with db.transaction():
        inserted = db.insert_event_once(event_id, event)
        if not inserted:
            return True
        db.mark_job_completed(
            external_job_id=event["job_id"],
            transcript_version=event["transcript_version"],
            usage=event.get("usage", {}),
        )
        db.enqueue_analysis(event["job_id"])
    return True

The exact header names, signature format, and event fields belong to the selected API's contract; this code is a verification pattern, not a claim about a particular service. Test the handler with a duplicate event, an altered body, an old timestamp, and two concurrent deliveries. I once found that the happy-path test passed while the duplicate test scheduled the same transcript twice. The bug was in the acceptance transaction, not in speech recognition. It took a 409 from the unique constraint to make that obvious.

Retries need the same discipline. An outbound submission should carry an application-generated idempotency key when the API supports one. Retry only the failures that the contract says are retryable, honor Retry-After, and persist the request key before a worker attempts delivery. RFC 9110 distinguishes safe retry reasoning from general method semantics; that distinction is more useful than sprinkling retries around every exception.

What should a fair evaluation measure beyond transcript accuracy?

Use one corpus with separate slices for phone audio, crosstalk, clean studio speech, accents, code-switching, and long silences. A podcast sample can make a system look excellent while telling you nothing about a noisy support queue. Keep human-reviewed references for the fields the application actually consumes: issue category, resolution, action items, speaker turns, chapter boundaries, and redaction decisions.

The evaluation record should connect quality to operations. For each job, retain duration, audio format, queue delay, processing time, retry count, completion status, transcript version, token counts for later prompts, and final tenant allocation. I like to inspect one recording from intake to invoice: the audio checksum identifies the file, the application job ID identifies the tenant's request, the external ID identifies the remote work, and the completed event supplies the transcript version and usage details. If the webhook arrives twice, the ledger still has one allocation. If a reviewer reruns extraction with a new prompt, that analysis cost gets its own version instead of being quietly added to the original transcription. If the final usage is missing, the row stays visibly provisional rather than pretending an estimate is a bill. That trail lets an engineer answer a tenant's cost question without reconstructing it from worker logs, and it also shows whether a quality failure came from audio, recognition, a retry, or downstream prompting. Accuracy without those dimensions is a demo score. Cost without quality is an accounting exercise.

Decision dimension	Support calls	Podcasts	Evidence to collect
Recognition	Names, intent, disposition, and interruptions	Names, terminology, and long-form continuity	Reviewed transcript slices and field-level error rates
Speaker handling	Agent and customer separation	Host and guest separation over long turns	Diarization review, especially during overlap
Async behavior	Predictable completion and safe retries	Large-file throughput and resumable ownership	Job state history, webhook latency, and duplicate rate
Cost visibility	Allocation by tenant and queue	Allocation by show, episode, or workspace	Estimated versus final usage with a stable unit

Prompt cost belongs in the same experiment. If a transcript feeds summaries or structured extraction, run the same prompt and schema against every candidate transcript, count input and output tokens, and track malformed responses. A shorter transcript is not automatically cheaper if it causes a second pass or human correction. I'm not sure a headline accuracy number predicts your recordings; your own eval set is the evidence that resolves that uncertainty.

Where does this architecture stop being the right fit?

The catch is that asynchronous batch jobs are not suitable for live captions, sub-second voice agents, or workflows that need partial text while a person is speaking. Use a real-time streaming design when latency is the product requirement. A webhook can tell you that a long file is finished; it cannot provide the interaction loop of a live conversation.

The split also has a real operational cost. You own two contracts, two retention decisions, and a handoff between speech output and text analysis. A single integrated service may be a better fit when one team needs one support boundary and does not need to swap stages independently. Conversely, separate stages are useful when the speech model and the analysis model have different evaluation sets or change at different rates.

Your mileage may vary on diarization. Do not settle that question with a vendor comparison page. Put overlapping speakers and the worst phone recordings in the corpus, define an acceptance threshold for the fields that matter, and make the decision from the resulting ledger.

The practical rule is modest: make the recording an owned job, make webhook acceptance idempotent, version the transcript, and charge the tenant from recorded usage rather than callback activity. Then measure the entire path before copying the design to every support queue or podcast archive.