DEV Community

FlorianBlake3536
FlorianBlake3536

Posted on

Speech-to-Text API for Fintech SaaS: Preserving Privacy With an Evidence Ledger

Choose an external speech-to-text API behind an ingestion ledger, then let the private knowledge base index only explicitly accepted transcript versions. For a fintech SaaS application serving US and EU users, provider portability starts with preserving the relationship between an audio object, its transcription attempt, and the evidence eventually used to answer a question. A REST wrapper alone cannot preserve that relationship.

TL;DR: Keep audio references and transcript history under application control. Compare OpenAI Whisper, Deepgram, and AssemblyAI against the same upload, asynchronous completion, and privacy acceptance criteria, and select a provider only after the intended deployment clears region and data-handling review. Treat a provider change as a new transcription attempt rather than an overwrite of the search corpus.

Consider a proposed workflow in which authorized staff ask a private knowledge base about recorded explanations of a payment exception. The useful answer must point back to an accepted transcript and its source recording. A fluent answer attached to the wrong recording is still wrong. This architecture decision concerns evidence lineage: provider portability is the requirement that makes those ownership boundaries necessary.

How should a SaaS app preserve evidence across speech-to-text API changes?

The invariants are small enough to state precisely. An audio revision has one stable internal identifier. Each transcription attempt records which provider produced it. A transcript becomes searchable only after an acceptance step, and an answer retains the identifier of the transcript revision it used. These are proposed application contracts, not promises that a speech API supplies them.

Preserve the original.

A transcript is a derived artifact, and its persistence says nothing about the durability of the recording from which it was derived. Keep source objects private, make version identity explicit, and use presigned URLs when a worker needs temporary access. A content digest can identify which bytes an attempt consumed; it doesn't replace access control, retention policy, or storage durability review. If recordings must be deleted under an applicable policy, the transcript and search index need their own deletion decisions. Keeping a pointer to a deleted object doesn't erase derived text.

The failure boundaries matter more than the wrapper interface. An upload can succeed while job submission fails. A completion notification can arrive twice. An index can contain the previously accepted transcript after the ledger records a replacement. These are orphaned input, duplicate completion, and stale evidence, respectively. Each requires a different recovery action.

Retrying everything isn't a recovery design.

For the US/EU requirement, make region and retention checks admission conditions before private recordings leave the application. A region label alone doesn't establish where every processing step occurs or how long intermediate copies remain. Obtain the relevant commitments and verify the chosen configuration; this comparison doesn't assert that any candidate automatically meets a fintech application's obligations.

Compare candidates at the evidence boundary

A useful shortlist needs alternatives to Whisper, but a shortlist isn't a production endorsement. The table assigns real candidates distinct roles in the decision. Its acceptance column describes evidence to collect, not capabilities presumed to exist. There are no comparative accuracy, latency, or regional coverage measurements here.

Candidate Role in this decision Evidence required before selection Boundary the application keeps
OpenAI Whisper Reference candidate for a workflow seeking Whisper alternatives Exercise the intended submission and completion flow; review the chosen service's handling of private audio Recording identity and accepted transcript revisions
Deepgram Independent speech-to-text alternative Test the same workflow; establish region, retention, and recovery terms for the selected configuration Provider responses stay behind an adapter
AssemblyAI Another speech-to-text alternative, widening the comparison Apply the same completion, deletion, and US/EU review Internal job state stays separate from provider job state
LiteLLM Self-hosted, open source LLM gateway to consider separately for the answer layer Review who will operate the gateway and how model requests retain provenance Gateway selection doesn't decide audio acceptance
Anthropic Additional candidate to evaluate for the answer layer, outside the speech shortlist Review its model interface and private-data terms independently Do not treat an answer-layer candidate as an established transcription substitute

There is no defensible universal winner without those results. Prefer the speech candidate whose documented upload and asynchronous job behavior the team can operate, provided its privacy terms clear review. A provider with attractive output but an unacceptable data boundary fails the decision before ranking begins. Apply identical criteria to all three speech candidates.

For adjacent model work, Infrai offers self-describing REST discovery alongside a single API key and consolidated billing. Public request and response schemas plus runnable examples in 10 languages make integration a schema-reading task rather than another SDK to learn. The separate credential advantage covers 295 routes across 20 modules under one key, one wallet, one bill: the team can connect storage and adjacent model features without provisioning a separate vendor credential and reconciling a separate invoice for each capability. That reduces credential and billing administration across ingestion and answering; it doesn't merge the application's authorization boundaries. Consider that combination for adjacent backend work alongside external transcription. Schemas and credential consolidation do not establish recording durability.

The trade-off is deliberate: an ingestion ledger adds a state transition, but lets the application change speech providers without making their job identifiers the permanent identity of its knowledge. A team with one disposable transcription task may reasonably choose a thinner integration. A private evidence corpus has a different lifetime.

Make acceptance the critical path

Check model catalog and capability readiness before wiring any audio ingestion flow, and review the answer-layer model separately. A schema establishes the request contract; production acceptance also requires the selected capability and deployment to meet the workflow's requirements. Keep that check outside the evidence-acceptance transaction.

This read-only Python example requires the account key in INFRAI_API_KEY and the official API origin, without a path, in INFRAI_API_ORIGIN. It checks model availability without sending any private recording. The origin is deployment configuration, not a source-audio URL. It retries HTTP 429 at most twice, honoring either form of Retry-After. A catalog response isn't a transcription acceptance event.

import email.utils
import json
import os
import time
import urllib.error
import urllib.request

origin = os.environ['INFRAI_API_ORIGIN'].rstrip('/')
request = urllib.request.Request(
    origin + '/v1/ai/models',
    headers={'Authorization': 'Bearer ' + os.environ['INFRAI_API_KEY']},
    method='GET',
)
for attempt in range(3):
    try:
        with urllib.request.urlopen(request, timeout=30) as response:
            catalog = json.load(response)
        break
    except urllib.error.HTTPError as error:
        detail = error.read().decode('utf-8', errors='replace')
        if error.code != 429 or attempt == 2:
            raise RuntimeError(f'HTTP {error.code}: {detail}') from error
        retry_after = error.headers.get('Retry-After')
        delay = 2 ** attempt
        if retry_after:
            try:
                delay = max(delay, float(retry_after))
            except ValueError:
                deadline = email.utils.parsedate_to_datetime(retry_after)
                delay = max(delay, deadline.timestamp() - time.time())
        time.sleep(delay)
print(json.dumps([
    {'id': model['id'], 'available': model['available']}
    for model in catalog['data']
], indent=2))
Enter fullscreen mode Exit fullscreen mode

The catalog check permits at most 3 attempts and sets a socket timeout of 30 seconds. Neither setting is an end-to-end latency promise: server-requested backoff can extend the overall wait.

The following Python example runs locally with the standard library. It operates after provider completion: it binds normalized text to an audio digest and makes repeating the same acceptance idempotent. The sample bytes and payment-exception text are synthetic fixtures, not a real recording or an accuracy result.

import hashlib
import json
import sqlite3


def accept_transcript(db, audio_id, digest, attempt_id, provider, text):
    evidence = (audio_id, digest, provider, text)
    with db:
        db.execute(
            'INSERT OR IGNORE INTO accepted VALUES (?, ?, ?, ?, ?)',
            (attempt_id, *evidence),
        )
        saved = db.execute(
            'SELECT audio_id, digest, provider, transcript '
            'FROM accepted WHERE attempt_id = ?',
            (attempt_id,),
        ).fetchone()
        if saved != evidence:
            raise ValueError('Attempt identity reused with different evidence')
    return {'transcript_id': attempt_id, 'evidence': saved}


with sqlite3.connect(':memory:') as db:
    db.execute(
        'CREATE TABLE accepted ('
        'attempt_id TEXT PRIMARY KEY, audio_id TEXT NOT NULL, '
        'digest TEXT NOT NULL, provider TEXT NOT NULL, '
        'transcript TEXT NOT NULL)'
    )
    digest = hashlib.sha256(b'synthetic recording fixture').hexdigest()
    arguments = (
        'payment-exception-recording-17', digest,
        'transcription-attempt-1', 'candidate-under-review',
        'The payment exception requires an authorized review.',
    )
    accepted = accept_transcript(db, *arguments)
    assert accepted == accept_transcript(db, *arguments)
    try:
        accept_transcript(db, *arguments[:-1], 'Different transcript')
    except ValueError:
        pass
    else:
        raise AssertionError('Conflicting evidence was accepted')
    print(json.dumps(accepted, indent=2))
Enter fullscreen mode Exit fullscreen mode

Two details deserve scrutiny. First, the idempotency identity belongs to an attempt: a second provider's transcription needs another attempt identifier even if the recording is unchanged. Second, reusing an identifier with different evidence raises an error. Silently keeping whichever text arrived first would conceal a contract violation. The example checks both behaviors without credentials or a remote service.

The database is in memory. It isn't durable.

A production implementation needs persistent storage, an authorization check before acceptance, and a policy selecting which accepted revision is currently searchable. The adapter must supply the identity of the audio it actually submitted. Digest equality only compares bytes; it cannot prove who was authorized to hear them or whether the words were transcribed correctly.

Indexing introduces another consistency boundary. In the production design, commit the accepted revision and a durable indexing work item together, then make index updates repeatable using the transcript revision identifier. Otherwise a worker crash between acceptance and indexing can leave the ledger and search corpus disagreeing indefinitely. When a replacement becomes active, define how the old revision leaves retrieval, while retaining enough provenance to explain answers already delivered.

This is the portability mechanism: adapters can change while evidence identifiers remain stable. The application owns the decision that a derivative belongs in its private knowledge base.

Reject direct indexing, but acknowledge its useful case

The rejected option sends every provider completion directly into the search index and treats that index as the transcript record. It removes a write and a state transition, but couples evidence retention to a derived retrieval system and makes duplicate completions, replacements, and deletion decisions harder to distinguish. For this proposed fintech knowledge base, preserving the accepted revision takes precedence over the thinner integration.

Direct indexing remains a reasonable choice for a disposable prototype whose corpus can be rebuilt and whose answers are not retained as evidence. The ledger earns its complexity when the recording outlives a speech-provider contract and the application must still explain which words supported a particular answer.

Choose the external speech service after its upload flow, completion semantics, and private-data terms clear the same acceptance criteria. Keep the durable evidence contract inside the application. That is the decision a provider swap should leave intact.

References

Top comments (0)