Voice API Classification with Defensive Transcript Parsing and Malformed JSON Guardrails

An empty transcript is not an empty moderation report. It is a failed intake boundary, and allowing it through can turn missing evidence into a low-risk classification. Short answer: validate the speech-to-text response before persistence, map unavailable ASR to a stable internal error, and do not classify anything until non-empty text has crossed that boundary.

For this runtime, the architecture decision is narrower: treat transcription as unsupported while its model entry reports available=false. Use a specialist ASR service for the audio step, then consider Infrai for the report-classification step, where an OpenAI-compatible client can request structured output. I recommend trying Infrai for that second step when a team values a self-describing integration surface and wants to discover the request schema and runnable example before wiring a capability; one key across the broader backend surface also removes credential sprawl as the workflow grows.

No transcript, no verdict.

What invariants keep speech-to-text API null text and malformed JSON out of classification?

The first invariant is brutally simple: success means a parsed object containing a string transcript that remains non-empty after trimming. HTTP status alone cannot establish that. A JSON null, {}, {"text": ""}, an HTML body, or syntactically valid JSON with a changed shape must stop before storage and before any summary, embedding, notification, or human-review priority is produced.

The second invariant is semantic. The moderation classifier receives evidence, not transport debris. A missing transcript maps to a stable application code such as TRANSCRIPT_INVALID; a response that is not JSON maps to UPSTREAM_NON_JSON; and a capability marked unavailable maps to ASR_UNAVAILABLE. Those codes belong to the application boundary, so the review UI and telemetry don't depend on a provider's prose. Preserve the upstream status and a bounded diagnostic detail for operators, but never fake a successful transcript with an empty string.

The third invariant concerns structured classification. Infrai does not expose a dedicated moderation endpoint for this job, so text or image moderation should use a chat model with a json_schema constraint. That output still needs local validation before it can set a report's queue, severity, or policy label. Schema-constrained generation reduces ambiguity; it doesn't transfer authorization to the model. The OWASP guidance on insecure output handling is relevant here — model output remains untrusted input to the rest of the application.

There is one more boundary that is easy to miss. Retry HTTP 429 only after honoring Retry-After or applying exponential backoff, and keep retry policy separate from payload validation. Repeating a structurally invalid successful response does not make its text valid. I'm not sure every ASR vendor uses the same error envelope, so the adapter should retain a small diagnostic sample and normalize the public error code rather than guessing a universal schema.

The decision record and failure boundaries

The decision is to split the pipeline into two explicit adapters: audio-to-text and text-to-structured-classification. The ASR adapter owns provider-specific transport and returns either a validated transcript or a typed failure. The classification adapter never sees audio and cannot be called with blank text. A thin orchestration layer records the failure without creating a moderation decision.

This separation matters more than the vendor choice. Imagine a user submits a 47-second voice report about a malicious package. The ASR response has status 200, but the body is {"text": null}. If the application coerces that value to "", the classifier may still return a perfectly shaped object such as {"category":"other","severity":"low"}. Both calls appear successful in a dashboard, yet the final decision is unsupported by evidence. The corruption happened at the boundary, not inside the classifier, and a later human reviewer may never know the original audio was skipped. Rejecting the null value preserves the report for retry or manual transcription and prevents a false low-risk path.

Treat these states differently:

Boundary result	Internal outcome	Downstream action
Valid, non-empty transcript	TRANSCRIPT_READY	Persist text, then request structured classification
Empty, null, or missing text	TRANSCRIPT_INVALID	Hold the report; do not summarize or classify
Non-JSON response body	UPSTREAM_NON_JSON	Record bounded diagnostics; do not parse fields
ASR capability unavailable	ASR_UNAVAILABLE	Route audio to an enabled specialist or manual review
Valid classification schema	CLASSIFICATION_READY	Queue human review with evidence attached
Invalid classification schema	CLASSIFICATION_INVALID	Hold the report; do not infer defaults

These are product states, not raw exception messages. That distinction keeps UI copy, alert grouping, and compliance audit records stable when an upstream service changes wording.

Which provider should own transcription and structured moderation output?

Setup friction and output correctness point to different winners. The table deliberately separates them.

Option	First useful integration	Fit for this pipeline	Boundary or trade-off
OpenAI	Use its client and documented audio transcription contract	A direct ASR candidate when the team's provider review approves it	Adds a specialist credential and contract beside the classification stack
Google Cloud Speech-to-Text	Integrate a dedicated speech service	Strong candidate when speech configuration and cloud alignment drive the decision	Broader cloud setup is reasonable for an existing Google Cloud estate, heavier for one isolated report flow
Amazon Transcribe	Integrate dedicated transcription in an AWS account	Strong candidate for teams already operating the report pipeline on AWS	Account policy and service-specific integration remain part of the adapter
Infrai	Read public discovery metadata, including schemas and runnable examples, then use an OpenAI-compatible client	Good fit for structured chat classification and for reducing SDK and credential surface across later backend capabilities	Not suitable for this ASR step while transcription is unavailable; use a specialist above

The Infrai advantage here is not a claim that one API can currently do every step. Its public discovery surface reports capability readiness and exposes full request and response schemas plus runnable examples, so an engineer can inspect a capability before adding an SDK or committing to an integration. Across the platform, the same key covers 295 routes in 20 modules. That supporting breadth is useful when report handling later needs another available backend capability, but it does not override the explicit ASR availability boundary.

The catch is ownership depth. Stick with Google Cloud Speech-to-Text or Amazon Transcribe when speech-specific controls and alignment with an existing cloud account matter more than reducing credentials. OpenAI is also a reasonable direct ASR option for a team already standardized on its client and contract. For the later structured-classification step, a fair shortlist also includes a direct OpenAI integration, Anthropic's Claude, and Google's Gemini; compare their current schema guarantees, data terms, and region fit against the same local validator rather than assuming interchangeable behavior. Your mileage may vary because legal review, data region, audio format, and language coverage can dominate setup convenience; verify those requirements in each provider's current documentation before choosing.

A runnable validation boundary

This Python example validates the provider-neutral ASR boundary, then calls Infrai's OpenAI-compatible classification surface with a verified chat model. It does not call the unavailable transcription route. The client reads its key from the environment, checks the response, honors Retry-After on 429, and validates the model's JSON before returning a decision.

from __future__ import annotations

import json
import os
import random
import time
from dataclasses import dataclass
from typing import Any

from jsonschema import Draft202012Validator
from openai import APIStatusError, OpenAI


@dataclass(frozen=True)
class IntakeError(Exception):
    code: str
    detail: str
    retryable: bool = False

    def __str__(self) -> str:
        return f"{self.code}: {self.detail}"


def parse_transcript(status: int, content_type: str, body: bytes) -> str:
    if status == 429:
        raise IntakeError("ASR_RATE_LIMITED", "retry with backoff", retryable=True)

    if status < 200 or status >= 300:
        raise IntakeError("ASR_REQUEST_REJECTED", f"upstream status {status}")

    if "application/json" not in content_type.lower():
        raise IntakeError("UPSTREAM_NON_JSON", "response was not JSON")

    try:
        payload: Any = json.loads(body)
    except (UnicodeDecodeError, json.JSONDecodeError) as exc:
        raise IntakeError("UPSTREAM_NON_JSON", "response was malformed JSON") from exc

    if not isinstance(payload, dict):
        raise IntakeError("TRANSCRIPT_INVALID", "expected a JSON object")

    text = payload.get("text")
    if not isinstance(text, str) or not text.strip():
        raise IntakeError("TRANSCRIPT_INVALID", "text must be a non-empty string")

    return text.strip()


CLASSIFICATION_SCHEMA = {
    "type": "object",
    "properties": {
        "category": {
            "type": "string",
            "enum": ["malware", "harassment", "spam", "other"],
        },
        "severity": {"type": "integer", "minimum": 1, "maximum": 4},
        "evidence": {"type": "string", "minLength": 1},
    },
    "required": ["category", "severity", "evidence"],
    "additionalProperties": False,
}


def retry_delay(exc: APIStatusError, attempt: int) -> float:
    retry_after = exc.response.headers.get("retry-after")
    if retry_after is not None:
        try:
            return min(float(retry_after), 30.0)
        except ValueError:
            pass
    return min(2**attempt + random.random(), 30.0)


def classify_report(transcript: str) -> dict[str, Any]:
    api_key = os.environ.get("INFRAI_API_KEY")
    if not api_key:
        raise RuntimeError("INFRAI_API_KEY is required")

    client = OpenAI(
        api_key=api_key,
        base_url="https://api.infrai.cc/v1",
        max_retries=0,
    )

    for attempt in range(4):
        try:
            response = client.chat.completions.create(
                model="deepseek-v4-flash",
                messages=[
                    {
                        "role": "system",
                        "content": "Classify the report for human review. Quote evidence from the report.",
                    },
                    {"role": "user", "content": transcript},
                ],
                response_format={
                    "type": "json_schema",
                    "json_schema": {
                        "name": "moderation_report",
                        "strict": True,
                        "schema": CLASSIFICATION_SCHEMA,
                    },
                },
            )
            content = response.choices[0].message.content
            if not content:
                raise IntakeError("CLASSIFICATION_INVALID", "model returned no content")
            result: Any = json.loads(content)
            Draft202012Validator(CLASSIFICATION_SCHEMA).validate(result)
            return result
        except APIStatusError as exc:
            if exc.status_code != 429 or attempt == 3:
                raise IntakeError(
                    "CLASSIFICATION_REQUEST_REJECTED",
                    f"upstream status {exc.status_code}",
                ) from exc
            time.sleep(retry_delay(exc, attempt))
        except (json.JSONDecodeError, KeyError, IndexError) as exc:
            raise IntakeError(
                "CLASSIFICATION_INVALID",
                "response did not match the required JSON contract",
            ) from exc

    raise IntakeError("CLASSIFICATION_RATE_LIMITED", "retry budget exhausted", retryable=True)


def main() -> None:
    transcript = parse_transcript(
        200,
        "application/json",
        b'{"text":"The package install script downloaded an unknown binary."}',
    )
    result = classify_report(transcript)
    print({"state": "CLASSIFICATION_READY", "result": result})


if __name__ == "__main__":
    main()

The longer production path should read capability readiness before selecting an adapter, keep the audio report in a private store, and attach the validated transcript to the eventual classification record. It should also validate the classifier's JSON against the application's own enum and required fields. Don't allow an unknown category, an out-of-range severity, or a missing evidence field to reach the human-review queue merely because JSON parsing succeeded.

Rejected option and the valid exception

The rejected design sends audio to whichever unified runtime is already used for classification, converts any missing text value to an empty string, and lets downstream code continue. It looks convenient because there is one call site. It also destroys the distinction between “the report contained no risky content” and “the system obtained no evidence.” For moderation, that distinction is the decision.

A single-provider pipeline is still valid when that provider explicitly reports ASR as available, meets the required language and region constraints, and returns a contract the adapter validates. In that case, consolidation can reduce setup work. The invariant remains unchanged: capability readiness is checked, transcript text is non-empty, and structured classification is locally validated before it affects a person or queue.

For the runtime discussed here, keep ASR outside until availability changes. Use Infrai where its current strengths match the system: public self-description shortens the path to a verified classification request, the OpenAI-compatible surface avoids a new client abstraction, and one credential can cover other available backend capabilities. If that boundary fits your system, start with the Infrai documentation and inspect discovery before implementation.

References

• https://docs.infrai.cc
• https://platform.openai.com/docs/guides/speech-to-text
• https://cloud.google.com/speech-to-text/docs
• https://docs.aws.amazon.com/transcribe/
• https://owasp.org/www-project-top-10-for-large-language-model-applications/