Your Speech-to-Text API Is Only as Good as the Audio You Send It

A speech-to-text API can perform well on clean benchmark recordings and still struggle with your users’ actual audio.

That gap is not always caused by the model.

It can come from:

• Background noise
• Incorrect sample rates
• Unsupported audio formats
• Accents and code-switching
• Domain-specific vocabulary
• Choosing batch transcription when the product needs streaming
• Reading the wrong field from the API response

The fastest way to evaluate speech recognition is not to upload one perfect recording. It is to test the API against the audio your application will encounter in production.

This guide shows how to integrate the Pulse speech-to-text API from Python and Node.js, process pre-recorded audio, build a real-time WebSocket client, and avoid the response-handling mistakes that often break otherwise successful integrations.

Pulse supports two transcription modes

Pulse is the speech-recognition model from Smallest AI.

It supports two distinct integration patterns.

Pre-recorded transcription

Use pre-recorded transcription when the entire audio file already exists.

Common examples include:

• Call recordings
• Voicemails
• Podcast episodes
• Uploaded videos
• Meeting archives
• Offline transcription jobs

The application sends the complete file through an HTTPS request and receives one structured response.

Audio file
    ↓
HTTPS request
    ↓
Complete transcript
    ↓
Storage or downstream processing

Real-time transcription

Use real-time transcription when audio is still being captured.

Common examples include:

• Voice agents
• Live captions
• Phone calls
• Browser microphones
• Meeting assistants
• Real-time conversation analytics

The application keeps a WebSocket connection open, sends audio in small binary chunks, and receives partial and final transcript events while the speaker is still talking.

Live audio
    ↓
WebSocket audio chunks
    ↓
Partial transcript events
    ↓
Final transcript events
    ↓
Application action

The choice is not primarily about which transport looks more advanced.

Use batch transcription when the product can wait. Use streaming when the product must react while the audio is still arriving.

Create and store the API key

Keep the API key in an environment variable rather than hard-coding it into the application.

Before running the snippet, create a Smallest.ai API key in the dashboard and store it in the SMALLEST_API_KEY environment variable.

export SMALLEST_API_KEY="your-api-key-here"

Every authenticated request sends the value through the Authorization header:

Authorization: Bearer SMALLEST_API_KEY

Keep the key on your server. Do not expose it in browser JavaScript, mobile application code, public repositories, screenshots, or client-side logs.

Transcribe a pre-recorded file with Python

The current pre-recorded endpoint is:

POST https://api.smallest.ai/waves/v1/stt/

The model, language, and optional features are supplied as query parameters. The audio file is sent as raw bytes in the request body.

Before running the snippet, create a Smallest.ai API key in the dashboard and store it in the SMALLEST_API_KEY environment variable.

import os
from pathlib import Path
from typing import Any

import requests


STT_ENDPOINT = "https://api.smallest.ai/waves/v1/stt/"


def transcribe_file(
    file_path: str,
    language: str = "en",
) -> dict[str, Any]:
    audio_path = Path(file_path)

    if not audio_path.is_file():
        raise FileNotFoundError(f"Audio file not found: {audio_path}")

    with audio_path.open("rb") as audio_file:
        response = requests.post(
            STT_ENDPOINT,
            params={
                "model": "pulse",
                "language": language,
                "word_timestamps": "true",
            },
            headers={
                "Authorization": (
                    f"Bearer {os.environ['SMALLEST_API_KEY']}"
                ),
                "Content-Type": "application/octet-stream",
            },
            data=audio_file,
            timeout=120,
        )

    response.raise_for_status()
    return response.json()


result = transcribe_file("recording.wav")

print(result["transcription"])
print(result.get("words", [])[:5])

One response-handling detail matters immediately:

The pre-recorded response uses transcription, not transcript.

A simplified response can look like this:

{
  "status": "success",
  "transcription": "Hello, this is a test transcription.",
  "words": [
    {
      "start": 0.48,
      "end": 1.12,
      "word": "Hello,"
    },
    {
      "start": 1.12,
      "end": 1.28,
      "word": "this"
    }
  ],
  "utterances": [
    {
      "start": 0.48,
      "end": 3.76,
      "text": "Hello, this is a test transcription."
    }
  ]
}

The returned structures support different downstream workflows:

Field	Useful for
transcription	Complete readable transcript
words	Captions, subtitle alignment and audio navigation
utterances	Sentence-level segments and readable conversation records
language	Language-aware routing and transcript metadata
Speaker fields	Multi-speaker calls and meetings

Do not assume every optional field will always be present. Use .get() or validate the response against a schema before sending it downstream.

Transcribe audio from a URL

Pulse can also process an audio file hosted at a publicly accessible URL.

This is useful when recordings already live in object storage and downloading them to the application server would add unnecessary work.

Before running the snippet, create a Smallest.ai API key in the dashboard and store it in the SMALLEST_API_KEY environment variable.

import os
from typing import Any

import requests


STT_ENDPOINT = "https://api.smallest.ai/waves/v1/stt/"


def transcribe_from_url(
    audio_url: str,
    language: str = "en",
) -> dict[str, Any]:
    response = requests.post(
        STT_ENDPOINT,
        params={
            "model": "pulse",
            "language": language,
            "word_timestamps": "true",
        },
        headers={
            "Authorization": (
                f"Bearer {os.environ['SMALLEST_API_KEY']}"
            ),
            "Content-Type": "application/json",
        },
        json={"url": audio_url},
        timeout=120,
    )

    response.raise_for_status()
    return response.json()


result = transcribe_from_url(
    "https://example.com/recordings/customer-call.wav"
)

print(result["transcription"])

The remote file must be accessible to the API. A URL that requires browser cookies, local authentication, or access to a private network will not work without a suitable signed URL or public access policy.

Make the same request from Node.js

Modern Node.js versions provide fetch without requiring node-fetch.

Before running the snippet, create a Smallest.ai API key in the dashboard and store it in the SMALLEST_API_KEY environment variable.

import { readFile } from "node:fs/promises";

const STT_ENDPOINT = "https://api.smallest.ai/waves/v1/stt/";

async function transcribeFile(filePath, language = "en") {
  const audio = await readFile(filePath);

  const params = new URLSearchParams({
    model: "pulse",
    language,
    word_timestamps: "true",
  });

  const response = await fetch(`${STT_ENDPOINT}?${params}`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.SMALLEST_API_KEY}`,
      "Content-Type": "application/octet-stream",
    },
    body: audio,
  });

  if (!response.ok) {
    const errorBody = await response.text();

    throw new Error(
      `Transcription failed with ${response.status}: ${errorBody}`,
    );
  }

  return response.json();
}

const result = await transcribeFile("recording.wav");

console.log(result.transcription);
console.log(result.words?.slice(0, 5) ?? []);

For a remotely hosted file, change the request body and content type.

Before running the snippet, create a Smallest.ai API key in the dashboard and store it in the SMALLEST_API_KEY environment variable.

const STT_ENDPOINT = "https://api.smallest.ai/waves/v1/stt/";

async function transcribeFromUrl(audioUrl, language = "en") {
  const params = new URLSearchParams({
    model: "pulse",
    language,
    word_timestamps: "true",
  });

  const response = await fetch(`${STT_ENDPOINT}?${params}`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.SMALLEST_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      url: audioUrl,
    }),
  });

  if (!response.ok) {
    const errorBody = await response.text();

    throw new Error(
      `Transcription failed with ${response.status}: ${errorBody}`,
    );
  }

  return response.json();
}

const result = await transcribeFromUrl(
  "https://example.com/recordings/customer-call.wav",
);

console.log(result.transcription);

Enable only the enrichment features you need

A production transcript often needs more than plain text.

Pulse supports optional features such as:

• Word timestamps
• Sentence-level utterances
• Speaker diarization
• Emotion detection
• Gender detection
• Language detection
• Keyword boosting
• PII and PCI redaction
• Inverse text normalization

These capabilities are enabled through query parameters.

Before running the snippet, create a Smallest.ai API key in the dashboard and store it in the SMALLEST_API_KEY environment variable.

import os

import requests


with open("call_recording.wav", "rb") as audio_file:
    response = requests.post(
        "https://api.smallest.ai/waves/v1/stt/",
        params={
            "model": "pulse",
            "language": "en",
            "word_timestamps": "true",
            "diarize": "true",
            "emotion_detection": "true",
            "gender_detection": "true",
            "itn_normalize": "true",
        },
        headers={
            "Authorization": (
                f"Bearer {os.environ['SMALLEST_API_KEY']}"
            ),
            "Content-Type": "application/octet-stream",
        },
        data=audio_file,
        timeout=120,
    )

response.raise_for_status()

result = response.json()

print(result["transcription"])

Enable features because the workflow requires them, not simply because the parameters exist.

Word timestamps

Useful for:

• Captions
• Subtitle tracks
• Transcript review
• Search
• Audio playback navigation

Timing information is difficult to reconstruct accurately after transcription. Enable it early when the product may eventually need synchronized text.

Speaker diarization

Useful for:

• Customer calls
• Interviews
• Meetings
• Podcasts
• Sales conversations

Diarization converts a wall of text into speaker-attributed turns.

Emotion detection

Useful for enriching offline call analysis and quality-assurance workflows.

Treat emotional scores as model-generated signals rather than definitive statements about a person.

Inverse text normalization

Useful when spoken expressions should appear in conventional written form.

For example:

"twenty five dollars"

can become:

"$25"

This matters for dates, currencies, phone numbers, amounts, and other entities that downstream software must parse.

Accuracy begins with representative audio

A published benchmark is not a substitute for testing your own workload.

Build an evaluation set that includes the conditions your users will create:

• Clean studio recordings
• Browser microphones
• Mobile devices
• Telephony audio
• Background conversations
• Fast speech
• Hesitations
• Multiple speakers
• Regional accents
• Domain-specific vocabulary
• Mixed-language conversations

Track more than overall Word Error Rate.

A transcript can be mostly correct and still fail the application by misrecognizing the exact information that matters.

Evaluate:

Area	What to inspect
General transcription	Missing, substituted or invented words
Entity accuracy	Names, dates, amounts, addresses and identifiers
Domain vocabulary	Product names, abbreviations and technical terms
Speaker separation	Whether speaker labels remain consistent
Timestamp quality	Alignment and drift
Language handling	Accents, language switching and detection
Failure behavior	Empty responses, timeouts and invalid files
Downstream usefulness	Whether the next system can reliably use the result

For domain terms, product names, and uncommon vocabulary, keyword boosting may be more valuable than repeatedly switching providers.

Audio format is part of the model input

Speech recognition cannot recover information that the audio pipeline has already damaged.

Before blaming the API, inspect:

• Sample rate
• Channel count
• Codec
• Container format
• Clipping
• Background noise
• Compression artifacts
• Volume
• Packet loss

For speech workloads, mono audio at an appropriate speech-oriented sample rate is usually more useful than sending oversized stereo files with no additional spoken information.

Real-time streaming also requires the client to describe the incoming encoding correctly. Sending μ-law audio while declaring linear16, for example, will produce unusable results even though the WebSocket connection itself succeeds.

Language selection should be deliberate

When you know the language, send its explicit language code.

For example:

en
hi
de
fr
es

When the language is unknown or the conversation may switch between related languages, use the appropriate regional auto-detection option supported by the chosen transcription mode.

Do not assume that batch and streaming expose identical language sets or regional routes. Check the current model documentation before shipping a multilingual workflow.

Explicit language selection is usually the safer choice when the source language is known.

Real-time transcription changes the application

Batch transcription produces one result after the file is complete.

Streaming produces a sequence of changing hypotheses.

The current WebSocket endpoint is:

wss://api.smallest.ai/waves/v1/stt/live?model=pulse

A client must:

• Open and authenticate the connection
• Send correctly encoded binary audio
• Receive events concurrently
• Replace partial hypotheses
• Commit final segments
• Handle connection failures
• Close the stream deliberately

Before running the snippet, create a Smallest.ai API key in the dashboard and store it in the SMALLEST_API_KEY environment variable.

import WebSocket from "ws";

const params = new URLSearchParams({
  model: "pulse",
  language: "en",
  encoding: "linear16",
  sample_rate: "16000",
  word_timestamps: "true",
});

const websocketUrl =
  `wss://api.smallest.ai/waves/v1/stt/live?${params}`;

const ws = new WebSocket(websocketUrl, {
  headers: {
    Authorization: `Bearer ${process.env.SMALLEST_API_KEY}`,
  },
});

ws.on("open", () => {
  console.log("Connected to Pulse STT");
});

ws.on("message", (rawMessage) => {
  const event = JSON.parse(rawMessage.toString());

  if (event.is_final) {
    console.log("\n[FINAL]", event.transcript);
  } else {
    process.stdout.write(`\r[PARTIAL] ${event.transcript ?? ""}`);
  }

  if (event.is_last) {
    console.log("\nStream complete");
  }
});

ws.on("error", (error) => {
  console.error("WebSocket error:", error);
});

function sendAudioChunk(audioBuffer) {
  if (ws.readyState !== WebSocket.OPEN) {
    throw new Error("WebSocket is not open");
  }

  ws.send(audioBuffer);
}

function closeStream() {
  if (ws.readyState === WebSocket.OPEN) {
    ws.send(
      JSON.stringify({
        type: "close_stream",
      }),
    );
  }
}

Send audio as binary frames. A commonly recommended chunk size is 4096 bytes, although the correct capture and buffering strategy still depends on the source encoding and application.

When the audio is complete, send:

{
  "type": "close_stream"
}

This asks the service to flush the remaining buffered audio and return the last event with is_last: true.

Stream microphone audio from Python

A real-time microphone client needs separate send and receive loops. Otherwise, waiting for one operation can prevent the other from progressing.

Before running the snippet, create a Smallest.ai API key in the dashboard and store it in the SMALLEST_API_KEY environment variable.

import asyncio
import json
import os
from urllib.parse import urlencode

import pyaudio
import websockets


SAMPLE_RATE = 16_000
CHUNK_SIZE = 4_096

params = {
    "model": "pulse",
    "language": "en",
    "encoding": "linear16",
    "sample_rate": str(SAMPLE_RATE),
    "word_timestamps": "true",
}

websocket_url = (
    "wss://api.smallest.ai/waves/v1/stt/live?"
    + urlencode(params)
)

headers = {
    "Authorization": f"Bearer {os.environ['SMALLEST_API_KEY']}",
}


async def stream_microphone() -> None:
    audio = pyaudio.PyAudio()

    microphone = audio.open(
        format=pyaudio.paInt16,
        channels=1,
        rate=SAMPLE_RATE,
        input=True,
        frames_per_buffer=CHUNK_SIZE,
    )

    async with websockets.connect(
        websocket_url,
        additional_headers=headers,
    ) as websocket:
        print("Listening. Press Ctrl+C to stop.")

        async def send_audio() -> None:
            try:
                while True:
                    chunk = microphone.read(
                        CHUNK_SIZE,
                        exception_on_overflow=False,
                    )
                    await websocket.send(chunk)
                    await asyncio.sleep(0.01)
            except asyncio.CancelledError:
                await websocket.send(
                    json.dumps(
                        {
                            "type": "close_stream",
                        }
                    )
                )
                raise

        async def receive_events() -> None:
            async for message in websocket:
                event = json.loads(message)
                transcript = event.get("transcript", "")

                if event.get("is_final"):
                    print(f"\n[FINAL] {transcript}")
                else:
                    print(
                        f"\r[PARTIAL] {transcript}",
                        end="",
                        flush=True,
                    )

                if event.get("is_last"):
                    return

        sender = asyncio.create_task(send_audio())

        try:
            await receive_events()
        finally:
            sender.cancel()
            await asyncio.gather(
                sender,
                return_exceptions=True,
            )

            microphone.stop_stream()
            microphone.close()
            audio.terminate()


asyncio.run(stream_microphone())

Production code should also handle:

• Operating-system audio permissions
• Device disconnection
• Queue limits
• Slow network writes
• Reconnection
• Session timeouts
• Duplicate segments
• Graceful shutdown

Partial transcripts are not permanent text

A streaming recognizer may emit:

I need to update

Then revise it to:

I need to update my address

Then finalize:

I need to update my billing address.

Do not append every partial result to the transcript.

Maintain two states:

Stable final segments
Current replaceable partial segment

Only persist or act on text after the API marks the segment final, unless the product explicitly accepts the risks of acting on provisional text.

The source article’s related guide on handling reconnects, dropouts and duplicated streaming transcripts covers the next layer of production complexity.

Batch and streaming return different structures

This distinction frequently causes silent bugs.

Pre-recorded response

Expect fields such as:

transcription
words
utterances
language

Real-time response

Expect a sequence of events containing fields such as:

transcript
is_final
is_last
session_id

Notice the difference:

Batch:     transcription
Streaming: transcript

Do not route both modes through one untested parser and assume their payloads are interchangeable.

A useful application-level interface can normalize both into your own internal format:

text
is_final
start_time
end_time
speaker
language
source_mode

That keeps provider-specific response details at the integration boundary.

Choose the mode based on the product’s clock

Use pre-recorded transcription when:

• The full file already exists.
• The workflow can wait for one complete response.
• Simpler HTTP request-response code is preferable.
• You are processing archived audio.
• You are running offline batch jobs.

Use real-time transcription when:

• Partial words must appear while the user speaks.
• A voice agent must begin reasoning before the turn is complete.
• Live captions must update continuously.
• Turn detection drives application logic.
• You are processing a live phone call or microphone.
• User experience depends on low first-transcript latency.

A WebSocket is not automatically a better architecture.

It is justified only when waiting for the complete recording would prevent the product from meeting its user-facing requirement.

Production readiness is more than a successful transcript

Before launch, test the complete pipeline.

Input

• Are audio formats validated?
• Is the declared encoding accurate?
• Are sample rates and channels appropriate?
• Are unsupported or corrupted files rejected clearly?

Authentication

• Is the API key server-side?
• Are secrets excluded from logs?
• Can keys be rotated safely?

Response handling

• Does the application distinguish transcription from transcript?
• Are optional fields handled safely?
• Are partial results replaced instead of appended?
• Are final events persisted exactly once?

Reliability

• Are timeouts configured?
• Are transient failures retried?
• Can retries create duplicate processing?
• Can the WebSocket reconnect safely?
• Are queues bounded?

Evaluation

• Does the test set represent real users?
• Are names, numbers and domain terms measured separately?
• Are latency percentiles tracked?
• Are failures visible through logs and metrics?

Start with one representative recording

The first useful integration does not need a microphone, animated waveform, or persistent WebSocket.

Start with:

One representative audio file
        ↓
One authenticated HTTP request
        ↓
One structured transcript
        ↓
Evaluation

Then inspect:

• Accuracy
• Entity preservation
• Speaker labels
• Timestamps
• Language handling
• Response structure
• Failure behavior

Move to real-time streaming only when the application genuinely needs results before the audio ends.

That progression keeps the integration understandable and gives every layer of complexity a clear reason to exist.

Build with Pulse STT

To test the workflow:

1. Create a Smallest AI API key.
2. Open the Pulse speech-to-text product page.
3. Start with one recording captured under realistic conditions.
4. Validate the transcript and response fields.
5. Add timestamps or diarization only when the workflow needs them.
6. Move to WebSocket streaming when the product requires live results.

You can also explore the original Smallest AI integration guide and the Smallest AI Cookbook for additional examples.

What caused the hardest speech-to-text failure in your application: noisy audio, response handling, language coverage, or streaming state?