DEV Community: api

Your Pipeline Is 27.3h Behind: Catching World Sentiment Leads with Pulsebit

Pulsebit News Sentiment API — Wed, 22 Jul 2026 17:02:25 +0000

Your Pipeline Is 27.3h Behind: Catching World Sentiment Leads with Pulsebit

We recently uncovered a striking anomaly: a 24h momentum spike at +0.210. This spike reveals something significant in the landscape of sentiment surrounding mental health, as evidenced by the leading English press coverage leading by 27.3 hours. Specifically, the article titled "World Brain Day: Mental health is as important as physical health, says Ballari" is pivotal in this frame, clustered around themes like brain, health, and world. If you’re not adjusting your models to account for these shifts, you’re missing critical insights.

The problem here is straightforward: your pipeline may be lagging significantly. With a leading language of English and the dominant entity being 'brain', your model missed this sentiment spike by 27.3 hours. This gap highlights a structural issue in handling multilingual origins and entity dominance. If you’re not capturing these signals promptly, you risk making decisions based on outdated data.

English coverage led by 27.3 hours. Af at T+27.3h. Confidence scores: English 0.85, Spanish 0.85, Id 0.85 Source: Pulsebit /sentiment_by_lang.

Let’s dive into the code that can help catch these anomalies effectively. We’ll first set up a geographic origin filter to query only English language sources. Here’s how to do that:

import requests

![Left: Python GET /news_semantic call for 'world'. Right: ret](https://pub-c3309ec893c24fb9ae292f229e1688a6.r2.dev/figures/g3_code_output_split_1784739743929.png)
*Left: Python GET /news_semantic call for 'world'. Right: returned JSON response structure (clusters: 3). Source: Pulsebit /news_semantic.*


url = "https://api.pulsebit.com/articles"
params = {
    "topic": "world",
    "lang": "en",
    "score": +0.062,
    "confidence": 0.85,
    "momentum": +0.210
}

response = requests.get(url, params=params)
data = response.json()

Now that we have the relevant articles, the next step is to run the cluster reason string back through our sentiment analysis endpoint. This allows us to score the narrative framing itself, providing deeper insights. Here’s how to implement this:

sentiment_url = "https://api.pulsebit.com/sentiment"
payload = {
    "text": "Clustered by shared themes: brain, health, world, day:, mental."
}

sentiment_response = requests.post(sentiment_url, json=payload)
sentiment_data = sentiment_response.json()

With this setup, you’re not just capturing data; you’re gaining a comprehensive view of the narrative that’s shaping public sentiment around critical issues.

Now, let's discuss three specific builds you can implement with this pattern. First, use the geographic filter to identify spikes in sentiment around discussions of 'world' with a threshold of +0.210. This ensures you’re catching relevant discussions early. Second, leverage the meta-sentiment loop to analyze narratives around 'brain' and 'health' with a focus on emotional framing; consider a threshold of sentiment score +0.062 to prioritize content that resonates deeply. Finally, track forming themes like 'world', 'cup', and 'after' as these can indicate emerging conversations that often precede larger trends in mental health discussions.

Geographic detection output for world. India leads with 46 articles and sentiment +0.33. Source: Pulsebit /news_recent geographic fields.

Curious to get started? Head over to pulsebit.lojenterprise.com/docs. You can copy-paste the code snippets above and run them in under 10 minutes. Let’s ensure our pipelines are responsive and in tune with the world’s sentiment—don’t let valuable insights slip through the cracks.

Building an Agent-Native Data Layer over MCP

Arman Obosyan — Wed, 22 Jul 2026 16:58:09 +0000

Agents are good at reasoning and bad at remembering how to talk to every data vendor. The usual pattern is a growing pile of one-off integrations: one client for markets, another for macro series, another for sanctions lists, another for weather. Each has its own auth, schema, rate limit, and failure mode. The agent either hardcodes a few endpoints or invents numbers when a call fails.

This post is about a different shape: one MCP server in front of one API that already normalizes a large catalog of world data. The server is Sugra MCP (pip install sugra-api-mcp). Behind it sits the Sugra API - 1,500+ endpoints across 36 domains, backed by 160+ primary sources (sovereign statistics agencies, central banks, intergovernmental bodies, academic sources, and a smaller commercial slice under Sugra-branded wrappers).

The problem agents actually hit

Tool-using agents do not need "more models." They need reliable access to facts with provenance. In practice teams rebuild the same glue:

Auth headers and key rotation per vendor
Divergent JSON shapes and pagination
Silent empty responses that the model fills with guesses
No single place to discover what is even available

MCP fixes the client-side contract (tools, prompts, resources over one protocol). It does not by itself fix the data plane. You still need a stable catalog, a consistent call shape, and errors the agent can act on.

What MCP solves - and what it does not

MCP gives you:

A standard way for Claude, ChatGPT, Gemini, xAI, Cursor, VS Code, and other clients to list and call tools
Hosted HTTP and local stdio transports
Room for prompts and resources next to tools

MCP does not give you:

A curated multi-domain data catalog
Source attribution and freshness on every number
One rate limit and one API key across domains

Those belong in the data layer behind the server. Sugra MCP is that layer's agent face, not a replacement for the protocol.

Bundled catalog instead of live schema scraping

Discovery should not cost quota and should not depend on a network hop to the API for every search or describe.

The package ships a bundled endpoint catalog. Search and describe run locally against that catalog. Only real data fetches hit sugra.ai. That keeps agent planning loops cheap and makes the server useful offline for "what exists?" questions.

When the API grows, the catalog is regenerated and released with the package (and mirrored on the hosted endpoint after deploy). Agents get a versioned view of the surface instead of scraping OpenAPI on every session.

The search / describe / call pattern

The tool surface is intentionally small:

Search - natural language or keywords over the catalog
Describe - parameters, domains, and source notes for an endpoint
Call / fetch - execute with structured arguments

On the hosted endpoint the surface is wider (11 tools vs 8 local), including composed helpers for common agent paths. The idea is the same: the agent finds the right operation, inspects it, then calls it - instead of you hardcoding paths into the system prompt.

Six workflow prompts (market snapshot, macro briefing, sanctions screening, sector compare, earth/conditions-style flows, source overview) package multi-step patterns for clients that surface MCP prompts. Three read-only catalog resources expose discovery material without burning data quota.

Structured errors instead of exceptions the model swallows

When an endpoint is missing, a parameter is wrong, or a source is temporarily empty, the tool returns a structured error the agent can branch on: retry, pick another series, or tell the user the data is unavailable.

That is more valuable than "covering every source on day one." An honest failure with a machine-readable reason beats a fluent wrong number.

Attribution and freshness as first-class fields

Every successful data result carries:

Source attribution - where the number came from (sovereign and intergovernmental names are shown as-is; commercial upstreams appear under Sugra product wrappers on public surfaces)
Freshness - how current the observation is

Agents can cite sources in the user-facing answer. Humans can audit a chain of tool calls. This is the difference between a demo scraper and something you can put near a research or compliance workflow.

Hosted vs local: one contract

Mode	How	Tools	Best for
Local / self-hosted	`pip install sugra-api-mcp` over stdio or your HTTP	8 tools + prompts/resources	IDEs, private runners, offlin
Hosted	app.sugra.ai/mcp (Streamable HT

Same API key model (SUGRA_API_KEY / account key). Free tier is 50 requests/day so you can evaluate without a card. Paid tiers scale volume only - the full endpoint surface is on every plan.

The server is also listed in the official MCP Registry as ai.sugra/api-mcp and in the OpenAI Plugins Directory for ChatGPT and Codex. On Smithery: sugra-systems/sugra-api.

Proof (live)

Claim	Check
1,500+ endpoints / 36 domains / 160+ sources	sugra.ai/stats
MCP package	PyPI · GitHub
Official MCP Registry	`ai.sugra/api-mcp`
Hosted MCP	app.sugra.ai/mcp
OpenAI Plugins	listing
Smithery	@sugra-systems/sugra-api
Free tier	50 req/day - register

MIT license. Every tool result carries source attribution and freshness - not a black box.

What is next

Keep the catalog honest as domains grow
Dual-host OAuth hardening for mcp.sugra.ai alongside app.sugra.ai/mcp
Tighter agent evals for search ranking and entity resolution
More runnable recipes in the public cookbook

If you build research, compliance, or multi-domain agent workflows and you are tired of re-wiring vendors:


bash
pip install sugra-api-mcp

• Repo: github.com/Sugra-Systems/sugra-api-mcp (https://github.com/Sugra-Systems/sugra-api-mcp?utm_source=devto&utm_medium=article&utm_campaign=mcp_launch_202607)
• API and free key: sugra.ai (https://sugra.ai?utm_source=devto&utm_medium=article&utm_campaign=mcp_launch_202607)
• Hosted MCP: app.sugra.ai/mcp (https://app.sugra.ai/mcp?utm_source=devto&utm_medium=article&utm_campaign=mcp_launch_202607)

Feedback on the tool surface - especially from people wiring external data into coding agents - is welcome.

Why Your CRM Keeps Failing You (And What I Learned Running Sales Solo for Three Years)

Nano brain — Wed, 22 Jul 2026 15:30:17 +0000

I spent three years running growth for an early-stage company where I was, on any given week, the marketer, the salesperson, and the person answering support tickets at 11pm. I tried five different CRMs in that time. Every single one of them was built on the same wrong assumption: that I had a team.
I did not. And if you are reading this, there is a good chance you do not either.
This is not a theoretical argument about tools. It is what I learned the hard way, and what I now understand backed by how these systems are actually built.
The Experience: What It Actually Feels Like to Run Growth Alone
Here is a Tuesday I remember clearly. A prospect replied to a cold email I had sent three weeks earlier. I could not remember which campaign it was from. I opened my CRM: no note. I opened my email: found the thread, but no context on why I had reached out. I checked my content calendar, because I vaguely recalled they had liked a LinkedIn post. Fifteen minutes later I had pieced together enough to reply like I knew what I was talking about.
That fifteen minutes happened multiple times a week, for every lead who took more than one touch to convert. Which, if you have ever sold anything, is most of them.
The tools were not broken individually. The CRM did what CRMs do. The email client did what email clients do. But none of them knew about each other, so the burden of connecting the dots fell entirely on me, every single time. I was the integration layer. That is an exhausting place to live.
The Expertise: Why This Keeps Happening
Once I started paying attention to how CRM software is actually designed, the pattern became obvious. Most systems in this category are modeled on a "funnel and handoff" structure: marketing generates a lead, hands it to sales, sales closes it and hands it to success, success owns the renewal. Each stage gets its own tool because, historically, each stage had its own team and its own budget line.
That model works fine when three different people are actually holding those three roles. It breaks down completely when one person is holding all three, because the handoffs that used to happen between people now have to happen inside a single person's head, across tools that were never designed to talk to each other.
This is not a minor UX complaint. It is a structural mismatch between how the software is organized and how small teams actually operate. You can layer on integrations and Zapier workflows to patch the gaps, and I tried that too, but patched gaps are still gaps. Something always falls through, usually the thing you needed most: context on why this person matters and what happened last time.
The Authority: What the Better Systems Get Right
The fix, when I finally found something that worked, was not a smarter individual tool. It was a change in the underlying unit of the system. Instead of the record being a "lead" that changes shape and location as it moves through stages, the record is the person, and everything, the first LinkedIn comment, the cold email reply, the closed deal, the renewal conversation, lives on that same profile permanently.
This is the model behind Distribution Engine, built by Nanobrains, an engagement CRM designed specifically for the founder or small team doing marketing, sales, and retention without a department for each. A universal profile holds the whole relationship in one place: identity, channel history, notes from every stage, and a live read on how warm or cold the relationship currently is. Discovery, content publishing, messaging across email, LinkedIn, and WhatsApp, and re-engagement all write to and read from that same record.
The busywork that used to eat my time, finding the right people to reach out to, turning one idea into platform-specific posts, tracking who has gone quiet, runs through agents that handle the repetitive parts and surface what needs a human decision. I still make every real call. I am just no longer the one manually stitching six tools together to figure out what is even happening.
The Trust: What I'd Tell You to Actually Check
I am not going to pretend one tool fixes every problem in solo sales and marketing, because it does not. Here is what I'd actually verify before you commit to anything, this system included:
Does the tool keep one record per person across every stage, or does it still split "leads" from "contacts" from "customers" the moment they convert? That split is where context dies.
Can you see, at a glance, when you last touched a relationship and how long it has been? Warm leads go cold quietly, and the tools that surface this before you notice on your own are doing something genuinely useful.
Are the automated parts, discovery, drafting, warming reminders, actually saving you decision time, or are they just moving the busywork to a different screen? Automation that still requires you to review, edit, and approve everything is not automation. It is a to-do list with extra steps.
I ask these questions because I lost real deals to cold relationships I forgot existed, and I do not want anyone else to learn that lesson the expensive way. A CRM built around one person, one record, from first contact to renewal, is not a nice-to-have feature. For a team of one to three people, it is the difference between running your pipeline and your pipeline running you.
That is the problem Nanobrains set out to solve with Distribution Engine, and it is why I hold it to a higher bar than I would hold a typical SaaS tool. If you are doing this work alone or with one or two other people, it is worth seeing whether it holds up against your own Tuesday afternoons.

Enhance Your JPG Images with Ellipse Annotations Using Java REST API

Shahzad Ashraf — Wed, 22 Jul 2026 15:27:51 +0000

Adding ellipse annotations to JPG images can be surprisingly straightforward with the right tools. The GroupDocs.Annotation Cloud SDK for Java empowers you to annotate images efficiently via a REST API, making complex image manipulation accessible and fast. This approach eliminates the need for heavy local processing, allowing developers to focus on core functionalities instead of tedious image-editing tasks.

The SDK provides a simple and intuitive way to implement annotations, supporting various file formats. By leveraging REST API principles, developers can easily integrate image annotation features into their applications without the overhead of complex setups. The cloud-based nature of the SDK means you can work from anywhere, ensuring flexibility in your development process.

If you're ready to enhance your image handling capabilities, check out the working code examples in the article to get started. Dive into the documentation and elevate your projects with powerful annotation features.

https://kb.groupdocs.cloud/annotation/java/add-ellipse-annotations-to-jpg-images-using-java-rest-api/

Unleash Your APIs: Why Swagger UI is a Game-Changer for Developers!

GitHubOpenSource — Wed, 22 Jul 2026 15:01:05 +0000

Quick Summary: 📝

Swagger UI is a JavaScript-based tool that generates interactive API documentation from OpenAPI specifications. It allows developers and consumers to visualize and interact with API resources without needing to implement the API logic, making API exploration and integration significantly easier.

Key Takeaways: 💡

✅ Swagger UI generates interactive and visually appealing API documentation from your OpenAPI Specification.
✅ It allows developers to visualize, understand, and even test API endpoints directly in the browser.
✅ Documentation is automatically kept up-to-date with your API definition, preventing discrepancies.
✅ It significantly improves collaboration between backend and frontend teams by providing a clear API contract.
✅ Available as various npm packages (swagger-ui, swagger-ui-dist, swagger-ui-react) for flexible integration into different project types.

Project Statistics: 📊

⭐ Stars: 28917
🍴 Forks: 9263
❗ Open Issues: 1284

Tech Stack: 💻

✅ JavaScript

Hey fellow developers! Have you ever found yourself wrestling with cryptic API documentation, trying to figure out what endpoints do what, or what kind of data you're supposed to send? We've all been there, and it can be a real productivity killer. But what if I told you there's a fantastic open-source project that makes exploring and interacting with APIs an absolute breeze? Prepare to be amazed by Swagger UI!

Swagger UI is nothing short of brilliant. It's a powerful tool that takes your OpenAPI (formerly Swagger) Specification and transforms it into stunning, interactive, and human-readable API documentation right in your browser. Imagine a beautifully designed web page where every single one of your API's resources is laid out clearly. You can see all the available endpoints, understand their parameters, view example requests and responses, and even make live API calls directly from the UI! It's like having a sandbox for your API, without writing a single line of client-side code to test it.

The beauty of Swagger UI lies in its simplicity and automation. Once you have your OpenAPI Specification defined (which describes your API's structure), Swagger UI automatically generates this interactive documentation. This means your documentation is always up-to-date with your API's definition, eliminating the dreaded problem of outdated or incorrect docs. It acts as a single source of truth, making life easier for everyone involved.

For backend developers, it's a fantastic way to validate your API design and ensure it's easily consumable. For frontend developers, it's an invaluable resource for quickly understanding how to integrate with the backend, reducing guesswork and speeding up development cycles. Collaboration between teams becomes seamless because everyone is looking at the same, clear, and executable documentation. No more endless back-and-forth trying to clarify API contracts!

Swagger UI is also incredibly versatile in how you can integrate it into your projects. You can use the swagger-ui npm module for modern single-page applications, swagger-ui-dist if you need a dependency-free bundle for server-side projects, or even swagger-ui-react if you're building a React application. This flexibility means there's almost certainly a way to get this incredible tool working for you. Seriously, if you're building or consuming APIs, you absolutely need to check out Swagger UI. It's a massive time-saver and a huge boost to developer experience!

Learn More: 🔗

View the Project on GitHub

🌟 Stay Connected with GitHub Open Source!

📱 Join us on Telegram

Get daily updates on the best open-source projects

GitHub Open Source

👥 Follow us on Facebook

Connect with our community and never miss a discovery

GitHub Open Source

Amazon Scraping Tools Compared — Which One Should You Use in 2026?

Nick Davies — Wed, 22 Jul 2026 14:59:17 +0000

If you need to scrape Amazon data, there are multiple tools available. But which one should you pick? I compared the top 5 options.

Tool	Developer	Rating	Users	Pricing
Amazon Product Scraper	Junglee	4.4	20K	Pay per event
Amazon Scraper	Junglee	5.0	14K	Pay per event
Amazon Reviews Scraper	Junglee	3.2	11K	Pay per event
Amazon Bestsellers Scraper	Junglee	5.0	4K	Pay per event
Amazon ASINs Scraper	Junglee	5.0	2K	Pay per event

Detailed Breakdown

1. Amazon Product Scraper

Use this Amazon scraper to collect data based on URL and country from the Amazon website. Extract product information without using the Amazon API, including reviews, prices, descriptions, and Amazon

Users: 20K
Rating: 4.4/5
Pricing: Pay per event

Try it on Apify

2. Amazon Scraper

Gets you product data from Amazon. Unofficial API. Scrapes and downloads product information without using the Amazon API, including reviews, prices, descriptions, and ASIN.

Users: 14K
Rating: 5.0/5
Pricing: Pay per event

Try it on Apify

3. Amazon Reviews Scraper

Amazon scraper to extract reviews from Amazon products. Scrape and download detailed reviews without using the Amazon API, including rating score, review description, reactions and images. Download yo

Users: 11K
Rating: 3.2/5
Pricing: Pay per event

Try it on Apify

4. Amazon Bestsellers Scraper

Scrape the Amazon Best Sellers categories and extract details on top 100 most popular items on Amazon. Download product name, price, URL, and thumbnail image. Best used on .com, .co.uk, .de, .fr, .es,

Users: 4K
Rating: 5.0/5
Pricing: Pay per event

Try it on Apify

5. Amazon ASINs Scraper

Gets you product data from Amazon. Unofficial API. Scrapes and downloads product information without using the Amazon API, including reviews, prices, descriptions.

Users: 2K
Rating: 5.0/5
Pricing: Pay per event

Try it on Apify

All tools are available on Apify with $5/month in free credits for new accounts.

Full directory: 26,000+ tools on GitHub

Transcription Webhooks & Callbacks: The Complete Guide

Mart Schweiger — Wed, 22 Jul 2026 14:56:03 +0000

If you're polling GET /v2/transcript/{id} every few seconds waiting for a transcript to finish, you're burning requests and adding latency for no reason. AssemblyAI can just call you back. Set one field, expose one endpoint, and you get a POST the moment transcription completes.

This guide covers the whole path—not just "set webhook_url," but the part most docs skip: what the payload actually contains, how to verify the request is really from us, and how retries behave when your server hiccups. If you're building anything production-grade on top of speech-to-text, read to the end.

Polling vs. webhooks: when to use which

Both approaches answer the same question—"is my transcript done yet?"—but they trade off differently.

Polling means repeatedly calling GET /v2/transcript/{id} (say, every 3 seconds) until status comes back completed or error. It's dead simple, needs no public endpoint, and works from a script on your laptop. The cost is wasted requests and a delay equal to your polling interval.

Webhooks flip the direction. You register a URL, AssemblyAI POSTs to it the instant the job finishes, and you do zero waiting. The tradeoff: you need a publicly reachable endpoint that returns a 2xx within 10 seconds, plus a little plumbing to secure and verify it.

Rule of thumb: prototypes and CLI tools poll; backend services that process audio at any real volume use webhooks. If you're running a queue of transcription jobs behind a web service, webhooks are the obvious choice.

Enabling a webhook

You enable a webhook by setting webhook_url in the body of your POST /v2/transcript request against the base URL https://api.assemblyai.com. Auth is a plain authorization header with your API key—no Bearer prefix.

Here it is with curl:

curl https://api.assemblyai.com/v2/transcript \
  --header "authorization: <YOUR_API_KEY>" \
  --header "content-type: application/json" \
  --data '{
    "audio_url": "https://example.com/audio.mp3",
    "webhook_url": "https://your-app.com/webhooks/transcript"
  }'

In Python, build a TranscriptionConfig, call set_webhook(url), and submit the job with submit()—not transcribe(). Using submit() matters: it returns immediately instead of blocking until the transcript is ready, which is the entire point of a webhook:

import assemblyai as aai

aai.settings.api_key = "<YOUR_API_KEY>"

config = aai.TranscriptionConfig().set_webhook(
    "https://your-app.com/webhooks/transcript"
)

transcriber = aai.Transcriber(config=config)
transcript = transcriber.submit("https://example.com/audio.mp3")

print(f"Submitted {transcript.id}. We'll get a callback when it's done.")

And in JavaScript:

import { AssemblyAI } from "assemblyai";

const client = new AssemblyAI({ apiKey: "<YOUR_API_KEY>" });

const transcript = await client.transcripts.submit({
  audio: "https://example.com/audio.mp3",
  webhook_url: "https://your-app.com/webhooks/transcript",
});

console.log(`Submitted ${transcript.id}`);

That's the whole setup. Full field reference lives in the submit endpoint docs.

What the payload actually contains

Here's the gotcha that trips up almost everyone. When transcription finishes, AssemblyAI POSTs to your URL with a minimal payload:

{
  "transcript_id": "your-transcript-id",
  "status": "completed"
}

status is either completed or error. That's it.

The payload does not contain the transcript text. It does not contain the error details either. To get the words—or to read the error field when something failed—you have to call GET /v2/transcript/{transcript_id} yourself. The webhook is a notification, not a delivery of the result.

So your receiver's job is really two steps: catch the notification, then go fetch the actual transcript:

import requests

def fetch_transcript(transcript_id, api_key):
    response = requests.get(
        f"https://api.assemblyai.com/v2/transcript/{transcript_id}",
        headers={"authorization": api_key},
    )
    response.raise_for_status()
    return response.json()

If you internalize one thing from this article, make it this: the webhook tells you when, the GET tells you what.

Building a receiver endpoint

Let's put it together into a complete, working receiver. Here's a Python/Flask service that validates the custom auth header, checks the status, fetches the transcript on success, and reads the error field on failure:

import os
import requests
from flask import Flask, request, abort

app = Flask(__name__)

ASSEMBLYAI_API_KEY = os.environ["ASSEMBLYAI_API_KEY"]
WEBHOOK_SECRET = os.environ["WEBHOOK_SECRET"]

def fetch_transcript(transcript_id):
    response = requests.get(
        f"https://api.assemblyai.com/v2/transcript/{transcript_id}",
        headers={"authorization": ASSEMBLYAI_API_KEY},
    )
    response.raise_for_status()
    return response.json()

@app.route("/webhooks/transcript", methods=["POST"])
def handle_webhook():
    # 1. Verify the request via your custom auth header.
    if request.headers.get("X-My-Webhook-Secret") != WEBHOOK_SECRET:
        abort(401)

    payload = request.get_json(silent=True) or {}
    transcript_id = payload.get("transcript_id")
    status = payload.get("status")

    if not transcript_id:
        # Malformed request—4xx tells AssemblyAI to stop retrying.
        abort(400)

    # 2. Branch on status. Fetch the real data with a GET.
    if status == "completed":
        transcript = fetch_transcript(transcript_id)
        print(f"Transcript {transcript_id} ready: {transcript['text'][:120]}...")
        # ... enqueue downstream work, write to your DB, etc.
    elif status == "error":
        transcript = fetch_transcript(transcript_id)
        print(f"Transcript {transcript_id} failed: {transcript['error']}")
        # ... alert, retry the job, flag the customer record, etc.

    # 3. Return 2xx fast so AssemblyAI marks delivery successful.
    return "", 200

Notice the shape of it. Verify first, respond fast, and keep heavy work out of the request path. If fetching and processing the transcript is slow, push it onto a background queue and return 200 immediately—you've only got a 10-second budget, which we'll get to.

The same structure works in Node/Express:

import express from "express";

const app = express();
app.use(express.json());

const API_KEY = process.env.ASSEMBLYAI_API_KEY;
const WEBHOOK_SECRET = process.env.WEBHOOK_SECRET;

app.post("/webhooks/transcript", async (req, res) => {
  if (req.header("X-My-Webhook-Secret") !== WEBHOOK_SECRET) {
    return res.sendStatus(401);
  }

  const { transcript_id, status } = req.body;
  if (!transcript_id) return res.sendStatus(400);

  // Respond first, then do the work off the request path.
  res.sendStatus(200);

  const result = await fetch(
    `https://api.assemblyai.com/v2/transcript/${transcript_id}`,
    { headers: { authorization: API_KEY } }
  ).then((r) => r.json());

  if (status === "completed") {
    console.log(`Ready: ${result.text.slice(0, 120)}...`);
  } else if (status === "error") {
    console.error(`Failed: ${result.error}`);
  }
});

app.listen(3000);

Securing your webhook

There's no HMAC or signature scheme here—AssemblyAI doesn't sign the payload. So don't go looking for a signature header to validate. Instead, you get three complementary controls, and you should use more than one.

Custom auth header. Attach a secret header when you submit the job, and check it on every incoming request. Set webhook_auth_header_name and webhook_auth_header_value in the submit body:

curl https://api.assemblyai.com/v2/transcript \
  --header "authorization: <YOUR_API_KEY>" \
  --header "content-type: application/json" \
  --data '{
    "audio_url": "https://example.com/audio.mp3",
    "webhook_url": "https://your-app.com/webhooks/transcript",
    "webhook_auth_header_name": "X-My-Webhook-Secret",
    "webhook_auth_header_value": "secret-value"
  }'

In Python, pass the extra arguments to set_webhook:

config = aai.TranscriptionConfig().set_webhook(
    "https://your-app.com/webhooks/transcript",
    "X-My-Webhook-Secret",
    "secret-value",
)

AssemblyAI will send that header back on the webhook request, and your receiver rejects anything without the right value—that's the abort(401) in the example above.

Source-IP allow-list. Webhook requests originate from fixed IPs: 44.238.19.20 in the US and 54.220.25.36 in the EU. Restrict your endpoint to those addresses at the firewall or load balancer for a second layer of defense.

webhook_status_code. After delivery, the transcript object carries a webhook_status_code field—the HTTP status code AssemblyAI received from your endpoint. Pull it with a GET to confirm your receiver actually returned what you think it did. It's the fastest way to debug a webhook that "isn't firing" (spoiler: it usually is, and your endpoint returned a 500).

Use the auth header and the IP allow-list together. Neither is a signature, but combined they're a solid perimeter.

Reliability and retries

Your endpoint has to return a 2xx status within 10 seconds. That's the contract.

If you don't—timeout, 5xx, connection refused—AssemblyAI retries. You get up to 10 total attempts, spaced 10 seconds apart. That gives your service a bit under two minutes to recover from a transient blip without losing the notification.

One sharp edge: a 4xx response marks the delivery as failed and stops retries immediately. The reasoning is that 4xx means "your request is wrong and will always be wrong"—retrying won't help. So return 4xx only for genuinely bad requests (a missing transcript_id, a failed auth check). If your database is momentarily down, return a 5xx so the retry machinery kicks in and you get another shot.

This is exactly why the receiver examples respond 200 before doing the heavy lifting. Fetching the transcript, running downstream analysis, writing to storage—none of that should happen inside the 10-second window. Acknowledge fast, process async.

Adding metadata via query params

The webhook payload only carries transcript_id and status, so how do you know which customer or order a given callback belongs to? Put it in the URL. AssemblyAI POSTs to your webhook_url exactly as you supplied it, query string and all:

config = aai.TranscriptionConfig().set_webhook(
    "https://your-app.com/webhooks/transcript?customer_id=1234&order_id=5678"
)

When the callback lands, read those params off the request:

customer_id = request.args.get("customer_id")
order_id = request.args.get("order_id")

This is the clean way to correlate deliveries and do things like per-customer usage tracking without a lookup table keyed on transcript_id. Just don't put secrets in the query string—keep those in the auth header.

Testing locally

Your receiver needs a public URL, which is awkward on localhost. Use a tunneling tool—ngrok, Cloudflare Tunnel, or similar—to expose your local port to the internet, then submit a job pointing webhook_url at the tunnel's HTTPS address.

ngrok http 3000
# Use the forwarding URL (e.g. https://abc123.ngrok.io/webhooks/transcript)
# as your webhook_url when submitting a transcript.

Submit a short audio file, watch the request hit your terminal, and confirm you return 200. If nothing arrives, GET the transcript and check webhook_status_code—it'll tell you whether AssemblyAI reached your endpoint and what it got back.

A note on streaming webhooks

Everything above is for pre-recorded (async) transcription. If you're working with real-time audio, streaming has its own webhook mechanism—see the streaming webhooks docs. Don't mix the two; the payloads and lifecycle differ.

Next steps

Webhooks turn transcription from something you wait on into something that notifies you. Set webhook_url, verify the request, respond fast, and fetch the transcript with a GET—that's a production-grade integration.

From here, dig into the pre-recorded audio webhooks guide for the canonical reference, or read up on what speech-to-text is and how accurate it is in 2026 if you're still evaluating models.

View the full API reference for every field on the transcript object.

Frequently asked questions

Does AssemblyAI support webhooks?

Yes. Set webhook_url in the body of your POST /v2/transcript request (or use set_webhook() in the Python SDK / webhook_url in the JS submit() call), and AssemblyAI will POST to that URL when transcription finishes. It's the recommended alternative to polling for backend services.

What's in the webhook payload?

Just two fields: transcript_id and status (completed or error). The payload does not include the transcript text or error details. To get the actual result—or the error field on failure—call GET /v2/transcript/{transcript_id}.

How do I secure and verify webhook requests?

There's no HMAC signature. Use a custom auth header (webhook_auth_header_name and webhook_auth_header_value) that your receiver checks on every request, restrict your endpoint to AssemblyAI's source IPs (44.238.19.20 in the US, 54.220.25.36 in the EU), and inspect the webhook_status_code field on the transcript to confirm what your endpoint returned.

What happens if my endpoint is down when a webhook fires?

Your endpoint must return a 2xx within 10 seconds. If it doesn't, AssemblyAI retries up to 10 total attempts, 10 seconds apart. A 4xx response marks the delivery failed and stops retries immediately, so reserve 4xx for genuinely bad requests and return 5xx for transient errors you want retried.

Should I use webhooks or polling?

Polling (GET /v2/transcript/{id} every few seconds) is simplest and needs no public endpoint—good for scripts and prototypes. Webhooks push a notification the instant a job finishes with no wasted requests, but require a publicly reachable endpoint that returns 2xx in 10 seconds. Use webhooks for production backend services.

Can I attach my own metadata to a webhook?

Yes. Add query parameters to your webhook_url, like ?customer_id=1234&order_id=5678. AssemblyAI POSTs to the URL exactly as you supplied it, so you read those params off the incoming request to correlate deliveries. Keep secrets in the auth header, not the query string.

How to Transcribe Audio From a Mobile App (iOS, Android)

Mart Schweiger — Wed, 22 Jul 2026 14:55:56 +0000

If you searched for an AssemblyAI mobile SDK, here's the short version: there isn't one. The official SDKs are Python (pip install assemblyai) and JavaScript/TypeScript (npm install assemblyai). No Swift, no Kotlin, no React Native package.

That's not the problem you think it is. The bigger constraint is security: don't ship your API key to client-side code. A mobile binary can be decompiled, and anyone who pulls your key out of it can spend your account balance. So even if a native SDK existed, calling AssemblyAI directly from a phone would be the wrong move.

The right pattern is a thin backend you control. Your mobile app records audio and sends it to your server. Your server holds the API key, talks to AssemblyAI over the REST API, and returns plain text. That's the whole lesson, and it's a good one—you get key security, request logging, rate limiting, and the freedom to swap models later without shipping a new app build.

Here's what you'll build:

A backend (Node or Python) that receives an audio file, uploads it to AssemblyAI, creates a transcript, polls for the result, and returns the text.
Three thin clients—Swift, Kotlin, and React Native—that record audio and call your backend, never AssemblyAI.

Prerequisites

An AssemblyAI account and API key. Get your free API key from the dashboard.
Node 18+ or Python 3.8+ for the backend.
Xcode for iOS, Android Studio for Android, or a React Native toolchain—whichever platforms you're targeting.

If you're new to the API, the transcribe an audio file quickstart covers the async flow end to end. This tutorial adapts it for mobile.

Architecture overview

One rule shapes everything: the API key lives on the server, and only the server talks to AssemblyAI.

┌──────────────┐        ┌──────────────────────┐        ┌─────────────────┐
│  Mobile app  │        │   Your backend       │        │   AssemblyAI    │
│  (Swift /    │        │   (holds API key)    │        │   REST API      │
│   Kotlin /   │        │                      │        │                 │
│   RN)        │        │                      │        │                 │
│              │──audio─▶│ POST /transcribe     │        │                 │
│              │  file  │   1. POST /v2/upload ─┼──bytes─▶ upload_url      │
│              │        │   2. POST /v2/transcript──JSON─▶ transcript id   │
│              │        │   3. GET  /v2/transcript/{id}──poll──▶ completed │
│              │◀─text──│   return { text }     │        │                 │
└──────────────┘        └──────────────────────┘        └─────────────────┘

Note what's not in the diagram: the mobile client never sees the API key and never has AssemblyAI's URL. It only knows about your endpoint. If you rotate the key or change models, the app doesn't change.

Why a backend proxy specifically? For pre-recorded audio there's no temporary-token mechanism—temporary tokens exist only for streaming. So for file transcription, the key has to live somewhere trusted, and that's your server.

The AssemblyAI async flow

Everything the backend does is three REST calls against https://api.assemblyai.com. Auth is your API key in the authorization header—no Bearer prefix.

Upload (optional): POST /v2/upload with the raw audio bytes as the body. Returns { "upload_url": "..." }. Skip this if your audio already lives at a public URL.
Create transcript: POST /v2/transcript with JSON { "audio_url": "..." }. Returns a transcript object with an id and a status.
Get result: GET /v2/transcript/{id}. Poll every few seconds until status is completed or error.

The status lifecycle is queued → processing → completed (or error).

One gotcha that bites people on the upload step: the body must be raw bytes, not JSON. If you send a JSON envelope, the upload appears to succeed but transcription fails later with "Transcoding failed." Send the file bytes directly.

Backend implementation

Pick your language. Both versions expose a single POST /transcribe endpoint that accepts a multipart file upload from the mobile client and returns { "text": "..." }. Both read the API key from an environment variable—never hardcode it.

Node (Express)

npm install express multer
export ASSEMBLYAI_API_KEY="your_key_here"

import express from "express";
import multer from "multer";

const app = express();
const upload = multer({ storage: multer.memoryStorage() });

const API_KEY = process.env.ASSEMBLYAI_API_KEY;
const BASE_URL = "https://api.assemblyai.com";

// Poll until the transcript is done or errors out.
async function pollTranscript(id) {
  while (true) {
    const res = await fetch(`${BASE_URL}/v2/transcript/${id}`, {
      headers: { authorization: API_KEY },
    });
    const transcript = await res.json();

    if (transcript.status === "completed") return transcript;
    if (transcript.status === "error") {
      throw new Error(transcript.error);
    }
    await new Promise((r) => setTimeout(r, 3000)); // poll every 3s
  }
}

app.post("/transcribe", upload.single("audio"), async (req, res) => {
  try {
    // 1. Upload the raw audio bytes. Body is the buffer, NOT JSON.
    const uploadRes = await fetch(`${BASE_URL}/v2/upload`, {
      method: "POST",
      headers: { authorization: API_KEY },
      body: req.file.buffer,
    });
    const { upload_url } = await uploadRes.json();

    // 2. Create the transcript.
    const createRes = await fetch(`${BASE_URL}/v2/transcript`, {
      method: "POST",
      headers: {
        authorization: API_KEY,
        "content-type": "application/json",
      },
      body: JSON.stringify({ audio_url: upload_url }),
    });
    const { id } = await createRes.json();

    // 3. Poll for the result and return just the text.
    const transcript = await pollTranscript(id);
    res.json({ text: transcript.text });
  } catch (err) {
    res.status(500).json({ error: err.message });
  }
});

app.listen(8080, () => console.log("Listening on :8080"));

Python (FastAPI)

pip install fastapi "uvicorn[standard]" python-multipart requests
export ASSEMBLYAI_API_KEY="your_key_here"

import os
import time

import requests
from fastapi import FastAPI, UploadFile, HTTPException

app = FastAPI()

API_KEY = os.environ["ASSEMBLYAI_API_KEY"]
BASE_URL = "https://api.assemblyai.com"
HEADERS = {"authorization": API_KEY}

def poll_transcript(transcript_id: str) -> dict:
    while True:
        res = requests.get(f"{BASE_URL}/v2/transcript/{transcript_id}", headers=HEADERS)
        transcript = res.json()
        if transcript["status"] == "completed":
            return transcript
        if transcript["status"] == "error":
            raise RuntimeError(transcript["error"])
        time.sleep(3)  # poll every 3s

@app.post("/transcribe")
async def transcribe(audio: UploadFile):
    try:
        audio_bytes = await audio.read()

        # 1. Upload the raw bytes. data=audio_bytes sends them as-is, not JSON.
        upload = requests.post(
            f"{BASE_URL}/v2/upload", headers=HEADERS, data=audio_bytes
        )
        upload_url = upload.json()["upload_url"]

        # 2. Create the transcript.
        create = requests.post(
            f"{BASE_URL}/v2/transcript",
            headers={**HEADERS, "content-type": "application/json"},
            json={"audio_url": upload_url},
        )
        transcript_id = create.json()["id"]

        # 3. Poll and return the text.
        transcript = poll_transcript(transcript_id)
        return {"text": transcript["text"]}
    except Exception as err:
        raise HTTPException(status_code=500, detail=str(err))

Run it with uvicorn main:app --port 8080.

A few production notes that apply to both:

Polling vs. webhooks. Polling every 3 seconds is simple and fine for short clips. For longer audio, hold the HTTP connection open only if your infra tolerates it—otherwise return the transcript id immediately and let the client poll a GET /transcribe/{id} route, or register a webhook so AssemblyAI notifies you when the job finishes.
Limits. The upload endpoint accepts files up to 2.2 GB. The transcript request body caps at 5 GB, and audio duration must be between 160ms and 10 hours.
Add features here, not on the client. Want speaker labels or summarization? Add the options to the /v2/transcript JSON body server-side. The app doesn't change.

For every field you can pass, see the submit transcript reference, the upload reference, and the get transcript reference.

Client implementations

All three clients do the same three things: record audio to a file, POST that file to your backend as multipart form data, and show the returned text. None of them know AssemblyAI exists.

Set BACKEND_URL to wherever your server runs. Use http://localhost:8080 for a simulator, or your machine's LAN IP for a physical device.

iOS / Swift

Record with AVAudioRecorder, then upload with URLSession. Add NSMicrophoneUsageDescription to your Info.plist first.

import AVFoundation

final class Transcriber: NSObject, AVAudioRecorderDelegate {
    private var recorder: AVAudioRecorder?
    private let backendURL = URL(string: "http://localhost:8080/transcribe")!
    private lazy var fileURL: URL = {
        FileManager.default.temporaryDirectory.appendingPathComponent("recording.m4a")
    }()

    func startRecording() throws {
        let session = AVAudioSession.sharedInstance()
        try session.setCategory(.playAndRecord, mode: .default)
        try session.setActive(true)

        let settings: [String: Any] = [
            AVFormatIDKey: Int(kAudioFormatMPEG4AAC),
            AVSampleRateKey: 44100,
            AVNumberOfChannelsKey: 1,
        ]
        recorder = try AVAudioRecorder(url: fileURL, settings: settings)
        recorder?.record()
    }

    func stopAndTranscribe() async throws -> String {
        recorder?.stop()

        let audioData = try Data(contentsOf: fileURL)
        let boundary = "Boundary-\(UUID().uuidString)"

        var request = URLRequest(url: backendURL)
        request.httpMethod = "POST"
        request.setValue(
            "multipart/form-data; boundary=\(boundary)",
            forHTTPHeaderField: "Content-Type"
        )

        // Build the multipart body under the "audio" field name.
        var body = Data()
        body.append("--\(boundary)\r\n".data(using: .utf8)!)
        body.append(
            "Content-Disposition: form-data; name=\"audio\"; filename=\"recording.m4a\"\r\n"
                .data(using: .utf8)!
        )
        body.append("Content-Type: audio/m4a\r\n\r\n".data(using: .utf8)!)
        body.append(audioData)
        body.append("\r\n--\(boundary)--\r\n".data(using: .utf8)!)

        let (data, _) = try await URLSession.shared.upload(for: request, from: body)
        let result = try JSONDecoder().decode(TranscriptResponse.self, from: data)
        return result.text
    }
}

struct TranscriptResponse: Decodable {
    let text: String
}

No API key anywhere. The client's entire world is backendURL.

Android / Kotlin

Record with MediaRecorder, upload with OkHttp. Declare RECORD_AUDIO in your manifest and request it at runtime.

import android.media.MediaRecorder
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.MultipartBody
import okhttp3.OkHttpClient
import okhttp3.Request
import okhttp3.RequestBody.Companion.asRequestBody
import org.json.JSONObject
import java.io.File

class Transcriber(private val cacheDir: File) {

    private val client = OkHttpClient()
    private val backendUrl = "http://10.0.2.2:8080/transcribe" // 10.0.2.2 = host from emulator
    private val outputFile = File(cacheDir, "recording.m4a")
    private var recorder: MediaRecorder? = null

    fun startRecording() {
        recorder = MediaRecorder().apply {
            setAudioSource(MediaRecorder.AudioSource.MIC)
            setOutputFormat(MediaRecorder.OutputFormat.MPEG_4)
            setAudioEncoder(MediaRecorder.AudioEncoder.AAC)
            setAudioChannels(1)
            setOutputFile(outputFile.absolutePath)
            prepare()
            start()
        }
    }

    fun stopAndTranscribe(): String {
        recorder?.apply { stop(); release() }
        recorder = null

        // Multipart upload under the "audio" field name.
        val body = MultipartBody.Builder()
            .setType(MultipartBody.FORM)
            .addFormDataPart(
                "audio",
                "recording.m4a",
                outputFile.asRequestBody("audio/m4a".toMediaType())
            )
            .build()

        val request = Request.Builder().url(backendUrl).post(body).build()

        client.newCall(request).execute().use { response ->
            val json = JSONObject(response.body!!.string())
            return json.getString("text")
        }
    }
}

Call stopAndTranscribe() off the main thread (a coroutine on Dispatchers.IO works well).

React Native

Record with a library like react-native-audio-recorder-player, then upload with the built-in fetch and FormData:

import AudioRecorderPlayer from "react-native-audio-recorder-player";

const recorder = new AudioRecorderPlayer();
const BACKEND_URL = "http://localhost:8080/transcribe";

export async function startRecording() {
  return recorder.startRecorder(); // returns the file path
}

export async function stopAndTranscribe() {
  const filePath = await recorder.stopRecorder();

  const form = new FormData();
  form.append("audio", {
    uri: filePath,
    name: "recording.m4a",
    type: "audio/m4a",
  });

  const response = await fetch(BACKEND_URL, {
    method: "POST",
    body: form, // don't set Content-Type; fetch sets the multipart boundary
  });

  const { text } = await response.json();
  return text;
}

Because JavaScript is a first-class citizen here, you might be tempted to reach for the assemblyai npm package inside your React Native app. Don't. The Node SDK is built for server environments, and using it on the client means embedding your API key in the bundle—the exact thing we're avoiding. The fetch-to-your-backend pattern above is the correct one for React Native.

Testing and validation

Test the backend on its own before wiring up a phone. Record or grab any short audio file and hit the endpoint with curl:

curl -X POST http://localhost:8080/transcribe \
  -F "audio=@sample.m4a"

You should get back {"text":"..."} within a few seconds for a short clip. If you don't:

"Transcoding failed" almost always means the upload body wasn't raw bytes. Confirm you're sending the buffer/bytes directly, not a JSON wrapper.
401 Unauthorized means the authorization header is missing or wrong. Check that ASSEMBLYAI_API_KEY is set in the server's environment—and remember, no Bearer prefix.
The request hangs on long audio because polling holds the connection open. Switch to the return-id-and-poll pattern or a webhook.

Once curl works, point the app at the backend. On a physical device, localhost won't resolve to your dev machine—use its LAN IP (for example http://192.168.1.20:8080/transcribe), and on Android the emulator reaches the host at 10.0.2.2.

Realtime from mobile (next step)

If you need live captions instead of file-at-a-time transcription, AssemblyAI has a streaming API over WebSocket at wss://streaming.assemblyai.com/v3/ws. There's still no mobile-native streaming SDK, but streaming has something file transcription doesn't: short-lived tokens.

Your backend calls GET /v3/token?expires_in_seconds=60 (tokens last 1–600s and are single-use), hands the token to the app, and the app opens the WebSocket with the token as a token query param. The key still never leaves your server; the token is safe to give the client because it expires fast and works once. Audio goes up as mono 16-bit PCM.

That flow is documented in authenticate with a temporary token. For a working streaming client to model your backend on, see real-time transcription in Python.

Next steps

You now have a mobile transcription pipeline that keeps your API key where it belongs. From here:

Add speech understanding. Speaker diarization, summarization, and more are options on the /v2/transcript body—turn them on server-side and your app gets them for free. Start with what is speech-to-text and the Speech-to-Text product page.
Harden the backend. Add authentication on your /transcribe route, rate limiting, and request logging—the proxy is the natural place for all of it.
Explore every parameter. The full API reference documents the complete transcript object.

Frequently asked questions

Is there an official AssemblyAI iOS or Android SDK?

No. The only official SDKs are Python (pip install assemblyai) and JavaScript/TypeScript (npm install assemblyai). For mobile, call the REST API at https://api.assemblyai.com from a backend you control.

Can I call the AssemblyAI API directly from my mobile app?

Technically the REST API works from anywhere, but you shouldn't. Calling it directly means embedding your API key in the app binary, where it can be extracted. Route every request through your own backend instead.

How do I keep my API key safe in a mobile app?

Never ship it to the client. Store it as an environment variable on your server, have the app send audio to your backend, and let the backend attach the authorization header when it calls AssemblyAI. That's the architecture in this tutorial.

Does AssemblyAI support React Native?

There's no React Native SDK. Use fetch with FormData to send audio to your backend, exactly like the web. Avoid the assemblyai npm package on the client—it's a server SDK and would expose your key.

Can I do realtime transcription on mobile?

Yes, over the streaming WebSocket at wss://streaming.assemblyai.com/v3/ws. There's no mobile streaming SDK, so your backend mints a one-time token via GET /v3/token?expires_in_seconds=60 and the app connects with it. Audio is mono 16-bit PCM. See the temporary token docs.

What are the file size and duration limits?

The upload endpoint accepts files up to 2.2 GB. The transcript request body caps at 5 GB, and audio must run between 160ms and 10 hours.

AI Medical Scribe: Build vs Buy Nuance DAX & Abridge

Mart Schweiger — Wed, 22 Jul 2026 14:55:23 +0000

Every healthcare AI team lands on the same question eventually. You want ambient clinical documentation in your product, and there are two obvious paths. Buy a finished scribe like Nuance DAX Copilot or Abridge and wire it in. Or build your own.

Here's the reframe most comparison posts miss. DAX and Abridge are finished products. They're good ones. If you're a health system that wants a scribe in clinicians' hands next quarter and you're never going to differentiate on documentation itself, buying is often the right call. Full stop.

But if you're a product team building a clinical application, a specialty EHR, or an ambient scribe you plan to sell, the "which scribe do I buy" question is the wrong one. You're not shopping for a scribe. You're deciding whether documentation is a feature you own or a feature you rent. And if you build, the question becomes: what do you build on?

That's the fork. Let's walk it honestly.

The build-vs-buy fork for ambient clinical documentation

When you buy DAX or Abridge, you get a lot the same day: a working scribe, a note format clinicians already recognize, EHR integrations, a compliance posture, and a vendor who owns the roadmap. That's real value, and pretending otherwise is a mistake.

What you give up is control. The note format is theirs. The specialties they support are theirs. The pace of improvement is theirs. And the economics are per-seat, per-provider, forever—which means your gross margin on documentation is capped by someone else's price list.

When you build, you flip every one of those. You own the note structure, the specialty coverage, the UX, the model choices underneath, and the margin. You also own the work: accurate medical transcription, speaker separation, note generation, and compliance. That's not a weekend project. But none of it is mysterious anymore, and most of the hard part isn't where teams expect.

What "buy" actually costs you

The sticker price is the easy part. The real costs show up later.

Margin. Per-seat pricing scales with your clinician count, not your efficiency. Get 10,000 providers on the platform and your documentation cost grows linearly right alongside. You can't engineer that number down because you don't own the stack.

Control over the note. Cardiology notes and behavioral health notes shouldn't look the same. If your differentiation is a note tuned to a specialty or a workflow, a general-purpose scribe fights you on it. You file feature requests and wait.

Lock-in. Once your workflows, integrations, and clinician muscle memory are built around a vendor's output format, switching is a migration project, not a config change. That's leverage—theirs, not yours.

Differentiation. If you and three competitors all ship the same underlying scribe, documentation stops being a reason anyone chooses you. It becomes table stakes you're paying a premium to rent.

None of this makes buying wrong. It makes buying a strategic decision about where your product's value actually lives. If documentation isn't your edge, rent it and move on. If it is—or could be—renting your core is a strange way to build a company.

What "build" actually requires

An ambient scribe is a pipeline, and it's shorter than it looks. Four layers.

Medical speech-to-text. The audio-to-text layer. Multi-speaker clinical audio, drug names, procedures, dosages, accents, background noise. This is the input to everything downstream.
Speaker diarization. Who said what. You need clinician and patient separated cleanly, including the rapid back-and-forth and interruptions of a real visit.
Note generation. An LLM turns the diarized transcript into a structured note—SOAP, H&P, whatever your specialty needs. This is where teams love to spend their time.
Compliance. A BAA, PHI handling, access controls, audit trails.

Here's the part that surprises people. The LLM layer, the part everyone's excited about, is the most commoditized. Strong models are a few API calls away, and prompt engineering a good SOAP note is tractable. The layer that quietly decides whether your scribe is trustworthy is the first one—the speech-to-text foundation. Garbage transcript in, confident-sounding garbage note out. An LLM will happily summarize a hallucinated dosage into a clean, plausible sentence a clinician might sign.

So the build decision is really a decision about your transcription layer. Get that right and the rest of the pipeline has something solid to stand on.

The speech-to-text layer is make-or-break

This is the input that determines everything, so it's worth being specific about what "good enough for clinical" means.

Medical accuracy that isn't generic. General speech-to-text mangles clinical vocabulary—that's the whole problem. AssemblyAI's Universal-3.5 Pro is the flagship async model at $0.21/hr, and you turn on medical accuracy by adding one parameter: "domain": "medical-v1". That's Medical Mode, +$0.15/hr, so the flagship plus Medical Mode runs $0.36/hr combined. It reduces the Missed Entity Rate on drugs, conditions, procedures, and clinical terms by roughly 20%. No model switch, no separate pipeline—one param on the model you're already calling.

Context that adapts to the patient. Medical Mode gets you baseline clinical accuracy. Contextual prompting gets you the last mile. Prime the model with a patient's prior-visit note and it knows what to listen for. In an internal healthcare test, feeding a prior-visit note cut missed medical terms by 31%—even when the note was from an earlier visit. That's the kind of gain you can't buy off a shelf because it depends on data only your application has.

Diarization built in. Universal-3.5 Pro produces the transcript and the speaker turns together—the most accurate diarization we've shipped, optimized for cpWER and tuned for the short turns, rapid exchanges, and overlapped speech of an actual exam room. You're not bolting on a separate speaker model and hoping the timestamps line up.

A streaming option when you need it. Most ambient scribes are async—record the visit, generate the note. But if you're building live documentation, a real-time assistant, or clinician-facing prompts mid-visit, Universal-3.5 Pro Realtime offers Medical Mode too, with live diarization. Same foundation, real-time shape. That flexibility matters when your roadmap outgrows batch processing.

Languages. Medical Mode supports English, Spanish, German, and French—so a Spanish-language visit gets clinical-grade transcription, not a downgrade.

Compare that to buying: with DAX or Abridge, the transcription layer is a black box. You can't tune it, you can't prompt it with your data, and you can't fix it when it's wrong on a term that matters to your specialty. Building on a foundation you control means the accuracy ceiling is yours to raise. If you want the deeper argument for why this input dominates outcomes, we made it in the true cost of inaccurate transcription and in why transcription accuracy and transcription quality aren't the same thing.

Compliance, done right

Compliance is where "build" teams get nervous, and fair enough—PHI is not the place to improvise. The good news is that the transcription layer of your stack can carry a clean posture.

AssemblyAI enables covered entities and their business associates subject to HIPAA to use the AssemblyAI services to process protected health information (PHI). AssemblyAI is considered a business associate under HIPAA, and we offer a Business Associate Addendum (BAA) that is required under HIPAA to ensure that AssemblyAI appropriately safeguards PHI.

Practically, that BAA is available to sign without booking a sales call—so a compliance review doesn't turn into a procurement saga before you've written a line of code. For teams with data-residency requirements, EU processing is available too. Compliance is table stakes, but it shouldn't be the thing that slows your build to a crawl.

Build vs buy: the decision at a glance

Here's the comparison that actually matters—not a spec smackdown, but the strategic trade-offs.

Decision criterion	Buy (DAX / Abridge)	Build (on AssemblyAI)
Control over note format & UX	Vendor-defined; feature requests	Fully yours — tune per specialty and workflow
Gross margin on documentation	Capped by per-seat pricing	Yours to engineer; usage-based inputs
Time-to-market	Fastest — finished product	Longer, but the STT layer is an API call
Differentiation	Shared with everyone on the same scribe	Documentation becomes a product edge
Compliance ownership	Vendor's posture	Your posture; BAA available on the STT layer
Cost model	Per-seat, per-provider, recurring	Pay-as-you-go on transcription ($0.36/hr with Medical Mode)
Roadmap & pace of improvement	Vendor's priorities	Yours
Best fit	Orgs that won't differentiate on docs	Product teams selling or owning the experience

Who each path is for

Buy DAX or Abridge if: you're a health system or clinic that needs a working scribe in clinicians' hands fast, documentation isn't part of your competitive story, and you'd rather pay per seat than staff an engineering team against it. That's a legitimate, common, correct choice. These are mature products for exactly this buyer.

Build if: documentation is part of what you sell, you need note formats or specialty coverage a general scribe won't give you, per-seat economics break your margin at scale, or you're already a software company and owning the stack is how you compete. In that case, the smart move isn't building transcription from scratch—it's building on a transcription layer that's already solved medical accuracy, diarization, and compliance, so your team spends its time on the note logic and product experience that actually differentiate you.

Healthcare teams building their own documentation experiences tend to land in the second camp for the same reason: they want the margin and the control, and they don't want to reinvent speech-to-text to get there.

A quick decision checklist

Run your project through these before committing:

Is documentation a feature we sell, or a chore we want handled? If we sell it, lean build.
Do we need note formats or specialties a general scribe won't support? If yes, lean build.
What does per-seat pricing do to our margin at 1,000 and 10,000 providers? If it breaks, lean build.
Do we have (or want) engineering ownership of this workflow? If not, lean buy.
How fast do we need to ship? If it's this quarter and docs aren't our edge, lean buy.
If we build, is our transcription layer clinically accurate, diarized, and BAA-backed on day one? If not, fix that first—it's the foundation.

If you're weighing the broader landscape, our guides on Voice AI for healthcare market research and conversation intelligence show how the same speech foundation extends well past the exam room. For the numbers behind the models, the benchmarks and our take on how accurate speech-to-text is in 2026 are the receipts.

Explore Voice AI solutions to see the full platform under a scribe you own—or dig into the medical solution directly.

The insight most teams learn too late

Here's the thing nobody tells you at the start. Build vs buy isn't a one-time decision—it's a decision that gets more expensive to reverse the longer you wait. The team that buys "to move fast" and plans to build later usually can't—by then their workflows, integrations, and clinician habits are welded to a vendor's format. And the team that builds on a weak transcription layer finds the ceiling only after shipping, when clinicians stop trusting the notes.

So the real move is to decouple the decision. Even if you buy a finished scribe today, own the layer that's hardest to swap later—your speech-to-text foundation—so building stays a live option instead of a fantasy. Accuracy compounds. Lock-in compounds. Pick the one you want on your side.

Frequently asked questions

Should I build my own AI medical scribe or buy Nuance DAX or Abridge?

Buy if you need a finished scribe fast and documentation isn't your competitive edge—DAX and Abridge are mature products built for that. Build if documentation is part of what you sell, you need custom note formats or specialty coverage, per-seat pricing breaks your margin at scale, or you want to own the experience. If you build, don't build transcription from scratch—build on a medical-grade speech-to-text layer and spend your effort on note logic and UX.

Can I use AssemblyAI with PHI under HIPAA, and is a BAA available?

How accurate is AssemblyAI on medical terminology?

Universal-3.5 Pro with Medical Mode ("domain": "medical-v1") reduces the Missed Entity Rate on drugs, conditions, procedures, and clinical terms by roughly 20%. Contextual prompting goes further—feeding a patient's prior-visit note cut missed medical terms by 31% in an internal healthcare test, even when the note came from an earlier visit.

What does it cost to build a scribe on AssemblyAI?

Transcription is pay-as-you-go, billed per second with no minimums. Universal-3.5 Pro async is $0.21/hr, and Medical Mode adds $0.15/hr—so the flagship plus Medical Mode is $0.36/hr combined. That's a usage-based input cost you control, versus the recurring per-seat pricing of a finished scribe.

Which languages does Medical Mode support?

English, Spanish, German, and French. Medical Mode is available on Universal-3.5 Pro (async) and on Universal-3.5 Pro Realtime for streaming use cases.

How do I get started?

Add "domain": "medical-v1" to a Universal-3.5 Pro request and run it against your own clinical audio. Sign the BAA when you're ready to process PHI, and check the medical solution page for the full picture.

Best Voice Agent API for Startups: A First-Build Guide

Mart Schweiger — Wed, 22 Jul 2026 14:54:57 +0000

You've decided to build a voice product. Maybe it's a support agent, maybe an AI companion, maybe a voice layer on the app you already shipped. Either way, you're staring at a blank editor and a very fast-moving landscape, and the question in front of you isn't "which model is most accurate." It's "how do I get a working demo by Friday?"

If that's you, this guide is for you. Most people building their first voice product aren't voice agent power users. Roughly 80% are net-new to this. You're demo-driven, you're code-forward, and you don't want to spend your first week learning someone's proprietary conversation-design UI. So let's skip the theory and talk about the decision you're actually making.

What a voice agent API bundles, and why the pipeline is the hard part

A voice agent is a loop. Someone talks, your system transcribes it (speech-to-text), an LLM decides what to say, and a text-to-speech engine says it back. STT to LLM to TTS. That's the whole pipeline.

Sounds simple. It isn't.

The hard part isn't any single box in that diagram. It's the seams between them. When does the user stop talking? (Harder than it sounds - people pause mid-thought all the time.) What happens when they interrupt the agent? How do you keep latency low enough that the conversation doesn't feel like a walkie-talkie? And here's the one that quietly sinks most first builds: if the transcription is wrong, the LLM responds to the wrong thing. Garbage in, confident garbage out.

That last point matters more than founders expect. Your LLM prompt can be perfect, your TTS voice can sound gorgeous, but if the STT hears "Karen" as "Aaron" or drops the last four digits of an account number, your agent is now confidently wrong. Accuracy at the input is the foundation everything else sits on. If you want the deeper version of this argument, we wrote about what actually makes a good voice agent - the short version is that the listening layer is where good agents are won or lost.

A voice agent API bundles that whole pipeline - plus turn detection, interruption handling, and voice activity detection - so you don't have to hand-build the seams. That's the category. The question is how you get one.

The three paths for a startup

There are basically three ways to ship your first voice product. Each has a real trade-off, and the "right" one depends less on your budget than on how much of the plumbing you want to own.

Path	What you own	Trade-off	Best for
DIY multi-vendor	Separate STT, LLM, and TTS providers wired together yourself	Maximum control, but three vendors, three invoices, and you build turn detection, barge-in, and streaming orchestration	Teams whose differentiator is the pipeline itself
Platform (Retell, Vapi, etc.)	A managed environment, often no-code, with built-in telephony	Fast to launch, but opinionated conversation design, awkward custom integrations, and a real ceiling	No-code builders and telephony-first products
One API	Your system prompt, tools, and conversation logic — over a single connection	More control than a platform, far less overhead than DIY; you write code	Code-forward startups shipping a first voice product

Path 1: DIY multi-vendor. You pick an STT provider, an LLM provider, and a TTS provider, and you wire them together yourself. Maximum control. But you're now managing three vendors, three invoices, three sets of rate limits, and three places to debug when a call goes sideways. You also own the hard middle - turn detection, barge-in, streaming orchestration. For a first product, that's a lot of undifferentiated engineering before you've validated that anyone wants the thing.

Path 2: A platform (Retell, Vapi, and similar). These give you a managed environment, often with a no-code builder and built-in telephony. Great for getting something live fast without much code. The catch is the ceiling. Platforms are opinionated about conversation design, custom integrations can get awkward, and - candidly - a lot of platform-built agents end up sounding the same because they're built on the same rails. Vapi in particular is orchestration middleware, which means your agent is only as good as the weakest third-party link in its chain. We wrote more about where voice agent stacks start showing their limits if you want the honest version.

Path 3: One API. A single API that handles STT to LLM to TTS through one connection. You still write your own system prompt, define your own tools, and control the conversation - but you don't manage three vendors or build the streaming plumbing. It's the middle path: more control than a platform, far less overhead than DIY.

For a first voice product, path 3 is usually the sweet spot. You get to spend your time on product logic - the thing that makes your agent yours - instead of on infrastructure nobody will ever thank you for.

What to actually evaluate

Before you pick, here's the short list of things that genuinely move the needle for a first build. Ignore the spec-sheet noise; these five are what you'll feel in production.

Criterion	Why it matters	AssemblyAI Voice Agent API
Accuracy	Wrong transcription means the LLM answers the wrong thing — names, numbers, and addresses break flows	Built on Universal-3.5 Pro Realtime: 6.99% pooled WER and 15.31% entity error rate on Pipecat's open STT benchmark
Latency	End-to-end round-trip is what makes a conversation feel human	Around one second end-to-end
Developer experience	Time from "read the docs" to "talking to an agent" is the whole game for a first build	Standard JSON over one WebSocket, no SDK required; readable in ~10 minutes and works natively with Claude Code
Pricing	Predictable pricing lets you model unit economics before the product exists	Flat $4.50/hr billed by the minute — STT, LLM, and TTS included; roughly 4x cheaper than OpenAI's Realtime API (~$18/hr)
Lock-in	If you outgrow the tool, how painful is the exit?	A raw API over a standard WebSocket — you keep full control of conversation design, tools, VAD, and turn timing

Speech accuracy. This is the garbage-in problem from earlier. Look at word error rate on real conversational audio, not clean read-aloud benchmarks, and pay attention to entity accuracy - names, numbers, addresses - because that's what breaks support and booking flows. If you want to go deeper on why the usual metric is shaky, word error rate is broken is a good read.

Latency. End-to-end round-trip time is what makes a conversation feel human or robotic. You want something around a second. Don't obsess over shaving milliseconds off one stage; obsess over the full loop.

Developer experience and time-to-ship. How long from "read the docs" to "talking to a working agent"? For a first-timer this is the whole ballgame. SDK sprawl, dozens of event types, and framework lock-in all add days. A clean JSON API you can read in ten minutes does not.

Pricing model. Is it flat and predictable, or are you doing token math across three invoices plus per-minute STT plus TTS characters? Unpredictable pricing is its own kind of tax when you're trying to model unit economics for a product that doesn't exist yet.

Lock-in. If you outgrow the tool, how painful is the exit? No-code platforms can be sticky in ways that hurt later. An API you call over a standard WebSocket is a lot easier to walk away from.

How AssemblyAI's Voice Agent API maps to that list

Full disclosure: this is our product. I'm going to be specific so you can check the claims, not just take my word.

The Voice Agent API is one WebSocket API that runs the entire STT to LLM to TTS pipeline. You connect to a socket, stream audio in, and get audio back. You write the system prompt. That's the mental model.

On the five criteria:

Accuracy. It's built on Universal-3.5 Pro Realtime, our flagship streaming model, which posts a 6.99% pooled word error rate on Pipecat's open STT benchmark of real agent conversations. For comparison on that same benchmark, entity error rate lands at 15.31% - names, places, phone numbers, the stuff that actually breaks agents. It also takes your agent's question as context, so a mumbled one-word reply or a spelled-out account ID resolves correctly instead of getting mangled.

Latency. Around one second end-to-end. Fast enough to feel like a conversation, not a transaction.

Developer experience. Standard JSON over a WebSocket. No SDK required, no framework to learn. You can read the full API reference in about ten minutes, and most developers have something working the same afternoon. It also works natively with Claude Code - copy the docs, paste them in, and build - which is exactly why we designed it for coding agents.

Here's roughly what connecting looks like - one socket, JSON messages, no ceremony:

const ws = new WebSocket("wss://agents.assemblyai.com/v1/ws", {
  headers: { Authorization: `Bearer ${process.env.ASSEMBLYAI_API_KEY}` }
});

ws.on("open", () => {
  ws.send(JSON.stringify({ type: "session.update", session: {
    system_prompt: "You are a friendly booking assistant for a dental clinic.",
    // stream your audio frames in from here
  } }));
});

You can also update the system prompt, tools, and settings mid-conversation without reconnecting - handy when the flow branches.

Pricing. Flat $4.50/hr, billed by the minute, with STT, LLM, and TTS all included. One number, one bill, one set of logs. No reconciling three invoices. For context, that's roughly 4x cheaper than OpenAI's Realtime API (around $18/hr) and comes with a cleaner developer experience - their API has 30-plus event types to reason about.

Lock-in. It's a raw API over a standard WebSocket. We call it "invisible infrastructure" for a reason - your customers should feel like you built the thing from scratch, and you keep full control over conversation design, tools, VAD, and turn timing. We're not the agent. We're what you build the agent on.

It currently supports six languages - English, Spanish, French, German, Italian, and Portuguese - and ships drop-in plugins for LiveKit and Pipecat if you're already using those. When you're ready to go from "hello world" to something real, how to build with the Voice Agent API walks the full pipeline.

When a platform is the better call

I said this would be an honest guide, so here's the honest part. A one-API approach is not always right, and sometimes a platform genuinely is the better choice.

Reach for a platform like Retell or Vapi if:

You want no-code. If you're a non-technical founder or you need business users to build and tweak agents without touching a WebSocket, a visual builder is worth a lot. Our API assumes you can write code.
You need built-in telephony today. If your product is fundamentally about placing and receiving phone calls and you want that handled out of the box right now, platforms have mature telephony baked in. We're shipping telephony as a fast-follow, but "today" matters when you're shipping.
Call routing and hosted orchestration are the product. If the routing, queueing, and call-center-style workflow is your value, a platform gives you that scaffolding for free.

There's no shame in starting on a platform to validate demand and moving to an API when you hit the ceiling. Just go in knowing where that ceiling is.

And if the real question is DIY versus one API? Unless stitching STT, LLM, and TTS together is itself your differentiator, one API replaces three vendors, three invoices, and three debugging surfaces. For a first product, that's almost always the right trade.

Quickstart and a live demo to close

Here's the thing about voice products: reading about them tells you very little. Talking to one tells you everything. Accuracy, latency, turn-taking, the feel of an interruption landing cleanly - you catch all of it in about thirty seconds of conversation.

So the move is simple. Grab a key, wire up the socket, point it at a system prompt, and talk to it. If you can talk to it and it feels right, you're most of the way to a demo.

The best voice agent API for your first product is the one that gets you to that working demo fastest without boxing you in later. For most code-forward startups, that's a single API with strong accuracy at the input, predictable pricing, and an exit that isn't a trap.

Frequently asked questions

What's the best voice agent API for a startup building its first voice product?

The best one is whatever gets you to a working demo fastest without locking you in. For code-forward startups, that usually means a single API that bundles STT, LLM, and TTS over one connection - so you write product logic instead of managing three vendors. AssemblyAI's Voice Agent API fits that profile: one WebSocket, standard JSON, flat pricing, and around one-second end-to-end latency.

What's the cheapest voice agent API?

Watch out for pricing that looks cheap per stage but adds up across three invoices (STT per minute, LLM per token, TTS per character). A flat rate is easier to model. AssemblyAI's Voice Agent API is $4.50/hr all-in - STT, LLM, and TTS included - which is roughly 4x cheaper than OpenAI's Realtime API at around $18/hr. Cheapest on paper isn't always cheapest in practice; predictability matters when you're modeling unit economics.

One API or a platform like Retell or Vapi - which should I choose?

Choose a platform if you want no-code building, need telephony out of the box today, or if hosted call routing is your actual product. Choose a single API if you want full control over conversation design and integrations without hitting a platform ceiling, and you're comfortable writing code. Platforms trade complexity for constraints; an API keeps you in control but expects you to build.

How good does the developer experience really need to be?

For a first voice product, it's the whole game. The gap between an API you can read in ten minutes and one with dozens of event types and required SDKs is measured in days of your life. AssemblyAI's Voice Agent API uses standard JSON over a WebSocket with no SDK required, works natively with Claude Code, and lets you update prompts and tools mid-conversation without reconnecting.

Which languages does the Voice Agent API support?

Six today: English, Spanish, French, German, Italian, and Portuguese. If you need broader coverage for pre-recorded workflows, Universal-3.5 Pro handles native code-switching across 18 languages on the async side.

How do I get started?

Open the API reference, connect to the WebSocket, and stream audio at a system prompt. Most developers have a working agent the same afternoon.

Speech-to-Text API Fundamentals: Auth, Polling & JSON

Mart Schweiger — Wed, 22 Jul 2026 14:54:50 +0000

Every integration with a speech-to-text API comes down to three questions. How do I authenticate? How do I get results out of an async job? And what does the response actually contain? Get those right and everything else — speaker labels, PII redaction, language detection — is just another request option.

This is the canonical walkthrough. We'll go end-to-end: your first authenticated request, the status lifecycle of a transcription job, a correct polling loop, and a field-by-field tour of the JSON that comes back. Code in curl, Python, and JavaScript throughout.

If you're new to the space, What is speech-to-text? and What is Automatic Speech Recognition (ASR)? give you the conceptual grounding. Here, we're shipping a first successful call.

Overview

AssemblyAI's REST API lives at https://api.assemblyai.com. You submit audio, the API transcribes it asynchronously, and you read the result as JSON. There's no streaming to manage and no SDK required for the core flow — though we ship official SDKs that make it a two-liner.

The whole loop:

Authenticate — send your API key in the authorization header.
Submit — POST /v2/transcript with an audio_url. You get back an id and a status.
Poll — GET /v2/transcript/{id} every few seconds until status is completed or error.
Parse — read text, confidence, words[], and friends from the JSON.

Let's build it.

Prerequisites

An AssemblyAI API key. Get your free API key from the dashboard.
A publicly reachable audio file URL (or one you upload — that's a separate endpoint; here we'll use a URL).
Optionally, one of the official SDKs:
Python: pip install assemblyai
JavaScript/TypeScript: npm install assemblyai

Set your key as an environment variable so it never lands in source control:

export ASSEMBLYAI_API_KEY="your_api_key_here"

Every example below reads the key from ASSEMBLYAI_API_KEY.

Quick start

The SDKs collapse submit-and-poll into a single call. Here's the entire flow in each.

import os, assemblyai as aai

aai.settings.api_key = os.environ["ASSEMBLYAI_API_KEY"]

transcript = aai.Transcriber().transcribe(
    "https://assembly.ai/wildfires.mp3"
)

print(transcript.text)

import { AssemblyAI } from "assemblyai";

const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });

const transcript = await client.transcripts.transcribe({
  audio: "https://assembly.ai/wildfires.mp3",
});

console.log(transcript.text);

That's the destination. Now let's take apart what's happening underneath, because understanding it is what lets you debug, add features, and use webhooks later.

1. Authentication

AssemblyAI authenticates with a single header on every request:

authorization: <YOUR_API_KEY>

Two things trip people up here, so let's be explicit. There's no Bearer prefix — the header value is your raw key, nothing else. And the header goes on every request, both the POST that submits a job and the GETs that poll it.

Here's a first authenticated request that lists your account's transcripts:

curl https://api.assemblyai.com/v2/transcript \
  --header "authorization: $ASSEMBLYAI_API_KEY"

If your key is valid, you get a 200 and a JSON body. If it's missing or wrong, you get a 401. That's your authentication smoke test before you send any audio.

Never hardcode the key. Read it from the environment, a secrets manager, or your platform's config store.

2. Submitting a job and the status lifecycle

Transcription is asynchronous. You hand the API an audio URL, it returns immediately with a job id, and the work happens in the background. You don't get the transcript in the POST response — you get a receipt.

Submit with POST /v2/transcript:

curl https://api.assemblyai.com/v2/transcript \
  --header "authorization: $ASSEMBLYAI_API_KEY" \
  --header "content-type: application/json" \
  --data '{"audio_url": "https://assembly.ai/wildfires.mp3"}'

The response is a transcript object. The two fields that matter right now:

{
  "id": "106993b6-...",
  "status": "queued"
}

Persist that id. It's how you retrieve the result, and it's how the transcript is addressed for the rest of its life.

The status lifecycle

A job moves through a small, fixed set of status values. These are the exact strings the API returns:

queued — waiting to start. In practice you'll typically only see this when you're being rate-limited; otherwise jobs go straight to processing.
processing — the model is transcribing the audio.
completed — success. The text field and all the other output fields are now populated.
error — the job failed. The object carries an error string explaining why.

completed and error are the two terminal states. Your job is to poll until you hit one of them. Everything about handling a transcription job is really just handling this state machine correctly.

3. Polling correctly

Since the result isn't in the POST response, you fetch it with GET /v2/transcript/{id} and check the status. Loop, sleep, repeat until terminal.

The convention: poll every 3 seconds, break on completed (read text), break on error (read error). Three seconds is a sane interval — frequent enough to feel responsive, relaxed enough that you're not hammering the endpoint.

curl

TRANSCRIPT_ID="106993b6-..."

while true; do
  response=$(curl --silent https://api.assemblyai.com/v2/transcript/$TRANSCRIPT_ID \
    --header "authorization: $ASSEMBLYAI_API_KEY")
  status=$(echo "$response" | jq -r '.status')

  if [ "$status" = "completed" ]; then
    echo "$response" | jq -r '.text'
    break
  elif [ "$status" = "error" ]; then
    echo "$response" | jq -r '.error'
    break
  fi

  sleep 3
done

Python

import os
import time
import requests

API_KEY = os.environ["ASSEMBLYAI_API_KEY"]
BASE_URL = "https://api.assemblyai.com"
headers = {"authorization": API_KEY}

# Submit the job
resp = requests.post(
    f"{BASE_URL}/v2/transcript",
    json={"audio_url": "https://assembly.ai/wildfires.mp3"},
    headers=headers,
)
resp.raise_for_status()
transcript_id = resp.json()["id"]
print(f"Submitted: {transcript_id}")

# Poll until terminal
polling_url = f"{BASE_URL}/v2/transcript/{transcript_id}"
while True:
    transcript = requests.get(polling_url, headers=headers).json()
    status = transcript["status"]

    if status == "completed":
        print(transcript["text"])
        break
    elif status == "error":
        raise RuntimeError(f"Transcription failed: {transcript['error']}")

    time.sleep(3)

JavaScript

const API_KEY = process.env.ASSEMBLYAI_API_KEY;
const BASE_URL = "https://api.assemblyai.com";
const headers = { authorization: API_KEY };

// Submit the job
const submit = await fetch(`${BASE_URL}/v2/transcript`, {
  method: "POST",
  headers: { ...headers, "content-type": "application/json" },
  body: JSON.stringify({ audio_url: "https://assembly.ai/wildfires.mp3" }),
});
const { id } = await submit.json();
console.log(`Submitted: ${id}`);

// Poll until terminal
const pollingUrl = `${BASE_URL}/v2/transcript/${id}`;
while (true) {
  const transcript = await (await fetch(pollingUrl, { headers })).json();

  if (transcript.status === "completed") {
    console.log(transcript.text);
    break;
  } else if (transcript.status === "error") {
    throw new Error(`Transcription failed: ${transcript.error}`);
  }

  await new Promise((r) => setTimeout(r, 3000));
}

Three implementations, one pattern: submit, grab the id, poll on a 3-second interval, branch on the terminal states. When you're ready to skip the loop entirely, webhooks push the result to you instead — more on that below.

For the full endpoint contracts, see Transcribe an audio file and Check transcript status.

4. Parsing the JSON response

Once status is completed, the transcript object is fully populated. Here's the canonical shape, trimmed to the essentials:

{
  "id": "106993b6-...",
  "status": "completed",
  "text": "Smoke from hundreds of wildfires...",
  "language_code": "en",
  "audio_duration": 282,
  "confidence": 0.95,
  "words": [
    { "text": "Smoke", "start": 100, "end": 640, "confidence": 0.9, "speaker": null }
  ]
}

Let's walk the fields you'll actually use.

id — the job identifier. Persist it; you can GET the transcript again anytime.
status — where the job is in its lifecycle. On a completed job, completed.
text — the full transcript as one string. null until the job completes, so always gate on status first.
confidence — an overall confidence score as a float from 0.0 to 1.0. In the example, 0.95.
audio_duration — the length of the audio in seconds (an integer). 282 here means the file is 282 seconds long. This is what you meter usage against.
language_code — the detected or specified language, e.g. "en". When you use language detection, you'll also get a language_confidence between 0 and 1.
words[] — every word as its own object: text, start, end, confidence, and speaker. The start and end timestamps are in milliseconds. So start: 100, end: 640 means that word runs from 0.1s to 0.64s. This is what you build captions, search, and clip-to-timestamp features on.

Watch the units — this is the single most common parsing bug. audio_duration is in seconds; word timestamps are in milliseconds. Mix them up and your captions drift by three orders of magnitude.

Utterances and speakers

When you enable speaker_labels or multichannel, the response gains an utterances[] array. Each utterance groups contiguous speech by a single speaker:

{
  "utterances": [
    {
      "speaker": "A",
      "text": "Smoke from hundreds of wildfires...",
      "confidence": 0.94,
      "start": 100,
      "end": 4200,
      "words": [ { "text": "Smoke", "start": 100, "end": 640, "confidence": 0.9, "speaker": "A" } ]
    }
  ]
}

Same millisecond timestamps, plus a speaker label and a nested words[]. If you didn't request speaker labels or multichannel, utterances won't be present — so check before you iterate.

5. Error handling

Two failure modes, two different responses. Handle them differently.

HTTP-level errors show up as status codes on the request itself. A 400 means the request was malformed — bad JSON, a missing required field, an invalid option. Fix the request; retrying the same payload won't help. A 401 means your authorization header is missing or wrong. Server-side (5xx) errors can be transient — those are safe to resubmit.

Job-level errors show up as status: "error" on a job you already submitted successfully. The request was accepted (200), but transcription failed. Read the error field for the reason:

{
  "id": "106993b6-...",
  "status": "error",
  "error": "Download error, unable to download the audio file"
}

Common causes of an error status: an unsupported audio format, missing audio, or a URL the API couldn't reach. These are usually input problems — check that your audio_url is public and points at valid audio before resubmitting.

A resilient loop distinguishes the two:

if status == "error":
    # Job-level failure — inspect .error, likely a bad input
    raise RuntimeError(transcript["error"])

Wrap the POST in retry-with-backoff for 5xxs, and never blind-retry a 400.

Next steps

You've got the fundamentals. The same submit-poll-parse flow carries every feature — you just add request options to the POST body:

speaker_labels — who spoke when, via the utterances[] array.
language_detection and language_code — auto-detect or pin a language.
punctuate and format_text — control formatting of the output text.
multichannel — transcribe each audio channel separately.
redact_pii and filter_profanity — clean the transcript for compliance.
webhook_url — skip polling entirely.

That last one is the natural upgrade. Instead of a GET loop, set webhook_url on submission and AssemblyAI POSTs the finished result to your endpoint when the job hits a terminal state. It's the right pattern for production and for high volume. The API reference covers the webhook payload and every option in detail.

From here, explore the full speech-to-text product and the deep dive on how accurate speech-to-text is in 2026.

View the full API reference → assemblyai.com/docs/api-reference/overview

Frequently asked questions

How do I authenticate with the AssemblyAI API?

Send your API key in the authorization header on every request — authorization: . There's no Bearer prefix; the value is your raw key. Store it in an environment variable like ASSEMBLYAI_API_KEY and get your key from the dashboard.

Do I poll for results or get them back immediately?

Transcription is asynchronous, so you poll. POST /v2/transcript returns an id and a status immediately, but not the transcript. Fetch the result with GET /v2/transcript/{id} every 3 seconds until status is completed or error. To avoid polling entirely, set webhook_url on submission and AssemblyAI pushes the result to you.

What do the transcript status values mean?

There are four: queued (waiting — usually only when rate-limited), processing (actively transcribing), completed (success, read text), and error (failed, read the error string). completed and error are the terminal states you poll toward.

How do I read confidence scores and word timestamps?

confidence is an overall float from 0.0 to 1.0. Each object in words[] has its own confidence plus start and end timestamps in milliseconds. Note that audio_duration is in seconds while word timestamps are in milliseconds — don't mix the units.

Why is my transcript status error?

The job was accepted but transcription failed. Read the error field for the reason. Common causes are an unsupported audio format, missing audio, or a URL the API couldn't reach. That's different from a 400, which means the request itself was malformed and shouldn't be blindly retried; server (5xx) errors, by contrast, can be resubmitted.

What can I do beyond a basic transcript?

Add request options to the POST body — speaker_labels, language_detection, punctuate, format_text, multichannel, redact_pii, filter_profanity, and webhook_url. The submit-poll-parse flow stays identical. See Submit a transcript and Get a transcript for full field references.

Open Banking APIs Explained: What PSD2 Compliance Actually Requires From Your Engineering Team

Lycore Development — Wed, 22 Jul 2026 14:49:22 +0000

Open Banking APIs Explained: What PSD2 Compliance Actually Requires From Your Engineering Team

Every fintech founder we talk to eventually says some version of the same sentence: "we just need to connect to the bank's API." It sounds like a single integration task on a roadmap. In practice, it's a compliance program with an API attached, and the gap between those two framings is where a lot of engineering timelines quietly blow up.

We've built open banking integrations across payment initiation, account aggregation, and lending products in the UK and EU markets, plus US equivalents through providers like Plaid. This post is the technical walkthrough we wish more teams had before they scoped their first sprint — what PSD2 actually mandates at the API and architecture level, where the real engineering effort lives, and where teams consistently underestimate the work.

What PSD2 actually is, in engineering terms

The Second Payment Services Directive is an EU regulation, not a technical spec. It mandates that banks (Account Servicing Payment Service Providers, or ASPSPs) provide regulated third parties with API access to customer account data and payment initiation, provided the customer has consented. The regulation itself doesn't dictate exact API shapes — that's what the technical standards built on top of it, primarily the Berlin Group's NextGenPSD2 framework and the UK's Open Banking Standard, are for.

For an engineering team, PSD2 compliance breaks down into three functional API categories you'll be building against or exposing:

AISP (Account Information Service Provider) access — read-only access to account balances, transaction history, and account holder details, with explicit customer consent that expires and must be renewed (typically every 90 days under RTS rules).

PISP (Payment Initiation Service Provider) access — the ability to initiate a payment directly from the customer's bank account, bypassing card networks entirely. This is the piece with the real product upside (lower interchange costs, direct settlement) and the real engineering complexity (strong customer authentication flows, payment status polling, handling partial failures mid-flow).

CBPII (Card-Based Payment Instrument Issuer) access — confirmation of funds availability for card-based instruments, a narrower and less commonly implemented category unless you're specifically in the card issuing space.

If you're building a product that needs bank data or payment initiation, you're consuming these APIs from banks (or from an aggregator abstracting many banks' APIs). If you're a bank or e-money institution being asked to expose these APIs to third parties, you're implementing the provider side. The engineering shape of these problems is almost entirely different, so the first architectural decision is figuring out honestly which side of that line your product sits on.

Strong Customer Authentication is where most timelines go wrong

The single most underestimated piece of PSD2 work is Strong Customer Authentication (SCA). The regulation requires two-factor authentication for most account access and payment initiation — something the user knows (a PIN, password), something they have (a device, a hardware token), and something they are (biometrics) — with at least two of the three categories represented.

In practice, SCA in open banking means redirect-based or decoupled authentication flows where your app hands control to the bank's own authentication interface, the user authenticates directly with their bank (not with you — you never see their banking credentials, which is itself a hard compliance requirement), and control returns to your app via a redirect or a webhook-style callback.

1. Your app calls the bank's consent endpoint, specifying scope (balances, transactions, payments) and redirect URI
2. Bank returns an authorization URL
3. User is redirected to the bank's own login page (outside your app's control entirely)
4. User authenticates with the bank + completes SCA challenge (bank's own 2FA)
5. Bank redirects back to your redirect URI with an authorization code
6. Your backend exchanges the code for an access token + refresh token
7. Subsequent API calls use the access token until it expires or is revoked

This sounds straightforward until you build it against a second bank and discover their redirect flow has different timeout behavior, a different set of error codes for user-cancelled consent, and a different token refresh window. There is no single PSD2 API — there is a shared regulatory framework implemented slightly differently by every bank, and your integration layer has to absorb that variance. This is the actual engineering problem, and it's also exactly the layer where aggregators like Plaid, TrueLayer, or Tink earn their fee: they've already absorbed hundreds of these implementation differences so you don't have to build and maintain adapters for each bank individually.

We generally advise clients to run the numbers honestly on build-vs-aggregate here. Direct bank integration removes a middleman fee and gives you more control, but it means owning an ongoing maintenance burden every time a bank updates its API version, changes its SCA flow, or has downtime — and banks are not known for graceful API versioning. Unless bank integration volume is genuinely core to your product's moat, an aggregator is very often the right call, at least for an initial launch, with a migration path to direct integrations for your highest-volume banks once you have the transaction volume to justify the engineering investment.

Consent management is a data model problem before it's a UI problem

A recurring mistake we see in early-stage fintech codebases is treating consent as a boolean flag — "has_bank_consent: true/false" — rather than as a first-class, time-bound, scope-specific record. PSD2 consent has real structure that your data model needs to reflect:

CREATE TABLE bank_consents (
    id UUID PRIMARY KEY,
    user_id UUID NOT NULL REFERENCES users(id),
    institution_id VARCHAR NOT NULL,
    scopes TEXT[] NOT NULL,  -- e.g. ['balances', 'transactions', 'payments']
    access_token_encrypted BYTEA NOT NULL,
    refresh_token_encrypted BYTEA NOT NULL,
    granted_at TIMESTAMPTZ NOT NULL,
    expires_at TIMESTAMPTZ NOT NULL,
    revoked_at TIMESTAMPTZ,
    consent_reference VARCHAR NOT NULL  -- the bank's own consent ID, needed for status checks
);

The expiry field isn't decorative. RTS technical standards require re-authentication at least every 90 days even for an ostensibly "always on" data connection, and your product needs a real UX plan for what happens when a consent lapses mid-session — silently failing background jobs that pull transaction data is a bad default, but so is interrupting a user mid-task with a re-auth flow they weren't expecting. We generally build a proactive re-consent nudge that fires a week before expiry, with fallback handling for consents that lapse anyway.

Revocation needs equal attention. Users can revoke consent directly with their bank, entirely outside your product's UI, and your system needs to detect and handle that gracefully rather than continuing to hammer a dead token until it starts generating 401s in a retry loop. Most aggregators surface a webhook for this; if you're integrating directly with a bank, check whether their API even offers a revocation notification or whether you're stuck polling token validity — this detail alone has changed which banks we recommend integrating with directly on more than one project.

Payment initiation needs an honest state machine

Account data access is comparatively forgiving — a failed read can be retried. Payment initiation cannot, and this is where we've seen the most production incidents in fintech codebases we've inherited. A payment initiation request has more terminal and non-terminal states than most teams initially model:

PENDING → AUTHORIZED → ACCEPTED_SETTLEMENT_IN_PROCESS → ACCEPTED_SETTLEMENT_COMPLETED
                    ↘ REJECTED
PENDING → CANCELLED (user abandoned SCA)
PENDING → EXPIRED (SCA window timed out)

The trap is treating "the API call returned 200" as "the payment happened." A 200 on payment initiation typically means the request was accepted for processing, not that funds moved — actual settlement confirmation comes through a separate status check or webhook, sometimes minutes later, occasionally longer during bank-side processing delays. We've inherited codebases that displayed "Payment Successful" to the user the moment the initiation call returned, only to discover — via a spike in support tickets — that a meaningful percentage of those payments were later rejected by the receiving bank for reasons the initiating API call couldn't have known about.

The fix is building your product UX and your backend state machine around the real terminal states, not the optimistic ones: show "Payment Processing" rather than "Payment Successful" until you have actual settlement confirmation, build idempotent status-check polling with backoff, and — this is the part teams skip under deadline pressure — decide up front what your product does when a payment sits in an ambiguous state for longer than expected. Silent failure is worse than an honest "we're still confirming this" message.

Security requirements that go beyond "use HTTPS"

PSD2's RTS on Strong Customer Authentication and Common Secure Communication includes specific technical requirements that go well past generic API security hygiene:

Qualified certificates. Communication with ASPSPs typically requires eIDAS-qualified certificates (QWAC for website authentication, QSEAL for request signing) issued by an accredited Qualified Trust Service Provider — not a standard TLS certificate from a normal CA. Budgeting time and cost for certificate acquisition is a compliance line item, not an engineering afterthought, and the acquisition process itself can take weeks.

Request signing. Many bank APIs require signed requests (JWS) in addition to TLS, meaning your integration layer needs to build and attach a digital signature to outgoing requests using your QSEAL certificate — an extra step most standard HTTP client libraries don't handle out of the box, and one that needs its own test coverage since a malformed signature fails silently as an auth error that looks identical to a token problem.

Token encryption at rest. Access and refresh tokens for bank accounts are, functionally, keys to a customer's financial data. We treat them the way we'd treat any other high-sensitivity secret — encrypted at rest with envelope encryption, never logged (a mistake we've caught in code review more than once, usually in a debug log statement someone forgot to remove), and access-scoped so that a compromised application server doesn't hand over decryption keys alongside the encrypted tokens.

Fraud and anomaly monitoring. Regulated payment initiation providers have transaction monitoring obligations under PSD2's fraud reporting requirements. Depending on your license type (or your relationship with a licensed partner, if you're operating under someone else's e-money license via an agency model), you may need real-time anomaly detection on payment patterns, not just after-the-fact reporting — this is a genuine data engineering workstream, not a checkbox.

The licensing question that determines your whole architecture

Before any of the above, the foundational question is whether you're operating under your own PSD2 license (as an Authorised Payment Institution or Electronic Money Institution) or as an agent/distributor under someone else's license. This isn't primarily an engineering decision, but it determines your engineering scope enormously. Operating as an agent under an existing EMI's license (a common and often sensible path for early-stage fintechs) typically means integrating with their compliance and reporting APIs rather than building your own regulatory reporting pipeline from scratch — a substantial scope reduction that founders sometimes don't realize is available until well into a build.

We always push clients to nail down licensing strategy with actual regulatory counsel before committing to an architecture, because "should we build our own compliance reporting pipeline" is a multi-month engineering question whose answer depends entirely on a legal decision that has nothing to do with code.

Building an abstraction layer that survives multiple banks

Whether you're integrating directly with several banks or working through an aggregator that still exposes bank-specific quirks (aggregators smooth over authentication flows, not every data inconsistency), you need an internal abstraction layer that normalizes responses before they touch your product code. Without this, bank-specific field names, date formats, and transaction categorization conventions leak into your business logic, and every new bank integration becomes a scavenger hunt through conditional branches.

class NormalizedTransaction:
    def __init__(self, id, amount, currency, timestamp, description, category, raw_source):
        self.id = id
        self.amount = amount  # always minor units, always signed (negative = debit)
        self.currency = currency  # ISO 4217
        self.timestamp = timestamp  # always UTC
        self.description = description
        self.category = category  # normalized to your own taxonomy, not the bank's
        self.raw_source = raw_source  # keep the original payload for debugging/audit

class BankAdapter(ABC):
    @abstractmethod
    def fetch_transactions(self, account_id, since) -> list[NormalizedTransaction]:
        ...

    @abstractmethod
    def initiate_payment(self, payment_request) -> PaymentResult:
        ...

class BarclaysAdapter(BankAdapter):
    def fetch_transactions(self, account_id, since):
        raw = self._client.get_transactions(account_id, since)
        return [self._normalize(tx) for tx in raw['Data']['Transaction']]
    # bank-specific parsing lives here, never in product code

This pattern earns its cost the second bank you integrate, not the first — which is exactly why teams under deadline pressure skip it on bank one and then pay for the retrofit on bank three. We treat the adapter layer as non-negotiable scope on any open banking project regardless of how many banks are in the initial launch plan, because "we'll only ever support one bank" is rarely still true eighteen months later.

Sandbox testing has real limits — plan around them

Every major bank and aggregator provides a sandbox environment, and it's tempting to treat sandbox test coverage as equivalent to production readiness. It isn't. Sandbox environments consistently under-represent the messiness of production: real accounts with years of transaction history and inconsistent categorization, real SCA challenges with real user hesitation and abandonment, and real bank-side downtime during maintenance windows that sandboxes don't simulate.

We build a deliberate staged rollout for exactly this reason — a small cohort of real users on real bank connections before a full launch, specifically to surface the SCA abandonment rate (which is almost always higher than teams expect; redirecting a user out of your app to their bank's login and back is real friction, and a meaningful percentage of users don't complete the round trip) and the actual distribution of error codes your adapter layer needs to handle gracefully. Budgeting for this staged rollout as its own project phase, rather than treating "sandbox tests pass" as the finish line, has saved every fintech client we've worked with from a rockier production launch.

Observability for a system you don't fully control

The hardest part of operating an open banking integration long-term is that a meaningful chunk of your system's behavior is determined by infrastructure you don't own — the bank's API uptime, their SCA flow, their token expiry policy. Standard application monitoring (error rates, latency) still matters, but it needs to be paired with per-institution dashboards, because "our API error rate is elevated" is a much less actionable alert than "Barclays' consent endpoint has been returning elevated 503s for the last twenty minutes." We tag every metric and log line with the institution ID from day one, even when there's only one bank integrated, because retrofitting that tagging later means re-instrumenting a system that's already in production.

Alert thresholds also need to be bank-aware rather than global. A bank with a smaller user base on your platform will have a noisier error rate on low absolute volume — five failed calls out of ten looks alarming as a percentage but might be a single user's session, not a systemic issue. We set alerting on absolute counts as well as rates specifically to avoid page fatigue from a single small institution's normal variance.

What we tell teams starting this today

Open banking integration is genuinely buildable by a competent engineering team — this isn't a warning to avoid it. But it's a compliance-and-architecture problem wearing an API integration costume, and teams that scope it as "connect to the bank API" consistently blow their timelines by a factor of two or three once SCA edge cases, consent lifecycle management, and payment state machines surface mid-build.

The teams that ship this well start with the licensing and aggregator-vs-direct decisions before writing integration code, model consent and payment state as first-class, time-bound data rather than booleans, and build their UX around the actual terminal states banks return rather than the optimistic ones. Everything past that is solvable engineering work — it's just work that needs to be scoped honestly from day one, not discovered sprint by sprint.