Your users don't wait. Here's why streaming matters more than model size.

#ai #api #llm

You can have the smartest model in the world, and if the first word takes eight seconds, your users will call it "broken." The difference between AI magic and AI feels slow is one flag.

from openai import OpenAI

client = OpenAI(
    api_key="mb-xxxxxxxx",
    base_url="https://aibridge-api.com/v1",
)

stream = client.chat.completions.create(
    model="deepseek-v4-pro",
    messages=[{"role": "user", "content": "Write a short story about a robot."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

That stream=True is the entire feature. Tokens start arriving almost immediately, instead of the whole answer landing at once after a blank screen.

The dead-air problem

Non-streaming LLM calls have a brutal UX curve: the user hits enter, and then… nothing. For a long generation, that silence can stretch 5–15 seconds while the model drafts the entire response in memory before returning anything.

What users do during that silence:

Think it's frozen and click again (double-spend, double-billing)
Think it's broken and abandon
Feel the weight of every second — perceived latency is always worse than measured latency

Streaming collapses that wait to time-to-first-token — usually well under a second. After that, words flow as they're generated, and the user watches progress instead of staring at a spinner. The total time is the same. The perceived time is night-and-day.

The UX numbers that actually matter

Metric	Non-streaming	Streaming
Time to first visible output	5–15s	< 1s
User perception	"It's thinking…"	"It's answering"
Double-submit risk	High (feels stuck)	Low (feels alive)
Perceived latency	Full generation time	Time to first token

None of this requires a faster model. It requires stream: true — and an endpoint that implements it correctly.

Streaming works on every model

Because AIBridge speaks the OpenAI wire format, streaming is first-class across all 15 models. Want it snappier? Pair streaming with a fast tier:

# Autocomplete / suggestions — sub-second feels instant
stream = client.chat.completions.create(
    model="glm-4-flash",          # fast tier
    messages=[{"role": "user", "content": "Continue: 'The quick brown fox'"}],
    stream=True,
)

Fast tier — glm-4-flash, deepseek-v4-flash, moonshot-v1-8k for keystroke-speed interactions
Balanced tier — glm-4-air, deepseek-chat, qwen-plus for everyday generation
Flagship tier — deepseek-v4-pro, qwen3-235b-a22b, glm-4-plus for quality that earns the wait
Deep reasoning — deepseek-reasoner, kimi-k3 for genuinely hard problems

Stream the fast models for real-time surfaces; stream the flagships so even long answers feel immediate.

The full stack, one endpoint

15 models · 4 vendors · one OpenAI-compatible key:

DeepSeek — deepseek-v4-pro, deepseek-v4-flash, deepseek-reasoner, deepseek-coder, deepseek-chat
Qwen — qwen3-235b-a22b, qwen-plus (131K), qwen-max
GLM — glm-4-plus, glm-4-air, glm-4-flash
Moonshot — kimi-k3 (1M context), moonshot-v1-128k / -32k / -8k