DEV Community: Preetham

The 300 milliseconds that decide whether your AI sounds human

Preetham — Sun, 26 Jul 2026 17:55:38 +0000

Two people having a conversation leave about 200 milliseconds between
turns. That gap is remarkably consistent — across languages, across cultures,
across phone lines. It is roughly a fifth of a second, and we are exquisitely
sensitive to it.

Push it to half a second and the other person sounds hesitant. Push it to a
full second and they sound distracted, or like they did not hear you. You will
start to repeat yourself.

I spent last week measuring that gap on an AI voice agent. The number came back
at 1,107 milliseconds.

Why "it feels laggy" is such a useless bug report

I have written that bug report. You probably have too. It is genuinely all you
can say, because the thing that went wrong is a feeling — and feelings do not
show up in dashboards.

Here is what makes it hard. When someone says a voice assistant feels slow,
there are at least four completely different things they might mean:

It took a long time to start answering.
It answered quickly but rambled.
It kept stopping mid-sentence and starting again.
It talked over them.

Those are four separate faults with four separate fixes, and they feel almost
identical to the person on the phone. Worse, in the systems we normally use to
watch software, they look identical too — the same numbers, none of them wrong,
none of them useful.

So I built something to tell them apart.

What we were measuring, and what we were missing

Almost every tool for watching AI systems assumes a shape: you ask a question,
you get an answer, and the tool measures the time in between. That is a
perfectly good model for a chatbot.

Voice does not work like that. Both people are on the line the whole time. You
can interrupt. It can interrupt you. There is no clean moment where your turn
ends and its turn begins — that boundary is something you both negotiate,
constantly, without thinking about it.

Which means the most important measurement — how long you sat in silence
waiting — is not a thing the existing tools even have a name for. It falls in
the gap between the things they measure.

So I drew it.

Blue is the person talking. Violet is the agent. The amber band in the middle
is the silence, with the milliseconds counting up while it lasts.

I did not expect this to be the useful part. It was meant to be a debugging
view. But watching that number climb while you sit there waiting communicates
something a percentile chart cannot: you feel the delay rather than reading
about it. Two seconds of silence is unbearable on screen in a way that "2000ms"
simply is not.

The interruption is the interesting signal

The measurement I did not anticipate finding useful was where people
interrupt.

Not how often — that number turns out to be nearly meaningless on its own.
Two agents can have identical interruption rates and completely opposite
problems:

If people cut in almost immediately — within a few hundred milliseconds — they are usually not interrupting at all. The agent is mistaking background noise for speech and stopping itself. A door closing. A cough. Someone else in the room.
If people cut in several seconds into the answer, they are genuinely interrupting, and it means the agent is talking too much. They got what they needed in the first sentence and are trying to move on.

Same count. Opposite causes. Opposite fixes. You only see the difference if you
record when inside the reply the interruption landed, and almost nothing
records that.

What "good" should actually mean

Numbers are only useful once you decide which ones are acceptable. So I wrote
down six, with a reason for each rather than a round number that felt nice:

What	Target	Why
Silence before replying	under 350 ms	Past this it reads as hesitation
Interruptions per call	under 0.8	Above this, people are fighting to be heard
Talking over the user	under 150 ms	Beyond this it feels rude, not eager
Calls resolved without a human	above 72%	The number the deployment is funded on
Turns where the user repeated themselves	under 9%	The closest thing to "did it work"
Escalations to a human	under 11%	The commercial ceiling

The fifth one is my favourite, and it took the longest to arrive at. How
often did the person have to say it again? Not the latency, not the token
count — whether the conversation actually worked. When someone says "no, I
meant the other one" or "can you repeat that", something failed a turn
earlier. It is the closest thing to measuring understanding that I could find
without asking anybody to fill in a survey.

The part where I was wrong

I built all of this against a simulator, because you cannot design dashboards
against three hand-held test calls. I tuned that simulator to what I believed
good performance looked like: replies starting in about 250 milliseconds.

Then I pointed the whole thing at a real Gemini Live agent.

1,107 milliseconds. Three to four times slower than the world I had spent a
day designing for.

The orange line is reality. The two flat lines below it are what I had assumed.

I could have quietly adjusted the simulator so the numbers agreed. I left it,
because that gap is the finding — and because an instrument that only ever
confirms what you already believed is not an instrument, it is decoration.

It also caught two genuine bugs in my own measuring code that thirty-five
automated tests had missed. Which is its own small lesson: the tests checked
that the code did what I told it to. Only real data checked whether what I told
it was right.

If you are building one of these

Three things I would tell myself a week ago.

Measure the silence, not the duration. How long the whole exchange took is
almost irrelevant. How long the person waited before hearing anything is
nearly everything. They are different numbers and only one of them predicts
whether people enjoy talking to your agent.

Record where interruptions happen, not just that they did. A count tells
you something is wrong. The distribution tells you what.

Point it at reality earlier than feels comfortable. I lost a day to
assumptions that a single real call would have corrected. The simulator was
useful — it just was not true.

None of this is really about AI. It is about the fact that conversation has
rules we all follow without noticing, and we notice instantly when a machine
breaks them. The 200 millisecond gap. Yielding when someone cuts in. Not
repeating what was already understood.

We have got very good at making these systems say the right words. Getting the
timing right is a separate problem, and right now most of us are not even
measuring it.

The tooling is open source, along with a short demo:
github.com/preethamresearch/cadence
· watch it

Built for the Agents of SigNoz hackathon. Written by me; I used an AI
assistant for the code and for editing this post, which is declared in the
submission.

SigNoz scanned 36,000 rows and told me "No Data"

Preetham — Sun, 26 Jul 2026 17:53:37 +0000

I spent an hour convinced my instrumentation was broken. It wasn't. The
telemetry was arriving, ClickHouse was reading it, and every dashboard panel
still said No Data — over a table that had 36,291 rows in it.

That was one of eight bugs I hit last week building OpenTelemetry
instrumentation for real-time voice agents. Not one of them threw an
exception. Not one showed up in a log. Six of them I found after the code
was already "working."

This post is the list, with the commands I used to find each one. If you are
instrumenting anything that streams — voice, video, an agent loop — you will
probably hit at least three of these.

What I was building, briefly

Voice agents like Gemini Live hold a persistent bidirectional socket.
Audio goes both ways at once. There is no request and no response, so the
OpenTelemetry GenAI conventions — which define chat spans around a
request/response pair — have nothing to attach to.

The number that actually matters for a voice agent is time to first audio:
how long the human sat in silence after they stopped talking. That interval
lives between spans, so no waterfall draws it.

So I built cadence: a state machine that reconstructs turn structure from the
raw signal stream and emits it as ordinary OTLP.

realtime.session                        one connected session
├── realtime.turn                       one exchange
│   ├── realtime.audio.user_utterance   VAD start → end
│   ├── chat                            [standard gen_ai.* attributes]
│   │   └── execute_tool
│   └── realtime.audio.agent_utterance  first audio out → done
│       └── (event) realtime.barge_in   offset_ms into the reply

That is a real Gemini session in SigNoz. Getting there took the eight bugs
below.

1. The collector was running, and doing nothing

I installed SigNoz with Foundry, which brings up the backend and its MCP
server in one step:

curl -fsSL https://signoz.io/foundry.sh | bash
foundryctl cast -f deploy/casting.yaml

Containers healthy. Port 4318 open. And the exporter kept logging
RemoteDisconnected. curl confirmed it — TCP connected, then the server hung
up without an HTTP response:

* Connected to localhost (127.0.0.1) port 4318
> POST /v1/traces HTTP/1.1
* Empty reply from server

The effective collector config explains it:

docker exec signoz-ingester-1 sh -lc \
  "awk '/^service:/{f=1} f' /var/tmp/collector-config.yaml"

service:
  pipelines:
    traces:
      exporters: [nop]
      receivers: [nop]

Every pipeline was nop. The config file on disk was correct — the running
collector had never received it. The reason was in the SigNoz server log:

failed to find or create agent ... "cannot create agent without orgId"

SigNoz will not hand a collector its configuration until an organisation
exists. I had not completed the first-run signup at localhost:8080. Until
you do, the OTLP port accepts connections and answers nothing.

Do the signup, restart the ingester, and the pipelines populate.

2. The exporter silently dropped 60% of my spans

I generated 1,201 turns. ClickHouse had 497.

SELECT name, count() FROM signoz_traces.distributed_signoz_index_v3
GROUP BY name ORDER BY count() DESC

No error surfaced. The OpenTelemetry BatchSpanProcessor defaults to a
2,048-span queue, and a realtime workload overruns it instantly — each turn
produces four to six spans, so a few hundred concurrent turns is enough. When
the queue is full, spans are dropped, and the SDK does not raise.

BatchSpanProcessor(
    OTLPSpanExporter(endpoint=f"{endpoint}/v1/traces", headers=headers),
    max_queue_size=32_768,        # default 2048 — far too small for streaming
    max_export_batch_size=1_024,
    schedule_delay_millis=2_000,
)

Re-ran it: 1,201 turns generated, 1,201 delivered.

3, 4, 5. Three ways to get timestamps wrong

I replay recorded sessions to populate dashboards, and replayed spans came out
0.06 ms long while their attributes claimed 400 ms. Spans were being stamped
at wall-clock "now" rather than at the time the event carried, so a session
replayed in one second produced microsecond spans.

The OpenTelemetry API accepts explicit timestamps; use them:

span = tracer.start_span(name, start_time=event.wall_ns, ...)
span.end(end_time=event.wall_ns)

Then every span landed near 1970 — because I passed time.monotonic() as
the wall clock. Monotonic is time since boot, not since epoch.

And the session span came out with a duration of several hours, because I
opened it eagerly at "now" and closed it in the replayed past. Negative
duration, wrapped unsigned.

The one worth stealing: compare two views of the same quantity. The span
duration and the duration_ms attribute should agree, so check:

SELECT attributes_string['realtime.prompt.version'] AS src,
       round(avg(durationNano)/1e6) AS span_ms,
       round(avg(attributes_number['realtime.turn.duration_ms'])) AS attr_ms
FROM signoz_traces.distributed_signoz_index_v3
WHERE name = 'realtime.turn' GROUP BY src

v16    4863    26135501
v17    5344    26095317

A 4.8-second span with a duration_ms attribute of 26,135,501 — seven
hours. Durations were computed from time.monotonic() while spans were stamped
from the event clock. In a live session the two coincide and this is invisible.

6. Cumulative vs delta: the "No Data" over 36,000 rows

This is the one from the title.

Metrics were arriving. signoz_metrics.distributed_samples_v4 had them. Every
percentile and rate query returned nothing, and the response was quietly
explicit about how much work it had done:

"meta": { "rowsScanned": 36291, "bytesScanned": 495851 },
"data": { "results": [ { "queryName": "A", "aggregations": null } ] }

Scanned 36,291 rows. Aggregated to null.

SigNoz expects delta temporality for counters and histograms. The
OpenTelemetry Python SDK defaults to cumulative. Nothing errors — the data is
stored, and every rate and percentile computes to nothing.

from opentelemetry.sdk.metrics import Counter, Histogram
from opentelemetry.sdk.metrics.export import AggregationTemporality

OTLPMetricExporter(
    endpoint=f"{endpoint}/v1/metrics",
    headers=headers,
    preferred_temporality={
        Counter: AggregationTemporality.DELTA,
        Histogram: AggregationTemporality.DELTA,
    },
)

One dictionary. Every panel populated.

Worth checking with the MCP server, which auto-fetches the metric's metadata
and tells you what it thinks it is holding:

metricType: histogram (auto-fetched)
temporality: delta (auto-fetched)

7 and 8. The two bugs only the real API found

Everything above was found against a simulator. So I pointed it at the actual
Gemini Live API — and my adapter, the one component that touches the vendor,
had never processed a real message.

It found two more.

Barge-in counted against finished turns. Real sessions reported 22 barge-in
events and 0% interrupted turns. Those numbers must agree. The flag marking
"the agent is speaking" was only cleared by a playback event my application
never emitted, so every turn after the first looked like an interruption — and
the barge-in was attributed to a span that had been exported seconds earlier.

The subtlety in the fix: clear that flag when a turn completes, but not
when it is interrupted. On an interruption the turn ends the instant the user
cuts in while the agent's audio keeps playing — and that lingering audio is
exactly the overlap being measured.

And the latency was nothing like I assumed.

Source	Turns	Mean TTFA	p95
simulated `v16`	416	246 ms	436 ms
simulated `v17`	428	328 ms	716 ms
real Gemini Live	77	1,107 ms	1,658 ms

Real Gemini Live is three to four times slower than the baseline I had spent a
day designing against. My 350 ms objective comes from turn-taking research and
I still think it is right for a spoken turn — but it is nowhere near
achievable on text-driven turns through this model today.

I left that in the README as found rather than quietly retuning the simulator
to match. Instruments that agree with your assumptions are not instruments.

The thing I actually took away

Every one of these eight failures was silent. The application reported
success throughout. That is the property you want in production — your voice
agent must not die because telemetry is unhappy — and it is precisely what
makes this miserable to debug.

So I wrote a preflight check that walks the chain and names the broken link:

library     ✓ cadence imports            v0.1.0, schema 0.1.0
            ✓ turn state machine         TTFA 400ms (expected 400ms)
signoz      ✓ SigNoz set up              v0.134.0
            ✓ collector pipelines        real receivers and exporters
transport   ✓ OTLP port reachable
            ✓ OTLP endpoint answers      HTTP 200
round trip  ✓ span exported end-to-end

Two details in there matter more than the rest. It checks TCP reachability
separately from "does it speak HTTP", because a nop pipeline accepts the
connection and closes it — that distinction is bug #1 and it cost me an hour.
And it drives a synthetic turn through the real state machine and asserts TTFA
is 400 ms, not 2,400 ms, because silently wrong telemetry is worse than an
outage: nothing breaks, you just make decisions on bad numbers.

If you take one thing from this: compare two views of the same quantity.
Span duration against the duration attribute. Barge-in events against
interrupted turns. That single habit found three of my eight bugs, and none of
them would have surfaced any other way.

Code, the `realtime.` semantic conventions, and importable SigNoz dashboards:
github.com/preethamresearch/cadence
· demo · full walkthrough*

Built for the Agents of SigNoz hackathon. Written by me; I used an AI
assistant for the code and for editing this post, which is declared in the
submission.

Building G-Axis: A Voice AI Companion + Browser Agent with Gemini Live API

Preetham — Mon, 16 Mar 2026 23:57:26 +0000

Your browser already works. You don't need a new app to experience AI.

That was the idea behind G-Axis — a Chrome extension I built for the Gemini Live Agent Challenge that turns your
existing browser into an AI-powered workspace.

No new tabs. No new logins. Just intelligence, right where you already work.

The Problem

Ever had 6 tabs open just to do one thing? ChatGPT here, Calendar there, Google Search somewhere else. Every AI tool
lives in its own silo. And they can all talk — but none of them can actually do anything in your browser.

What G-Axis Does

Two things:

Talk to it — Click the mic, pick one of 8 AI personas, and have a real conversation. Not text-to-speech. Real
bidirectional audio via Gemini's Live API. Ask it anything — it searches the web in real-time via Google Search
grounding.

Delegate to it — Type "Plan a 5-day Japan itinerary" and watch it research, navigate, and generate a full
document. Type "Schedule a meeting tomorrow at 10am" and it opens Calendar, fills the form, and saves.

The Gemini Stack

Here's what powers it under the hood:

Gemini Live API — The Voice Engine

This was the breakthrough. Gemini's native audio model (gemini-2.5-flash-native-audio) handles real-time voice
natively — no separate STT/TTS pipeline. The extension's service worker connects directly via WebSocket. Zero hops.
Zero latency.

I built 8 personas on top of it, each with a different Gemini voice and personality:

| Persona | Voice | Vibe |
|---------|-------|------|
| Friendly Buddy | Puck | Your go-to friend |
| Wise Mentor | Charon | Guidance, not lectures |
| Creative Partner | Aoede | Ideas machine |
| Job Interviewer | Kore | Practice makes perfect |
| Chill Companion | Fenrir | Just vibes |
| Professional Coach | Kore | Sharpen your edge |
| Friendly Debater | Charon | Challenge your thinking |
| Storyteller | Aoede | Bring ideas to life |

Switch mid-conversation. The voice changes. The personality changes. The previous session saves automatically.

Google Search — Real-Time Grounding

Ask "What's the latest AI news?" and Gemini doesn't guess from training data — it searches the web live and answers
with current information. This is the google_search tool baked into the Live API config.

Gemini 2.5 Flash — The Brain

Task planning. Function calling. Session analysis. Every voice conversation gets analyzed for 5 communication skills:

Confidence
Clarity
Engagement
Listening
Pacing

Users earn XP, level up, and track progress on a dashboard.

### Gemini Vision — The Eyes

For browser automation, screenshots are sent to Gemini Vision. It understands what's on screen — buttons, forms,
navigation — and decides where to click, type, and scroll.

The Hard Parts

Mic permissions in Chrome extensions — Sidepanels can't access getUserMedia. I tried 4 approaches before landing
on a minimal popup window with an AudioWorklet processor streaming PCM audio via Chrome ports.

Audio playback — My first approach used onended callbacks to chain audio buffers. This caused 5-20ms gaps
between every chunk — speech sounded choppy. The fix: schedule each AudioBufferSource to start at the exact
timestamp the previous one ends. Gapless.

Session timeouts — Gemini Live sessions die after ~10 minutes. I built transparent auto-reconnection (up to 20x)
so conversations can last over an hour without the user noticing.

Security — The API key accidentally got committed to the public repo. I scrubbed it from git history with
filter-branch, rotated the key, and moved to OAuth2 short-lived tokens. The key now lives in Cloud Secret Manager
and never touches client code.

Google Cloud Setup

Cloud Run → Backend hosting (FastAPI, 2 vCPU, 2GB, autoscale)
Secret Manager → API key → OAuth2 tokens (60-min expiry)
Cloud Build → Docker image CI/CD
Terraform → Full IaC (one file, all resources)

One command deploys everything:


bash
  ./deploy.sh gaxis-488323

  Architecture

  https://raw.githubusercontent.com/preethamtjit20-spec/gaxis/main/architecture-v3.png

  Try It

  The backend is live:

  curl https://gaxis-132388856648.us-central1.run.app/health
  # {"status":"ok","agent":true}

  Full source + setup instructions: https://github.com/preethamtjit20-spec/gaxis

  ---
  Built for the https://geminiliveagentchallenge.devpost.com/. Your browser already works — G-Axis makes it intelligent.

  ---