Your Voice Agent Isn't Slow. It's Silent. Non-Blocking Tools, Explained.

In a chat UI, a 13 second database query gets a spinner and nobody complains. In a phone call, the same 13 seconds is dead air, and dead air sounds like a dropped call. People hang up.

Google's ADK team published a guide this week that puts a number on it. In their sales report benchmark on gemini-3.8-live, switching one tool from blocking to streaming cut the time to the agent's first words from 14.4 seconds to 1.2 seconds (Google Developers Blog, Oct 19 2026). The query still took 13 seconds. The caller just stopped noticing.

This post walks through why voice agents go silent, the two patterns that fix it, and the production details that bite you once real callers show up.

Why prompting doesn't fix it

The instinct is to add "tell the user you are working on it" to the system prompt. It doesn't work, and the reason is mechanical. A normal tool is a coroutine that returns once:

async def query_sales_database(region: str) -> dict:
    """Runs the quarterly sales report for a region."""
    report = await run_expensive_query(region)  # ~13 seconds
    return {"status": "success", "report": report}

In a live bidirectional session, the model's turn is waiting on that function response. It cannot speak the "one moment" line, because it has not been given control back yet. The fix has to be architectural: the tool must hand control back before the work is done.

Pattern 1: stream it when the wait has a story

If the job has real stages (connect, scan, aggregate, format), make the tool an async generator. In ADK, changing return to yield is enough. The framework marks the declaration as non-blocking and immediately sends the model an acknowledgement, so the agent can talk while the work runs:

import asyncio
from typing import AsyncGenerator

async def query_sales_database(region: str) -> AsyncGenerator[str, None]:
    """Runs the quarterly sales report. Call this ONCE; it pushes updates."""
    for message, delay in [
        ("Connecting to the sales warehouse", 1.5),
        ("Scanning order records", 3.5),
        ("Aggregating revenue by product line", 3.5),
        ("Formatting the report", 1.5),
    ]:
        yield f"PROGRESS: {message}"
        await asyncio.sleep(delay)
    yield f"DONE: {region} revenue is up 12.4% year over year."

Each yield arrives at the model as a partial function response, and the model paraphrases it ("Okay, I'm into the warehouse, pulling the order records now"). The caller hears progress instead of silence. If they ask an unrelated question mid-query, the agent answers it without cancelling the background task.

Pattern 2: background it when the wait is just a wait

Narrating every stage gets annoying when the user has already moved on. If there is nothing worth saying, yield exactly once, at the end:

async def query_sales_database(region: str) -> AsyncGenerator[dict, None]:
    """Runs the quarterly sales report. Call this ONCE; results arrive when ready."""
    yield await run_expensive_query(region)

Because it is still a generator, the model gets an immediate pending signal, says one line ("I'm pulling that together now"), and stays quiet until the result lands.

Deciding when the answer is spoken

Execution shape and delivery timing are separate knobs. ADK exposes the second one as response_scheduling on a FunctionTool:

Mode	What happens when the result arrives
WHEN_IDLE	Waits for the current turn to finish, then delivers at the next natural pause
INTERRUPT	Cuts into current speech immediately (save this for urgent results)
SILENT	Added to context only; the agent uses it if the topic comes up

from google.adk.tools import FunctionTool
from google.genai import types

sales_tool = FunctionTool(query_sales_database)
sales_tool.response_scheduling = types.FunctionResponseScheduling.WHEN_IDLE

In Google's benchmark, a report arriving under WHEN_IDLE waited politely while the model finished a separate 20 second answer, then slotted in. Under INTERRUPT it cut the model off mid sentence. Streaming tools can even choose per update: keep intermediate progress SILENT, deliver the final figure WHEN_IDLE.

The rule of thumb from the guide is worth memorising: if the wait has a story, stream it. If the wait is just a wait, background it.

The production details that bite

1. Duplicate calls. Because a streaming tool spans several turns, a model that hears the user repeat themselves may call it again. A docstring that says "Call this ONCE" helps, but a latch is what actually protects you, especially when the tool does something with side effects like booking a slot:

_inflight: dict[str, asyncio.Task] = {}

async def book_slot(customer_id: str, slot: str) -> AsyncGenerator[str, None]:
    key = f"{customer_id}:{slot}"
    if key in _inflight:
        yield "ALREADY_RUNNING: booking for this slot is in progress"
        return
    task = asyncio.create_task(_do_booking(customer_id, slot))
    _inflight[key] = task
    try:
        yield "PROGRESS: checking the calendar"
        yield f"DONE: {await task}"
    finally:
        _inflight.pop(key, None)

In a multi-instance deployment, move that key to Redis with a TTL, or pass it to the downstream API as an idempotency key.

2. Barge-in does not cancel tools. When the caller talks over the agent, the audio stops and you get an interrupted event, but running tools keep running and their results still arrive. If you want "actually, never mind" to stop the work, give the agent an explicit cancel tool. ADK has a built-in stop_streaming(function_name) that cancels the generator's task and raises CancelledError inside it, so put your connection cleanup in a finally.

3. Narration lags. The model paraphrases updates as they arrive, and narration can trail the tool by several seconds. Don't emit a progress line every 200 ms. Three or four meaningful stages beat twenty tiny ones.

4. Measure the right number. Backend latency did not change in any of this. What changed is time to first audio and the length of the longest silence in the call. Log both per call. If you only watch p95 tool latency, you will never see the problem your callers hang up on.

The same idea outside ADK

None of this is ADK specific. On any realtime stack (OpenAI Realtime compatible endpoints, Pipecat, LiveKit Agents), the pattern is the same: return an immediate acknowledgement to the model, run the work as a task, push results back into the session as they arrive, and decide whether each one should interrupt, wait or stay silent.

A web page that takes ten seconds feels slow. A voice agent that goes quiet for ten seconds feels broken. That gap is the whole reason this pattern exists.

We build voice agents and AI products end to end at Axionry, an AI product development studio. If you are planning one, our voice AI development work starts with exactly this kind of latency budget, and the voice AI cost calculator will tell you what a minute of it costs.