DEV Community: SAMEER SRIVASTAV

[Boost]

SAMEER SRIVASTAV — Tue, 18 Aug 2026 18:18:54 +0000

SAMEER SRIVASTAV

Aug 18

We killed the 900ms silence timer and made the phone stack agent-first

#ai #programming #python #opensource

6 min read

We killed the 900ms silence timer and made the phone stack agent-first

SAMEER SRIVASTAV — Tue, 18 Aug 2026 18:16:51 +0000

Human conversation does not wait 900ms after every pause.

Stivers et al. (PNAS, 2009) measured turn transitions across ten languages. The mode is 0–200ms. Levinson later pointed out the awkward implication: language production takes ~600ms to plan a word, so people are already encoding the reply while the other person is still talking. A voice agent that waits a fixed 0.9s after Deepgram says speech_final is not being careful. It is being obviously not-human.

That was us, six weeks ago. Fixed debounce after every pause. It felt polite in the lab and dead on a real phone. "What's your name?" sat in silence. "My number is…" got cut off. Same timer, opposite failure modes. Silence timers cannot tell "finished" from "thinking." LiveKit's open eot-bench puts numbers on this: a VAD threshold sits around ~1600ms of dead air to match the interruption rate a semantic detector hits at ~543ms.

We threw the timer out.

The new loop is one decision: did they hand the floor over?

Pipeline is still the boring one everyone has:

PSTN → SignalWire Stream → Deepgram (mulaw) → LLM → Cartesia → PSTN

The interesting part is what happens at the pause.

Deepgram speech_final is only a candidate end of turn. A cheap classifier then estimates P(caller finished), from the utterance plus the last few dialogue turns. Extra wait is a decreasing function of that probability:

P ≥ 0.75 → commit now (crisp question / complete request)
P ≤ 0.25 → wait up to 1.1s (trailing "and the…", "my number is")
in between → linear ramp 1.1s → 0s
classifier late or missing → 0.55s fallback

Obvious cases never hit the model. A trailing conjunction (and, but, because, the, to…) returns P=0.05 in a few microseconds. A question mark returns 0.95. One- or two-word acknowledgements ("okay", "yeah") lean incomplete, because on a phone they usually mean more is coming. Ambiguous text goes to a 4-token gpt-4o-mini call that is only allowed to emit a float.

The classifier runs in the background on every finalized STT segment, not at the endpoint. By the time Deepgram fires speech_final, a fresh P is usually already cached. Decision budget is 45ms of waiting for an in-flight score; if it misses, the HTTP call is shielded so the result warms the next pause instead of being thrown away. That is the difference between "classifier on the hot path" and "classifier adjacent to the hot path."

Speculative generation hides the remaining wait

While we decide, we already started LLM → streaming TTS into an asyncio queue. Committing a complete turn means flushing audio that already exists. The caller hears the first Cartesia chunk almost as they finish. If they resume mid-window we cancel the task, drop the queue, and reseed the turn detector with the earlier fragment so the next endpoint concatenates old + new words. The LLM never sees a half-thought as a finished turn.

That join matters more than it sounds. Without it you get:

Human: "My account number is"
(pause, we almost answer)
Human: "44821"
LLM hears only "44821"

With reseed it hears "My account number is 44821". Same audio. Different product.

Barge-in is a VAD event, not a transcript event

The playback loop used to notice interruptions when it got around to checking a flag. If TTS was starved, await audio_queue.get() parked and the agent talked over the human for another 200–400ms. That is the failure people describe as "it won't shut up."

SpeechStarted from Deepgram fires on voice energy, before any words. We now:

set the barge-in flag immediately
flush the provider audio buffer in the VAD handler, not in the playback loop (~50ms)
poll the flag at least every 50ms even when the TTS queue is empty
ignore barge-in for the first 250ms / 3 chunks of playback, because analogue line echo of the agent's own first syllable looks exactly like the caller starting to speak

Only words actually heard get committed. Partial agent replies are stored with [interrupted] so the next LLM turn knows what the human already heard. Context matching reality is a latency feature. A model that thinks it finished a sentence the caller never heard will double-speak.

Transcript writes, voice persistence, webhook fan-out for non-live events — all asyncio.create_task. The hot path is: receive mulaw, maybe cancel, maybe flush, maybe start generation.

"Agent-first" is not a landing-page adjective

The latency work is useless if the thing on the other end of the call is a dashboard a human has to click. The customer for AgentLine is an agent runtime — Hermes, OpenClaw, Claude Code, Codex, Claude.ai via MCP — that needs a real PSTN number.

That forced a different product surface.

Signup is email OTP over plain REST. MCP itself requires a Bearer key to connect, so you cannot put signup behind MCP. POST /v1/auth/otp then POST /v1/auth/verify creates the account and returns a key. Same email always maps to the same account, so the agent that signed up and the human who later logs in with that inbox are not two users. There is no "create account" form an agent can get stuck on.

Keys start with al_live_, not sk_live_. That is not branding. Stripe's secret-key pattern is in every agent runtime's redaction list. We shipped sk_live_ first. Hermes/OpenClaw/Warp ate the key and the agent only ever saw <REDACTED:STRIPE_SECRET>. The account existed. The tool output was empty. We renamed the prefix.

Live turns do not go through a human inbox. A one-file relay (agentline_relay.py install --agent-id agt_xxx) detects the local runtime, installs a user service (systemd / launchd / Task Scheduler), and holds a WebSocket. Each phone call maps to one persistent runtime session — Claude --resume, Codex thread id, Hermes X-Hermes-Session-Key, OpenClaw --session-key. Untrusted SMS and lifecycle events land in a local inbox the agent consumes later. Live utterances invoke the runtime now.

The webhook/WS payload carries a push_token. The agent POSTs the spoken answer back with X-Push-Token. No API key on the hot path, because the most common 401 we saw was "runtime has the key in env but the model refused to put it in the header." The text it pushes is spoken verbatim. If it writes a Slack message instead, the caller hears "let me check…" until the turn expires. That is why the event is loud about ACTION_REQUIRED: LIVE CALLER WAITING.

Hosted voice LLM handles social turns locally ("hey, how are you"). Factual / tool turns emit [CHECKING], play a stall phrase, and wait for the agent's push. If the webhook is known-dead we stop emitting call.utterance for the rest of the call and answer hosted. A dead agent must not hold the PSTN stream.

Inbound answer XML is the other sacred path. publish_event() inserts the mailbox row and schedules webhook delivery as a background task. We used to await the webhook before returning LaML. Slow or missing endpoints meant Stream never started, duration=0, failed / no-answer. A webhook that cannot delay the phone from ringing is an agent-first constraint, not a reliability nice-to-have.

Claude.ai talks to us over MCP + OAuth 2.1 (PKCE, dynamic client registration). Desktop uses mcp-remote + the Bearer key. Same tools either way: provision a number, start a call, push context, hang up.

What this actually sounds like

Caller: "Can you check if John emailed me back?"
Agent (hosted, 0 extra wait — it's a complete question): "Hmm, let me check that for you…"
[CHECKING]
Relay hits the local runtime, runtime reads mail, POSTs:
"Yeah — John replied twenty minutes ago, he can do Thursday at 2."
Cartesia speaks that sentence. Caller never sees a dashboard.

Caller: "My order number is…"
Classifier: P≈0.05, we wait. They continue "…44821."
Joined turn goes to the LLM once.

Caller talks over the agent two words in. Provider buffer clears on SpeechStarted. Transcript stores what was actually heard. Next turn does not repeat it.

Honest limits

The classifier is still a tiny chat completion, not a local audio turn-detector. We left the same decide_wait interface so we can swap in LiveKit's ONNX turn-detector (~50ms CPU), Pipecat SmartTurn, or a VAP model (Ekstedt & Skantze / MaAI) without touching the pipeline. Text-only P(finished) cannot hear a drawn-out "soooo…" that means "still thinking." Acoustic fusion is the next obvious cut.

We also do not claim sub-200ms end-to-end on PSTN. Carrier jitter, Deepgram endpointing (~300ms), Cartesia TTFA, and the PSTN itself are a floor. What we removed is the self-inflicted 900ms, the barge-in that waited on a playback loop, and the product shape that assumed a human would operate the phone.

If you have an agent that already does work, it can have a phone number. If you have been fighting a silence timer, steal the probability-to-wait function — it is forty lines and it is the whole trick.

https://agentline.cloud
https://api.agentline.cloud/mcp

The internet has a new power user. It's not human.

SAMEER SRIVASTAV — Sun, 05 Jul 2026 07:15:13 +0000

Agents with access to a SKILL.md file for a service are 4-5x more likely to successfully integrate it on the first try vs. reading raw API docs. They make fewer mistakes, waste fewer tokens, and produce better results.
Your API docs were written for humans. Your SKILL.md is written for agents. You need both. So what makes a software agent ready.

1. Agent-Native Payments
Agents don't have wallets. They don't have credit cards. They can't fill out billing forms.
But they CAN:

Pay per call via MCP Payment protocol
Handle HTTP 402 Payment Required responses
Use prepaid API credits deducted automatically The x402 standard is especially elegant, when an agent hits a paywalled endpoint, it receives an HTTP 402 with payment details, pays programmatically, and retries. All in milliseconds. No human involved.

2. Imagine an agent that needs to verify a phone number.
It discovers AgentLine → reads our SKILL.md → hits the number provisioning endpoint → receives an HTTP 402 → pays from its MCP wallet → gets the number → sends a verification call.
That entire flow took 3 seconds and 0 humans. That's the future.

3. llms.txt — Your Service's Front Door
Robots.txt told crawlers what to index. llms.txt tells AI agents what your service IS. An example looks like-

agentline.cloud/llms.txt

AgentLine provides phone numbers for AI agents.
API base: https://api.agentline.cloud/v1
Auth: Bearer token in header
SKILL.md: https://agentline.cloud/skill.md
One file at your root domain. Every AI that crawls the web sees it. Don't leave this blank it's your first impression to the agent economy.

4. Design APIs for Machine Consumers First
This one is controversial but hear me out.
When you design an endpoint, ask: "Could an agent call this without reading a tutorial?" If the answer is no, simplify it.

Consistent JSON responses (not HTML wrapped in JSON)
Clear error messages with actionable codes
Idempotency keys for safe retries
Pagination that works without guessing
Rate limit headers that agents can read and respect Humans can deal with quirky APIs. Agents can't. Clean design compounds.

5. Let me give you a concrete example of what NOT to do:
I tried integrating a popular CRM's API for my agent. The docs were 200 pages. Authentication required registering an app in their developer portal (which took 2 days to approve). The webhook system required me to host a server.
My agent couldn't use it. I switched to a competitor that had API keys, a SKILL.md, and webhook URLs I could configure in one POST request. Done in 10 minutes.

6. The companies that win the agent era won't have the best UI.
They'll have:

The clearest SKILL.md
The most programmatic auth
The simplest agent payment flow
The most discoverable llms.txt These are moats that compound. Every agent that successfully integrates your API becomes a distribution channel. Every SKILL.md shared in an agent community is free marketing.

7. We're already seeing this play out.
The tools agents recommend to each other aren't the ones with the best landing pages or the biggest ad budgets. They're the ones that work reliably when an agent calls them at 3 AM with zero human supervision.
AgentLine is also trying its best to Be that tool.

8. If you're building something right now, here's your checklist:
→ llms.txt at your root domain
→ SKILL.md for every API surface
→ Programmatic auth (API keys + machine-to-machine OAuth)
→ Agent payment support (MCP Payment / x402)
→ Clean, consistent, well-documented endpoints
Do these 5 things and you're ahead of most of the services on the internet.

9. AgentLine is our bet on this future.
Phone numbers for agents. No dashboards. No forms. No humans required. Because "The most important user of the next decade won't have thumbs."
agentline.cloud

How AgentLine solves calling for Agents.

SAMEER SRIVASTAV — Fri, 26 Jun 2026 11:32:43 +0000

AgentLine is solving one of the most important problems that AI agents face today. Agents can do a lot of tasks now because the models behind them have become more intelligent and the context size has also increased rapidly so they are doing more and more tasks now. Also after the release of openclaw and Hermes, agents have gone into mainstream and people are using them in their day-to-day life to automate different tasks for themselves. For, say, managing their emails, they give their agents an email to manage it, send emails, and do all kinds of tasks around it. At the same time the agents already have access to their file systems and terminal so that agents can write code, create images, read files, and manipulate things in all kinds of ways. Agents also have web access so they can search out things if they don't know something. Agents can be given access to external tools as well, such as your calendar, your Google meet, etc., so that they can manage them as well.

For an agent to act in the way a human does and complete tasks end-to-end, it must have all the necessary tools that a human has, like from email to phone numbers to access to the physical world to access to a payment method. Most of the tasks need at least one or two of these things combined to complete it end-to-end and not having one of these tools might break the task in between. The agent will probably offload that particular point of the task to their owner. Now email is already solved for agents by AgentMail. Payments are yet to be solved. Companies like rent a human etc. are giving AI agents a physical body in the real world to complete tasks. Phone number and calling was still one of these things that wasn't solved yet and AgentLine is solving it gracefully.

Getting a phone for IVR takes a lot of regulations and API tweaking in order to give it to agents. What people have been doing before is they wind up Twilio APIs with ElevenLabs APIs and then give it to their agents. They cannot manage the latency well. They themselves have to get approvals from Twilio in order to use it for longer periods. All these things AgentLine is solving by handling all these regulatory issues themselves: the speech-to-text and the text-to-speech themselves and also the phone number provisioning, everything on the backend. You don't have to wind up anything. They also take a look at latency and how the users are using it so it's very agent-specific and not built for humans.

Now how you can get a phone number for your agent: the easiest way is to give it a skill file and it will arrange everything on its own. That's how AgentLine does it. You just have to point your agent to agentline.cloud/skill.md. It will provision the phone number on its own. It will set the tone for itself. It will set its persona and call you all on its own. AgentLine has reduced the latency of these calls to less than 2 seconds, which is very human when it comes to the responses.

At the same time AgentLine is cost effective too. Other IVR providers or agents are costing more than $3 a number whereas AgentLine costs less than $2 a number per month, which makes sense for daily life use case because nobody is going to spend a dollar for booking a flight ticket for himself by calling an attendant. Now a lot of people don't find use cases at the forefront so giving your agent a phone number now is always a better choice so that the spectrum of tasks getting completed increases rapidly.

At the same time as you progress, giving more autonomy to your agent, you'll get to know what are the use cases of it. Having a phone number is very important because the agent might have to log in to some place where two-factor authentication is required so it can get the SMS and log into that place using its phone number. Sometimes people are also using it as a support agent. They just ask their agent to turn this number into a support agent and all the support calls are handled by their agent. This is one of the use cases.

Other use cases such as outbound sales calling: some people are using it for outbound sales calling. The agent calls them, books the meeting, and handles everything on its own. Some people are using it for feedback collection from their users. The agent calls their company users or their customers at definite amounts of interval and collects feedback from their customers. It notes it down, analyzes it, and plans its next move.

Now this was a small brief about AgentLine and how telephony for AI agents is changing the way agents interact with the real world. If you want a phone number for your agent as well, you can get it from AgentLine.