<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: RoundReply</title>
    <description>The latest articles on DEV Community by RoundReply (@roundreply).</description>
    <link>https://dev.to/roundreply</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4246318%2F30806cb1-a29b-456c-9ee0-40d77fb3498c.png</url>
      <title>DEV Community: RoundReply</title>
      <link>https://dev.to/roundreply</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/roundreply"/>
    <language>en</language>
    <item>
      <title>Where 3.6 seconds go in a real-time AI answer, and how we got it to 2.6</title>
      <dc:creator>RoundReply</dc:creator>
      <pubDate>Mon, 16 Nov 2026 03:51:57 +0000</pubDate>
      <link>https://dev.to/roundreply/where-36-seconds-go-in-a-real-time-ai-answer-and-how-we-got-it-to-26-2o26</link>
      <guid>https://dev.to/roundreply/where-36-seconds-go-in-a-real-time-ai-answer-and-how-we-got-it-to-26-2o26</guid>
      <description>&lt;p&gt;We build an interview copilot: it hears a question on a call and starts writing an answer. The number that matters is the time from the end of the question to the first visible words. In our browser call it was a median of 3.6 s. Here is where the time went, measured on 12 recorded questions with real providers.&lt;/p&gt;

&lt;h2&gt;
  
  
  The breakdown
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;End of speech to a final transcript line: about 1.6 s.&lt;/strong&gt; We ask the speech engine to end a line after 0.45 s of silence, but in practice the final line arrived a median 1.56 s after speech stopped.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Detecting the question and sending the request: under 0.05 s.&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Tokens, the call's lease and building the prompt: 0 to 0.3 s.&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Opening a connection to the model: 0.73 s,&lt;/strong&gt; because the connection was closed between questions.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Model request to first text: about 1.1 s.&lt;/strong&gt; The stable part of the prompt was already cached (about 3,000 of 3,400 input tokens).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Rendering: under 0.05 s.&lt;/strong&gt; Streaming the first sentence sooner would have saved nothing (6 ms).&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What worked
&lt;/h2&gt;

&lt;p&gt;Keeping the API's connection to the model open while a call is live. The model stage went from 1.74 s to 1.15 s. The whole path: median 3.58 s to 2.65 s, 17 of 17 questions answered whole. Cost: one tiny keep-alive request every few seconds per server, only while a call runs.&lt;/p&gt;

&lt;h2&gt;
  
  
  What did not (yet)
&lt;/h2&gt;

&lt;p&gt;Ending the question sooner from our own silence detection brought the final line to about 0.8 s, but in some runs the transcription then cut questions in two a second after they began. A faster wrong answer is worse than a slower right one, so it stays off until we understand why.&lt;/p&gt;

&lt;h2&gt;
  
  
  Takeaways
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Measure each stage on real audio before optimising: our guess (the model) was only half the story.&lt;/li&gt;
&lt;li&gt;Connection set-up is invisible in benchmarks and very visible in live calls.&lt;/li&gt;
&lt;li&gt;Speech end-of-utterance is the floor you fight next.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;We're building RoundReply (roundreply.com): live transcripts and real-time interview answers from your resume.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>webdev</category>
      <category>performance</category>
      <category>speechrecognition</category>
    </item>
  </channel>
</rss>
