<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Alperen Ayhan</title>
    <description>The latest articles on DEV Community by Alperen Ayhan (@alpayhann).</description>
    <link>https://dev.to/alpayhann</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4252533%2F81f32760-cd14-4f20-ae0b-a151f7904e6f.jpeg</url>
      <title>DEV Community: Alperen Ayhan</title>
      <link>https://dev.to/alpayhann</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/alpayhann"/>
    <language>en</language>
    <item>
      <title>How I cut my real-time voice AI cost by 62%</title>
      <dc:creator>Alperen Ayhan</dc:creator>
      <pubDate>Wed, 18 Nov 2026 20:12:55 +0000</pubDate>
      <link>https://dev.to/alpayhann/how-i-cut-my-real-time-voice-ai-cost-by-62-4hb6</link>
      <guid>https://dev.to/alpayhann/how-i-cut-my-real-time-voice-ai-cost-by-62-4hb6</guid>
      <description>&lt;p&gt;I build TalkSmart, an app where English learners practise speaking with an AI voice partner. A session is a real-time voice conversation of about 7 to 15 minutes. The voice runs on the Gemini Live API: one model hears the learner and answers with audio.&lt;/p&gt;

&lt;p&gt;Voice is the only part of the app that costs real money per use, so I measured it carefully. It turned out I was measuring the wrong thing.&lt;/p&gt;

&lt;h3&gt;
  
  
  My cost log said $0.11. The invoice said $0.40.
&lt;/h3&gt;

&lt;p&gt;My first estimate was simple. Live audio is billed per token, and audio tokens map to seconds. When I measured it, that came to about 32 tokens per second of input audio and 25 tokens per second of output audio. With $3 per million input tokens and $12 per million output tokens, that is roughly half a cent per minute of learner speech and about two cents per minute of AI speech.&lt;/p&gt;

&lt;p&gt;So my server logged &lt;code&gt;seconds of learner audio × input rate + seconds of AI audio × output rate&lt;/code&gt; for each session. A 6-minute test session came out at $0.046. A full session would be around $0.11. Great margins, I thought.&lt;/p&gt;

&lt;p&gt;Then I compared it with the usage numbers in Google's own console. A 12-minute session with no special handling cost about &lt;strong&gt;$0.40&lt;/strong&gt;, roughly 3.5 times my estimate.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why: every turn pays for the whole conversation again
&lt;/h3&gt;

&lt;p&gt;A live session is stateful. The model keeps the conversation so far in its context, and &lt;strong&gt;each new turn bills that whole context again as input tokens&lt;/strong&gt;. The first turn pays for one turn of context, the tenth turn pays for ten, and so on.&lt;/p&gt;

&lt;p&gt;My log only counted the new audio in each turn. It never counted the conversation being re-read.&lt;/p&gt;

&lt;p&gt;I hoped prompt caching would soften this. In my sessions the usage metadata showed &lt;code&gt;cachedTok=0&lt;/code&gt; on every turn. Implicit caching discounts did not apply to these live audio sessions.&lt;/p&gt;

&lt;h3&gt;
  
  
  The proof: cost tracked the square of duration
&lt;/h3&gt;

&lt;p&gt;If every turn re-pays for all previous turns, the total input cost is roughly 1 + 2 + 3 + … + n, which is about n²/2. Doubling the conversation should roughly quadruple the input cost.&lt;/p&gt;

&lt;p&gt;Two real sessions, same setup:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Session length&lt;/th&gt;
&lt;th&gt;Real cost&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;8.3 min&lt;/td&gt;
&lt;td&gt;13.18 TRY&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;9.7 min&lt;/td&gt;
&lt;td&gt;18.12 TRY&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;If cost were linear, 9.7 minutes would cost 13.18 × (9.7 / 8.3) ≈ 15.4. If it is quadratic, it would cost 13.18 × (9.7 / 8.3)² ≈ &lt;strong&gt;18.0&lt;/strong&gt;. The real number was 18.12.&lt;/p&gt;

&lt;p&gt;(I'm in Turkey, so my console shows Turkish lira. At the rate I used then, 18.12 TRY is about $0.39.)&lt;/p&gt;

&lt;p&gt;Two points don't make a law, but together with the billing model they were convincing enough. For a 15-minute product, a quadratic cost curve is a serious problem. A heavy user doing 33 sessions a month at $0.40 each would eat almost all of a $14.99 subscription.&lt;/p&gt;

&lt;h3&gt;
  
  
  What didn't work: context window compression
&lt;/h3&gt;

&lt;p&gt;The Live API has a context window compression setting with a sliding window. I tried it first.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="n"&gt;contextWindowCompression&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="n"&gt;slidingWindow&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;With the defaults, nothing changed. The default trigger sits near the model's large context limit, and a 12-minute voice chat never gets there. Same cost as before.&lt;/p&gt;

&lt;p&gt;Setting the trigger explicitly does kick in:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="n"&gt;contextWindowCompression&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="n"&gt;triggerTokens&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'8000'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;slidingWindow&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;targetTokens&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'4000'&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That cut the cost a lot in my test. But I didn't keep it. A sliding window drops the oldest turns blindly, without knowing which parts matter, and with a window this small it interfered with turn-taking in my push-to-talk setup. For a speaking tutor, remembering that the learner mentioned their sister's wedding five minutes ago is the product. I wanted to choose what to keep.&lt;/p&gt;

&lt;h3&gt;
  
  
  What worked: summarise and restart every N turns
&lt;/h3&gt;

&lt;p&gt;Instead of letting the context grow, I cut it on purpose and keep the meaning:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Every N learner turns (I started with N = 6), the server sends the transcript so far to a cheap text model and asks for a short summary: topic, what the learner said, mistakes worth coming back to, where the conversation was going.&lt;/li&gt;
&lt;li&gt;The server opens a &lt;strong&gt;new&lt;/strong&gt; live session with the same voice. The summary goes into the system instruction, with a note to continue the conversation, not greet the learner again.&lt;/li&gt;
&lt;li&gt;When the new session is ready, the server switches over and closes the old one.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;In pseudo-TypeScript:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;resetContext&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Session&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;summary&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;summarise&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;transcript&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;   &lt;span class="c1"&gt;// cheap text model&lt;/span&gt;
  &lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;swapping&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;                               &lt;span class="c1"&gt;// buffer incoming audio from now on&lt;/span&gt;

  &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;next&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;openLiveSession&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="n"&gt;voice&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;voice&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="n"&gt;systemInstruction&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;buildPrompt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;resumeSummary&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;summary&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;old&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;live&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;live&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;next&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="n"&gt;old&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;close&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;op&lt;/span&gt; &lt;span class="k"&gt;of&lt;/span&gt; &lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;buffered&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;next&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;send&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;op&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;   &lt;span class="c1"&gt;// replay in order&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;catch&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c1"&gt;// keep the old session; a failed reset must never end the conversation&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;finally&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;buffered&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;swapping&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The details that mattered:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Buffer the learner's audio during the swap.&lt;/strong&gt; Opening a new live session takes a moment. If the learner starts talking during the swap, their audio and their push-to-talk start/end events are queued and replayed in order on the new session. Otherwise you lose a turn, and the learner thinks the app didn't hear them.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fail safe.&lt;/strong&gt; If the summary or the new connection fails, keep the old session and try again later. A cost optimisation must never end someone's conversation.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Reset at a turn boundary&lt;/strong&gt;, after the AI has finished speaking, not in the middle of an answer.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Keep the cost log honest.&lt;/strong&gt; Each new session starts its own counters, so the session total has to add up all of them.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  The result
&lt;/h3&gt;

&lt;p&gt;Real numbers from a device test, a 10.4-minute session with three resets (after turns 6, 12 and 18):&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;No reset (9.7 min)&lt;/th&gt;
&lt;th&gt;With reset (10.4 min)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Input context per turn, near the end&lt;/td&gt;
&lt;td&gt;13,232 tokens&lt;/td&gt;
&lt;td&gt;1,307 tokens&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Total cost&lt;/td&gt;
&lt;td&gt;18.12 TRY (~$0.39)&lt;/td&gt;
&lt;td&gt;6.86 TRY (~$0.15)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cost per minute&lt;/td&gt;
&lt;td&gt;1.87 TRY&lt;/td&gt;
&lt;td&gt;0.66 TRY&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The longer session cost &lt;strong&gt;62% less&lt;/strong&gt; in total and about 65% less per minute. Cost now grows roughly linearly with session length. The quality held: after each reset the tutor remembered the topic and carried on naturally, with no second greeting and no lost turns.&lt;/p&gt;

&lt;p&gt;These are single sessions, and sessions vary naturally by 20–30% in my data (some learners talk more, some answers are longer). The direction and size of the change were clear, but treat the exact percentage as one measurement, not a benchmark.&lt;/p&gt;

&lt;h3&gt;
  
  
  How small should N be?
&lt;/h3&gt;

&lt;p&gt;It's tempting to reset every two turns. The maths says it's not worth it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Output audio doesn't change.&lt;/strong&gt; The AI's spoken answers cost the same however you manage context. In my sessions that was close to half the total after the reset, so it's a floor you can't get under this way.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Input cost inside each block grows with N.&lt;/strong&gt; Over a session of T turns with a reset every N turns, the re-billed context is about T × N / 2. It is linear in N, so halving N halves the input part, not the total.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;From that estimate: going from N = 6 to N = 5 would save about 7% in total, N = 4 about 15%, N = 3 about 22%. With 20–30% natural variation between sessions, a 7% saving is impossible to see in a handful of tests, so I skipped N = 5 and tried N = 4. It stayed in the expected range and the conversation still felt smooth.&lt;/p&gt;

&lt;p&gt;Each reset also adds a summary call and a short reconnect, and every summary loses a bit of detail. For me, 4–6 turns is the sweet spot.&lt;/p&gt;

&lt;p&gt;The next lever was the other half of the bill: shorter AI answers. Cutting the tutor's typical answer from about 60 to 40 words reduced cost per minute a bit more, and it was better teaching anyway. In a speaking app, the learner should do most of the talking.&lt;/p&gt;

&lt;h3&gt;
  
  
  Takeaways
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;With stateful real-time APIs, &lt;strong&gt;cost per turn depends on everything before it&lt;/strong&gt;. Estimate from tokens actually billed, not from seconds of audio.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Compare your own cost log with the provider's console in the first week&lt;/strong&gt;, before you set prices.&lt;/li&gt;
&lt;li&gt;Check whether caching applies to your mode. Don't assume it does.&lt;/li&gt;
&lt;li&gt;Generic context compression saves money but can break what makes the conversation good. A summary written for your use case keeps what matters.&lt;/li&gt;
&lt;li&gt;Make the context reset invisible: buffer input during the swap, keep the old session if anything fails.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you've solved this differently (with OpenAI's Realtime API, or with a different reset strategy), I'd love to hear how in the comments.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;I'm building &lt;a href="https://talksmartapp.com" rel="noopener noreferrer"&gt;TalkSmart&lt;/a&gt;, an AI speaking partner for English learners. Next post: the bugs that made most first conversations on mobile silent, and how I found them.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>webdev</category>
      <category>startup</category>
      <category>mobile</category>
    </item>
  </channel>
</rss>
