<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: devs</title>
    <description>The latest articles on DEV Community by devs (@madnessbrains_i).</description>
    <link>https://dev.to/madnessbrains_i</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4208612%2F8c1d2174-7f77-4a02-869b-d4d966403085.jpg</url>
      <title>DEV Community: devs</title>
      <link>https://dev.to/madnessbrains_i</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/madnessbrains_i"/>
    <language>en</language>
    <item>
      <title>OpenCode 2.3 Defaults Background Subagents — Parent Sessions Stay Live While Children Report Idle</title>
      <dc:creator>devs</dc:creator>
      <pubDate>Sun, 15 Nov 2026 05:15:50 +0000</pubDate>
      <link>https://dev.to/madnessbrains_i/opencode-23-defaults-background-subagents-parent-sessions-stay-live-while-children-report-idle-4jfk</link>
      <guid>https://dev.to/madnessbrains_i/opencode-23-defaults-background-subagents-parent-sessions-stay-live-while-children-report-idle-4jfk</guid>
      <description>&lt;p&gt;On 13 November 2026 OpenCode shipped v2.3.0. The release notes are short on marketing and long on the one change that actually moves the needle for anyone building multi-agent loops: subagents now return immediately, continue in the background, and report back whenever they become idle. The parent session receives a session ID at once and can keep issuing work or waiting on other results. When a child finishes (or goes idle while its own nested work is still running) it notifies the parent. The system explicitly avoids treating a partial idle as completion if background tasks remain.&lt;/p&gt;

&lt;p&gt;That is the opposite of the blocking model most agent frameworks still ship by default. In the old shape you spawn a subagent, the parent blocks until the child returns a final answer, then you continue. With any non-trivial tool use or nested delegation the wall-clock time compounds and the parent session becomes a single point of serialization. Background-by-default flips the control flow: the parent remains a live coordinator.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fylsx9t09rp7oc76as8p8.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fylsx9t09rp7oc76as8p8.png" alt="Agent harness components: model, tools, memory, sandbox, and feedback loop" width="800" height="416"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What the implementation actually does
&lt;/h2&gt;

&lt;p&gt;From the changelog:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Subagents always run in the background. They return their session ID immediately.&lt;/li&gt;
&lt;li&gt;They report results to the parent whenever they become idle.&lt;/li&gt;
&lt;li&gt;If a subagent goes idle while its own background work is still running, the report to the parent says it is still running, so the parent waits for the final report instead of acting on a partial answer.&lt;/li&gt;
&lt;li&gt;Experimental MCP event subscription sits behind the &lt;code&gt;mcp_events&lt;/code&gt; flag.&lt;/li&gt;
&lt;li&gt;Shell commands survive server handoffs and keep their output when interrupted. An experimental &lt;code&gt;shell_timeout_background&lt;/code&gt; flag moves timed-out shell commands to the background instead of killing them.&lt;/li&gt;
&lt;li&gt;Local Windows shell commands now run through the process daemon.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The TUI also defaults to lower verbosity: each run of tool calls and thoughts collapses into one activity row. Set verbosity to medium or high if you need the full trace. The desktop app adds a jump-to-message command (Cmd+Alt+G / Ctrl+Alt+G) so you can search earlier prompts in the current session.&lt;/p&gt;

&lt;p&gt;These are not speculative features. They are the defaults that change how you structure a session that launches multiple children.&lt;/p&gt;

&lt;h2&gt;
  
  
  Practical consequences for agent loops
&lt;/h2&gt;

&lt;p&gt;Most production agent loops look roughly like this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Parent receives a user or system goal.&lt;/li&gt;
&lt;li&gt;Parent decides which tools or subagents are needed.&lt;/li&gt;
&lt;li&gt;Parent issues the calls.&lt;/li&gt;
&lt;li&gt;Parent collects results and decides the next step or final answer.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;When step 3 blocks, the loop cannot overlap independent work. Background subagents let the parent issue several in parallel and continue reasoning or issuing more work while the children run. The idle-report protocol is the important part: it gives the parent a clear signal without forcing it to poll, and the “still running” case prevents the classic race where a parent acts on an incomplete result.&lt;/p&gt;

&lt;p&gt;For CI or scripted runs the same change surfaces a gotcha. If a subagent starts a long-lived process (dev server, file watcher, or any tool that never naturally exits), the parent can hang waiting for an idle that never arrives. The release notes and follow-up commentary call this out. The practical fix is to treat those children as fire-and-forget or to give them an explicit timeout / completion criterion that the harness can observe.&lt;/p&gt;

&lt;h2&gt;
  
  
  Architecture notes and tradeoffs
&lt;/h2&gt;

&lt;p&gt;OpenCode’s model is closer to a harness that owns the session lifetime than to a pure library you call from your own process. The parent session stays in the foreground; children are background sessions that share the same process and tool infrastructure. That gives you free concurrency without spinning extra OS processes for every agent, but it also means a hung or misbehaving child can still affect the shared environment (file locks, ports, GPU memory, etc.).&lt;/p&gt;

&lt;p&gt;Compare that with a pure MCP-style or CDP-style isolation model (the kind Cloudflare’s Kitesurf or a dedicated browser-agent binary uses). Isolation is cleaner for untrusted or long-running work; the shared-session model is cheaper and faster for trusted local tooling. OpenCode chose the latter as the default and left the more isolated options as experimental flags or future work.&lt;/p&gt;

&lt;p&gt;On Windows the process-daemon change for local shell commands is relevant if you are driving the agent from a long-lived desktop process. The daemon survives the parent session better and keeps output across handoffs. That matters when your outer loop is a Windows service, a tray app, or anything that restarts the OpenCode process without wanting to lose in-flight shell state.&lt;/p&gt;

&lt;p&gt;Latency-wise the win is obvious for any workload that has independent sub-tasks. The cost is bookkeeping: you now have to handle the idle and “still running” events correctly, and you must decide what “done” means for each child. In a simple sequential agent the old blocking model was easier to reason about. In anything that looks like a real multi-agent or multi-tool loop, the background default removes a serialization tax you were previously paying for free.&lt;/p&gt;

&lt;h2&gt;
  
  
  How this lands next to local audio and real-time pipelines
&lt;/h2&gt;

&lt;p&gt;If your outer system is already dealing with streaming audio, dual-track capture, or tight latency budgets (the usual Windows WASAPI + STT + overlay path), the same principle applies: keep the coordinator responsive. A parent that blocks on a long STT partial or a tool call that waits for network will miss the next speech segment or the next overlay update. Background subagents (or any equivalent async tool dispatch) let the parent stay in the critical path while the slower work happens elsewhere.&lt;/p&gt;

&lt;p&gt;The OpenCode change does not magically solve capture latency or model TTFS. It does remove one more place where an agent framework forces you to serialize work that does not need to be serialized. That is the kind of incremental improvement that compounds when you are already fighting the Windows audio engine, DPC spikes, and network RTT.&lt;/p&gt;

&lt;h2&gt;
  
  
  Minimal example shape
&lt;/h2&gt;

&lt;p&gt;A parent that launches two independent children now looks roughly like:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;parent: issue subagent A (research task)
parent: issue subagent B (code generation)
parent: continue local reasoning or other tool calls
# later
parent: receives idle report from A with final result
parent: receives idle report from B (or “still running” if B has nested work)
parent: decide next step or emit final answer
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;No explicit wait, no thread pool you have to manage yourself, and a defined protocol for “not done yet.”&lt;/p&gt;

&lt;p&gt;That is the practical difference between a library that forces you to invent your own concurrency and a harness that ships the common case as the default.&lt;/p&gt;

&lt;p&gt;OpenCode 2.3 is a small release on paper. For anyone who has watched a parent session sit idle while three subagents run sequentially, it is the change that finally matches the architecture you wanted.&lt;/p&gt;

&lt;p&gt;I'm building &lt;a href="https://talkaside.com/" rel="noopener noreferrer"&gt;Talkaside&lt;/a&gt;, a Windows app for job interviews and calls in another language.&lt;br&gt;
&lt;em&gt;Drafted with AI assistance from this project's code and logs.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>agents</category>
      <category>automation</category>
      <category>opensource</category>
      <category>tooling</category>
    </item>
    <item>
      <title>Cloudflare Open-Sources Kitesurf — Lightweight CDP Browser Engine for Agents on Workers</title>
      <dc:creator>devs</dc:creator>
      <pubDate>Sun, 15 Nov 2026 04:15:59 +0000</pubDate>
      <link>https://dev.to/madnessbrains_i/cloudflare-open-sources-kitesurf-lightweight-cdp-browser-engine-for-agents-on-workers-1170</link>
      <guid>https://dev.to/madnessbrains_i/cloudflare-open-sources-kitesurf-lightweight-cdp-browser-engine-for-agents-on-workers-1170</guid>
      <description>&lt;p&gt;On 13 November 2026 Cloudflare open-sourced Kitesurf under Apache 2.0. The repo is at github.com/cloudflare/kitesurf. It is a stateless browser engine written primarily in Rust (compiled to WebAssembly) that runs entirely inside Cloudflare Workers isolates. It parses HTML, applies CSS, executes page JavaScript, and can produce screenshots without shipping a Chromium binary or a browser container. It speaks the Chrome DevTools Protocol, so existing clients (Puppeteer, chrome-remote-interface, MCP servers that already talk CDP) can attach.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpcpvtxse2aw5fwmw17kb.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpcpvtxse2aw5fwmw17kb.png" alt="How a browser turns a URL into a rendered page" width="800" height="502"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That matters for anyone building agent loops that need to look at the web. Full Chromium is still the safe default when you need pixel-perfect layout, complex SPAs, or every Web Platform feature. It is also expensive: each session burns hundreds of megabytes and non-trivial CPU even when the agent is mostly waiting. Kitesurf’s published numbers (median of five runs across a small set of URLs) show 3.1–3.8× less CPU and 4.7–7.0× less memory than a warm Chromium pool for screenshot and HTML-extraction tasks, at the cost of roughly 1.7–1.8× higher wall time. The engine is explicitly not trying to be a human browser; tabs, extensions, and perfect fidelity are out of scope.&lt;/p&gt;

&lt;h2&gt;
  
  
  Architecture that fits the agent loop
&lt;/h2&gt;

&lt;p&gt;Kitesurf splits into three pieces that map cleanly onto Workers:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Engine — the only public-facing component. It owns the CDP WebSocket and REST surface and keeps session state.&lt;/li&gt;
&lt;li&gt;PageScript — a long-lived isolate per page (Dynamic Workers) that holds the DOM and runs JavaScript. Built on pieces of Blitz for HTML/DOM/layout and Stylo for CSS.&lt;/li&gt;
&lt;li&gt;PageRenderer — the rendering side.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Because everything is isolate-scoped and the heavy parts are stateless, you can scale by spinning more Workers rather than more OS processes or VMs. Local development is straightforward:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;git clone https://github.com/cloudflare/kitesurf.git
cd kitesurf
pnpm install --frozen-lockfile
pnpm run dev
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The first build compiles the Rust core; later builds hit the cache. Wrangler then serves a local endpoint (typically 127.0.0.1:8787). You connect a CDP client to the WebSocket at &lt;code&gt;ws://127.0.0.1:8787/devtools/browser&lt;/code&gt;. Deploying your own instance to Cloudflare requires the Workers Paid plan because of Dynamic Workers; the README also notes that there is no built-in authentication, so you are responsible for locking it down if you expose it.&lt;/p&gt;

&lt;p&gt;WebMCP support (added in the September update) lets sites expose callable tools instead of forcing the agent to reverse-engineer DOM selectors. That reduces the amount of brittle page-scraping logic agents have to carry.&lt;/p&gt;

&lt;h2&gt;
  
  
  Trade-offs that show up in real agent workloads
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Memory and CPU headroom: if your agent is opening many short-lived pages (search, extract, screenshot, close), the lower footprint lets you run more concurrent sessions on the same budget. For long-lived sessions that stay on complex sites, the gap shrinks and Chromium’s compatibility advantage often wins.&lt;/li&gt;
&lt;li&gt;Latency: wall-clock time is higher. In an agent loop the model is usually the dominant cost, so an extra few hundred milliseconds on the browser side is often acceptable. If you are chaining many tiny navigations, measure it.&lt;/li&gt;
&lt;li&gt;Compatibility surface: Kitesurf already handles TodoMVC variants, Wikipedia, Hacker News, and large parts of the Cloudflare dashboard. Anything that depends on obscure layout, WebGL, or certain service-worker patterns will still need Chromium.&lt;/li&gt;
&lt;li&gt;Isolation model: each page lives in its own isolate. That is good for security and resource accounting; it also means you cannot assume shared process memory or cheap cross-page communication the way you can with a single Chromium process.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Where this intersects local tooling
&lt;/h2&gt;

&lt;p&gt;Most of the interesting agent work still happens on the developer’s machine or inside a controlled environment. A lightweight CDP engine is useful when you want to drive browsing from a local process without pulling in a full browser binary and its sandbox. The same pattern shows up in other low-latency local systems: you separate the capture or rendering path from the model so that the expensive part can be scaled or swapped independently.&lt;/p&gt;

&lt;p&gt;On Windows, for example, real-time audio overlays for calls still live or die by the WASAPI capture path and how cleanly you can keep the audio engine callback short. The browser-side equivalent is keeping the page-isolation and rendering path cheap enough that the agent’s reasoning loop is not starved. Kitesurf is one concrete attempt to make that cheaper for the web half of the problem. Local dual-track capture (mic + loopback) plus a small STT model remains the other half when the “page” is actually a live call.&lt;/p&gt;

&lt;p&gt;You can run Kitesurf locally today, point an existing CDP or MCP client at it, and decide per workload whether the resource savings outweigh the compatibility gaps. The source is Apache 2.0, so the usual fork-and-adapt path is open.&lt;/p&gt;

&lt;p&gt;I'm building &lt;a href="https://talkaside.com/" rel="noopener noreferrer"&gt;Talkaside&lt;/a&gt;, a Windows app for job interviews and calls in another language.&lt;br&gt;
&lt;em&gt;Drafted with AI assistance from this project's code and logs.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>opensource</category>
      <category>automation</category>
      <category>agents</category>
    </item>
    <item>
      <title>Claude Meetings Beta Puts an Agent in Google Meet — Local Dual-Track WASAPI Still Owns the Live Overlay Budget</title>
      <dc:creator>devs</dc:creator>
      <pubDate>Sun, 15 Nov 2026 03:13:54 +0000</pubDate>
      <link>https://dev.to/madnessbrains_i/claude-meetings-beta-puts-an-agent-in-google-meet-local-dual-track-wasapi-still-owns-the-live-1hmc</link>
      <guid>https://dev.to/madnessbrains_i/claude-meetings-beta-puts-an-agent-in-google-meet-local-dual-track-wasapi-still-owns-the-live-1hmc</guid>
      <description>&lt;p&gt;On 12 November 2026 Anthropic rolled out Claude Meetings in beta. Claude can now join a Google Meet call as a visible participant, follow the conversation, answer questions out loud or in chat (replies stay in writing by default), build docs and whiteboards that teammates edit live, and drop notes plus action items afterward. Support is Google Meet only for now; other platforms are promised. It is limited to Team and Enterprise plans, stays out of calls with external guests unless an admin allows it, and leaves if one joins. It keeps a transcript and screen descriptions, not the raw audio.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyllf2qo6wqjgk1wjim0v.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyllf2qo6wqjgk1wjim0v.png" alt="Basic waveforms illustrating the raw signal before engine processing" width="800" height="684"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That is useful for teams that already live in Meet and want an agent that starts work while people are still talking. It does not solve the problem of a developer or interviewer who needs a local, low-latency overlay that hears both sides of any call, turns the remote audio into text in another language, and optionally surfaces a short translated reply or suggestion without leaving the Windows desktop.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the cloud participant does not move the first millisecond
&lt;/h2&gt;

&lt;p&gt;Claude Meetings runs on Anthropic’s infrastructure. The Meet client ships the media; Claude consumes the resulting transcript and context. The round-trip for a spoken question plus a written answer is whatever the network and model queue deliver. For many knowledge-work meetings that is fine. For a live language overlay that must stay under a few hundred milliseconds end-to-end so the listener can still follow the original speaker, the local path dominates:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Endpoint and mode — USB headset, Bluetooth HFP/A2DP, built-in, or exclusive-mode WASAPI.&lt;/li&gt;
&lt;li&gt;Dual-track acquisition: one capture client for the local mic, one render-loopback client so the remote party’s audio stays distinct and timed.&lt;/li&gt;
&lt;li&gt;Windows Audio Engine (&lt;code&gt;audiodg.exe&lt;/code&gt; + &lt;code&gt;audioeng.dll&lt;/code&gt;) in shared mode. Mixing, communications enhancements and vendor APOs routinely add 10–40+ ms and become non-deterministic under load.&lt;/li&gt;
&lt;li&gt;Consumer thread: lock-free SPSC ring copy, format conversion, resampling, VAD.&lt;/li&gt;
&lt;li&gt;Streaming STT (local GGUF or remote) plus optional local model polish for translation or short replies.&lt;/li&gt;
&lt;li&gt;Local TTS writing into a render client if a spoken reply is wanted.&lt;/li&gt;
&lt;li&gt;Overlay paint on a dedicated thread that never blocks the audio callback.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Bluetooth HFP alone can inject another 50–150 ms of buffering that no remote agent recovers. Even if Claude finishes a note in under a second, the overlay still waits for the Windows path that delivered the samples.&lt;/p&gt;

&lt;p&gt;A minimal shared-mode capture callback that stays under 1 ms and does no allocation looks like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight c"&gt;&lt;code&gt;&lt;span class="c1"&gt;// WASAPI shared-mode capture — callback must be allocation-free&lt;/span&gt;
&lt;span class="kd"&gt;static&lt;/span&gt; &lt;span class="no"&gt;HRESULT&lt;/span&gt; &lt;span class="no"&gt;STDMETHODCALLTYPE&lt;/span&gt; &lt;span class="nc"&gt;CaptureCallback&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;IAudioCaptureClient&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="no"&gt;UINT32&lt;/span&gt; &lt;span class="n"&gt;packetLength&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;BYTE&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;DWORD&lt;/span&gt; &lt;span class="n"&gt;flags&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;HRESULT&lt;/span&gt; &lt;span class="n"&gt;hr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nc"&gt;GetBuffer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;flags&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;FAILED&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;hr&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;hr&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="c1"&gt;// Writer side of a pre-allocated power-of-two SPSC ring (float32 frames)&lt;/span&gt;
    &lt;span class="nf"&gt;ring_push_float32&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;capture_ring&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nc"&gt;ReleaseBuffer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="no"&gt;S_OK&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The consumer owns resampling (libsamplerate or a fixed polyphase), VAD, and the write to the model or WebSocket. Keep the audio thread pure; any lock or malloc here shows up as glitches or higher DPC latency.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fqc2q4t2x34vm7o39447v.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fqc2q4t2x34vm7o39447v.png" alt="Labeled sine wave showing amplitude and trough" width="800" height="728"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Practical trade-offs that still matter
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Cloud agent vs local stack&lt;/strong&gt;: Claude Meetings is excellent when everyone is already in Meet and the goal is collaborative artifacts. A local dual-track pipeline is required when the call can be any app (Zoom, Teams, Discord, a raw SIP client) and the user needs the remote audio turned into text in a different language before the next sentence finishes.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Shared vs exclusive mode&lt;/strong&gt;: Shared mode is convenient and survives device changes, but the engine and APOs add jitter. Exclusive mode gives tighter periods at the cost of exclusive device ownership and more brittle device switching.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Local vs remote STT&lt;/strong&gt;: A 45 ms cloud final is excellent when the network is clean. Local models trade higher first-token latency and GPU/CPU load for privacy and offline use. Hybrid is common: local for the hot path, cloud for higher accuracy on finalized turns.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Overlay thread isolation&lt;/strong&gt;: The paint must never contend with the audio callback. A dedicated UI thread reading from a second lock-free queue keeps glitches out of the speakers.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Claude Meetings is a strong product for the meetings it targets. It does not replace the need for a careful Windows capture and render path when the goal is a live language overlay that works on any call.&lt;/p&gt;

&lt;p&gt;I'm building &lt;a href="https://talkaside.com/" rel="noopener noreferrer"&gt;Talkaside&lt;/a&gt;, a Windows app for job interviews and calls in another language.&lt;br&gt;
&lt;em&gt;Drafted with AI assistance from this project's code and logs.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>windows</category>
      <category>audio</category>
      <category>stt</category>
    </item>
    <item>
      <title>ZONOS3-Mini’s 80 ms TTFA and 15 Hz Codec — Dual-Track WASAPI Still Owns the Overlay Latency Budget</title>
      <dc:creator>devs</dc:creator>
      <pubDate>Sun, 15 Nov 2026 02:16:04 +0000</pubDate>
      <link>https://dev.to/madnessbrains_i/zonos3-minis-80-ms-ttfa-and-15-hz-codec-dual-track-wasapi-still-owns-the-overlay-latency-budget-23b9</link>
      <guid>https://dev.to/madnessbrains_i/zonos3-minis-80-ms-ttfa-and-15-hz-codec-dual-track-wasapi-still-owns-the-overlay-latency-budget-23b9</guid>
      <description>&lt;p&gt;On 12 November 2026 Zyphra released ZONOS3-Mini, an Apache-2.0 sparse MoE TTS model with roughly 800 M total parameters and only about 100 M active per token. It uses a custom 15 Hz dual-codec (ZAC-1) that packs 44.1 kHz audio into 10 codebooks of 4 096 entries (one semantic from Wav2Vec2-BERT plus nine acoustic), then renders frames with a causal multi-resolution iSTFT head. Reported numbers: 80 ms median time-to-first-audio on a single H100, under 150 ms on an M3 Air, under 570 ms in-browser via WebGPU on a Pixel 9, and peak throughput around 457× real-time. It ships 27 default voices across 43 languages and supports zero-shot cloning from a ≤60 s reference or a 192-d speaker embedding.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyllf2qo6wqjgk1wjim0v.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyllf2qo6wqjgk1wjim0v.png" alt="Basic waveforms illustrating the raw signal before engine processing" width="800" height="684"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Those numbers are real and useful for anyone trying to keep a full speech stack on-device. They do not move the first millisecond of a live Windows overlay that has to hear both sides of a call, turn the remote party’s audio into text, and optionally paint a short translated reply or suggestion before the next sentence finishes.&lt;/p&gt;

&lt;h2&gt;
  
  
  The capture and render tax still dominates
&lt;/h2&gt;

&lt;p&gt;A 100 M active TTS that can start emitting audio in under 100 ms is almost free once the text is ready. Getting clean, timed samples into the STT and then getting the synthesized reply out without glitches is not:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Endpoint choice — USB headset, Bluetooth HFP/A2DP, built-in, or exclusive-mode WASAPI.&lt;/li&gt;
&lt;li&gt;Dual-track acquisition: a dedicated capture client for the local mic plus a render-loopback client so the remote party’s audio stays distinct.&lt;/li&gt;
&lt;li&gt;Windows Audio Engine (&lt;code&gt;audiodg.exe&lt;/code&gt; + &lt;code&gt;audioeng.dll&lt;/code&gt;) in shared mode. Mixing, communications enhancements and vendor APOs routinely add 10–40+ ms and become non-deterministic under load.&lt;/li&gt;
&lt;li&gt;Consumer thread: lock-free SPSC ring copy, format conversion, resampling, VAD.&lt;/li&gt;
&lt;li&gt;Streaming STT (local GGUF or remote) plus optional local model polish for translation or short replies.&lt;/li&gt;
&lt;li&gt;Local TTS (ZONOS3-Mini via ggml/llama.cpp or a smaller model) writing into a render client.&lt;/li&gt;
&lt;li&gt;Overlay paint on a dedicated thread that never blocks the audio callback.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Bluetooth HFP alone can inject another 50–150 ms of buffering that no tokenizer or codec speed recovers. Even if the STT and TTS both run locally and finish in well under a frame, the overlay still waits for the Windows path that delivered the samples and the path that will play the reply.&lt;/p&gt;

&lt;p&gt;A minimal shared-mode capture callback that stays under 1 ms and does no allocation looks like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight c"&gt;&lt;code&gt;&lt;span class="c1"&gt;// WASAPI shared-mode capture — callback must be allocation-free&lt;/span&gt;
&lt;span class="kd"&gt;static&lt;/span&gt; &lt;span class="no"&gt;HRESULT&lt;/span&gt; &lt;span class="no"&gt;STDMETHODCALLTYPE&lt;/span&gt; &lt;span class="nc"&gt;CaptureCallback&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;IAudioCaptureClient&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="no"&gt;UINT32&lt;/span&gt; &lt;span class="n"&gt;packetLength&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;BYTE&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;DWORD&lt;/span&gt; &lt;span class="n"&gt;flags&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;HRESULT&lt;/span&gt; &lt;span class="n"&gt;hr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nc"&gt;GetBuffer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;flags&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;FAILED&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;hr&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;hr&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="c1"&gt;// Writer side of a pre-allocated power-of-two SPSC ring (float32 frames)&lt;/span&gt;
    &lt;span class="nf"&gt;ring_push_float32&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;capture_ring&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nc"&gt;ReleaseBuffer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="no"&gt;S_OK&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The consumer owns resampling (libsamplerate or a fixed polyphase), VAD, and the write to the model or WebSocket. Keep the audio thread pure; any lock or malloc here shows up as glitches or higher DPC latency.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Flw9sfsv55ditdwdt9l41.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Flw9sfsv55ditdwdt9l41.png" alt="Datapath of a typical audio DSP engine" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Dual-track still matters more than TTS speed
&lt;/h2&gt;

&lt;p&gt;A meeting bot that joins the call receives mixed audio after the conferencing platform has already done its own mixing and jitter buffering. A local overlay that needs speaker-separated, low-latency audio for an interview or support call does not have that luxury. You need:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Mic track: local speech (candidate or agent).&lt;/li&gt;
&lt;li&gt;Loopback track: remote party (interviewer or customer).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you keep them separate, you can send the remote-side track to STT with any agent text as context, and the mic track to a different stream or the same model with different biasing. Local models (faster-whisper, Whisper.cpp, or the smaller Nemotron/Granite speech checkpoints) avoid the network hop entirely. When the reply is short enough, you can feed it straight into a local TTS such as ZONOS3-Mini running through llama.cpp’s new &lt;code&gt;/v1/audio/speech&lt;/code&gt; endpoint or a custom ggml runner.&lt;/p&gt;

&lt;h2&gt;
  
  
  Trade-offs that actually show up in the wild
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Shared-mode WASAPI is convenient and works with every headset; exclusive mode or a virtual cable drops the engine tax but breaks system sounds and requires careful format negotiation.&lt;/li&gt;
&lt;li&gt;Bluetooth HFP gives you the mic and the remote audio on the same endpoint at the cost of 50–150 ms and mono 8/16 kHz. A2DP is better for playback but usually does not expose a clean loopback.&lt;/li&gt;
&lt;li&gt;Local GGUF STT + local TTS eliminates the round-trip but still has to share the CPU/GPU with the rest of the desktop. On a laptop under thermal limit the first token can still miss the 200 ms window that feels live.&lt;/li&gt;
&lt;li&gt;Overlay rendering must stay off the audio callback. A single UI thread that also does D2D or DirectComposition is fine; anything that blocks the message pump is not.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;ZONOS3-Mini (and the llama.cpp 1.0.0 &lt;code&gt;/v1/audio/speech&lt;/code&gt; surface that landed the same week) makes the generation side cheap enough that the remaining work is almost entirely in the Windows audio graph and the dual-track acquisition. That is the floor that still sets whether a language overlay feels instantaneous or just another delayed caption.&lt;/p&gt;

&lt;p&gt;I'm building &lt;a href="https://talkaside.com/" rel="noopener noreferrer"&gt;Talkaside&lt;/a&gt;, a Windows app for job interviews and calls in another language.&lt;br&gt;
&lt;em&gt;Drafted with AI assistance from this project's code and logs.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>windows</category>
      <category>audio</category>
      <category>tts</category>
      <category>ai</category>
    </item>
    <item>
      <title>LiquidCodec’s 361 Encode Rate on a Single CPU Thread — Dual-Track WASAPI Still Sets the Real Overlay Floor</title>
      <dc:creator>devs</dc:creator>
      <pubDate>Sat, 14 Nov 2026 13:24:39 +0000</pubDate>
      <link>https://dev.to/madnessbrains_i/liquidcodecs-361x-encode-rate-on-a-single-cpu-thread-dual-track-wasapi-still-sets-the-real-11gi</link>
      <guid>https://dev.to/madnessbrains_i/liquidcodecs-361x-encode-rate-on-a-single-cpu-thread-dual-track-wasapi-still-sets-the-real-11gi</guid>
      <description>&lt;p&gt;On 12 November 2026 Liquid AI published LiquidCodec, the ~74 M-parameter streaming audio tokenizer that sits under LFM3-Audio-2B and LFM3-Omni-2B. It turns 24 kHz speech into 12 discrete tokens per 80 ms frame (12.5 Hz) using grouped finite scalar quantization, then reconstructs the waveform. On a single Apple M4 Max CPU thread at FP32 the encoder runs at roughly 361× real time (RTF 0.0028) and the decoder at 87×. GGUF builds (Q4_0, Q4_K_M, Q8_0) are already available for llama.cpp. It also posts the best ViSQOL and lowest mel distance of the codecs they compared on eight reconstruction sets.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyllf2qo6wqjgk1wjim0v.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyllf2qo6wqjgk1wjim0v.png" alt="Basic waveforms illustrating the raw signal before engine processing" width="800" height="684"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That number is real and useful for anyone trying to keep a full speech stack on-device. It does not move the first millisecond of a live Windows overlay that has to hear both sides of a call, turn the remote party’s audio into text, and paint a translation or suggestion before the next sentence finishes.&lt;/p&gt;

&lt;h2&gt;
  
  
  The capture tax still dominates
&lt;/h2&gt;

&lt;p&gt;A 74 M codec that encodes in a few hundred microseconds per second of audio is almost free once the samples are in memory. Getting those samples cleanly and with stable timing is not:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Endpoint choice — USB headset, Bluetooth HFP/A2DP, built-in, or exclusive-mode WASAPI.&lt;/li&gt;
&lt;li&gt;Dual-track acquisition: a dedicated capture client for the local mic plus a render-loopback client so the remote party’s audio stays distinct.&lt;/li&gt;
&lt;li&gt;Windows Audio Engine (&lt;code&gt;audiodg.exe&lt;/code&gt; + &lt;code&gt;audioeng.dll&lt;/code&gt;) in shared mode. Mixing, communications enhancements and vendor APOs routinely add 10–40+ ms and become non-deterministic under load.&lt;/li&gt;
&lt;li&gt;Consumer thread: lock-free SPSC ring copy, format conversion, resampling, VAD.&lt;/li&gt;
&lt;li&gt;Streaming STT (local GGUF or remote) plus optional local model polish for translation or short replies.&lt;/li&gt;
&lt;li&gt;Overlay paint on a dedicated thread that never blocks the audio callback.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Bluetooth HFP alone can inject another 50–150 ms of buffering that no tokenizer speed recovers. Even if the codec and the language model both run locally and finish in well under a frame, the overlay still waits for the Windows path that delivered the samples.&lt;/p&gt;

&lt;p&gt;A minimal shared-mode capture callback that stays under 1 ms and does no allocation looks like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight c"&gt;&lt;code&gt;&lt;span class="c1"&gt;// WASAPI shared-mode capture — callback must be allocation-free&lt;/span&gt;
&lt;span class="kd"&gt;static&lt;/span&gt; &lt;span class="no"&gt;HRESULT&lt;/span&gt; &lt;span class="no"&gt;STDMETHODCALLTYPE&lt;/span&gt; &lt;span class="nc"&gt;CaptureCallback&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;IAudioCaptureClient&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="no"&gt;UINT32&lt;/span&gt; &lt;span class="n"&gt;packetLength&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;BYTE&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;DWORD&lt;/span&gt; &lt;span class="n"&gt;flags&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;HRESULT&lt;/span&gt; &lt;span class="n"&gt;hr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nc"&gt;GetBuffer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;flags&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;FAILED&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;hr&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;hr&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="c1"&gt;// Writer side of a pre-allocated power-of-two SPSC ring (float32 frames)&lt;/span&gt;
    &lt;span class="nf"&gt;ring_push_float32&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;capture_ring&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nc"&gt;ReleaseBuffer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="no"&gt;S_OK&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The consumer owns resampling (libsamplerate or a fixed polyphase), VAD, and the write to the model or WebSocket. Keep the audio thread pure; any lock or malloc here shows up as glitches or higher DPC latency.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Flw9sfsv55ditdwdt9l41.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Flw9sfsv55ditdwdt9l41.png" alt="Datapath of a typical audio DSP engine" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Dual-track still matters more than tokenizer speed
&lt;/h2&gt;

&lt;p&gt;A meeting bot that joins the call receives mixed audio after the conferencing platform has already done its own mixing and jitter buffering. A local overlay that needs speaker-separated, low-latency text has to start earlier. The remote-side track can be sent straight to STT (with or without the new codec as a front-end). The mic track can stay separate for local voice activity or for a second model that only listens for barge-in.&lt;/p&gt;

&lt;p&gt;LiquidCodec’s causal encoder and symmetric decoder make it a candidate for that front-end if you want to stay entirely on-device and still feed a small language model discrete tokens instead of raw spectrograms. The 12.5 Hz frame rate already matches the kind of chunking most streaming STT pipelines use. The Windows side of the budget does not change: you still need the two WASAPI clients, the lock-free rings, and an overlay that can paint without waiting for the next audio quantum.&lt;/p&gt;

&lt;h2&gt;
  
  
  Practical knobs that actually move the floor
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Prefer exclusive-mode WASAPI or a dedicated USB device when the call app allows it. Shared mode plus APOs is the common source of the extra 20–40 ms.&lt;/li&gt;
&lt;li&gt;Keep the capture callback allocation-free and under a millisecond. Profile with ETW + WPR if the overlay starts stuttering.&lt;/li&gt;
&lt;li&gt;Size the SPSC rings for the worst-case STT chunk plus one quantum of jitter, not for “whatever feels safe.”&lt;/li&gt;
&lt;li&gt;Run VAD and format conversion on the consumer thread so the audio engine never sees a slow callback.&lt;/li&gt;
&lt;li&gt;If you add LiquidCodec (or any other tokenizer) as a local front-end, pin it to the same process as the overlay only after you have measured that the capture path itself is stable. A 361× encoder is useless if the samples arrive 80 ms late.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The codec result is a real improvement for on-device speech pipelines. The part that still decides whether a language overlay feels instantaneous on a Windows laptop is the same dual-track WASAPI path it has been for the last several years.&lt;/p&gt;

&lt;p&gt;I'm building &lt;a href="https://talkaside.com/" rel="noopener noreferrer"&gt;Talkaside&lt;/a&gt;, a Windows app for job interviews and calls in another language.&lt;br&gt;
&lt;em&gt;Drafted with AI assistance from this project's code and logs.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>windows</category>
      <category>audio</category>
      <category>stt</category>
      <category>ai</category>
    </item>
    <item>
      <title>Claude Meetings Beta Drops into Google Meet — Your Local Audio Path Still Decides Language Latency</title>
      <dc:creator>devs</dc:creator>
      <pubDate>Sat, 14 Nov 2026 10:14:05 +0000</pubDate>
      <link>https://dev.to/madnessbrains_i/claude-meetings-beta-drops-into-google-meet-your-local-audio-path-still-decides-language-latency-1lhg</link>
      <guid>https://dev.to/madnessbrains_i/claude-meetings-beta-drops-into-google-meet-your-local-audio-path-still-decides-language-latency-1lhg</guid>
      <description>&lt;p&gt;On 12 November 2026 Anthropic rolled out Claude Meetings in beta. Claude now joins Google Meet calls as a visible participant, follows the conversation, answers questions out loud or in chat (replies in writing by default), and can draft documents, whiteboards or other artifacts while people are still talking. Notes, transcripts and most in-call answers stay free during the beta; created artifacts are free through 10 February 2027. Support starts on Google Meet for Team and Enterprise plans, with other platforms promised later. It keeps a transcript and screen descriptions, not the raw audio.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyllf2qo6wqjgk1wjim0v.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyllf2qo6wqjgk1wjim0v.png" alt="Basic waveforms illustrating the raw signal before engine processing" width="800" height="684"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That is useful for teams that already live in Meet and want an extra participant that can look things up or start a doc. It does not solve the problem of a developer or candidate sitting in a call conducted in a language they do not speak fluently. For that use case the interesting path is still the local Windows client that captures both sides of the call, runs STT, and paints a translation or suggestion overlay with as little added latency as possible.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the audio engine still owns the budget
&lt;/h2&gt;

&lt;p&gt;A meeting bot that joins the call receives mixed audio after the conferencing platform has already done its own mixing, jitter buffering and processing. A local overlay that needs speaker-separated, low-latency text has to start earlier:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Capture endpoint (USB headset, Bluetooth HFP/A2DP, built-in, or exclusive-mode WASAPI).&lt;/li&gt;
&lt;li&gt;Separate render-loopback track so the remote party's audio stays distinct from the local microphone.&lt;/li&gt;
&lt;li&gt;Windows Audio Engine (&lt;code&gt;audiodg.exe&lt;/code&gt; + &lt;code&gt;audioeng.dll&lt;/code&gt;) in shared mode. Mixing, communications enhancements and vendor APOs routinely add 10–40+ ms and become non-deterministic under load or Bluetooth.&lt;/li&gt;
&lt;li&gt;Consumer thread: lock-free SPSC ring copy, format conversion, resampling, VAD.&lt;/li&gt;
&lt;li&gt;Streaming STT (local GGUF or remote) plus optional local model polish for translation or short replies.&lt;/li&gt;
&lt;li&gt;Overlay paint on a dedicated thread that never blocks the audio callback.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Even if Claude (or any other meeting participant) can produce text quickly once it has the audio, the local path that feeds a personal language overlay still pays the full capture tax. Bluetooth HFP alone can inject another 50–150 ms of buffering that no model improvement recovers.&lt;/p&gt;

&lt;p&gt;A minimal shared-mode capture callback that stays under 1 ms and does no allocation looks like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight c"&gt;&lt;code&gt;&lt;span class="c1"&gt;// WASAPI shared-mode capture — callback must be allocation-free&lt;/span&gt;
&lt;span class="kd"&gt;static&lt;/span&gt; &lt;span class="no"&gt;HRESULT&lt;/span&gt; &lt;span class="no"&gt;STDMETHODCALLTYPE&lt;/span&gt; &lt;span class="nc"&gt;CaptureCallback&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;IAudioCaptureClient&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="no"&gt;UINT32&lt;/span&gt; &lt;span class="n"&gt;packetLength&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;BYTE&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;DWORD&lt;/span&gt; &lt;span class="n"&gt;flags&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;HRESULT&lt;/span&gt; &lt;span class="n"&gt;hr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nc"&gt;GetBuffer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;flags&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;FAILED&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;hr&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;hr&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="c1"&gt;// Writer side of a pre-allocated power-of-two SPSC ring (float32 frames)&lt;/span&gt;
    &lt;span class="nf"&gt;ring_push_float32&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;capture_ring&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nc"&gt;ReleaseBuffer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="no"&gt;S_OK&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The consumer owns resampling (libsamplerate or a fixed polyphase), VAD, and the write to the model or WebSocket. Keep the audio thread pure; any lock or malloc here shows up as glitches or higher DPC latency.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Flw9sfsv55ditdwdt9l41.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Flw9sfsv55ditdwdt9l41.png" alt="Datapath of a typical audio DSP engine" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Dual-track still matters more than the bot
&lt;/h2&gt;

&lt;p&gt;Meeting platforms mix everything into one stream for the bot. A local tool that wants accurate speaker labels (especially when the remote party has a different accent or language) must keep the tracks separate until after STT. Mixing early destroys identity and forces the recognizer to handle two voices in one beam.&lt;/p&gt;

&lt;p&gt;Practical controls that still move the needle:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Request the smallest period the driver will honor (often 10 ms shared-mode; exclusive can go lower when the device allows it).&lt;/li&gt;
&lt;li&gt;Prefer exclusive mode on the capture side when compatibility allows; shared mode is easier but inherits every APO the OEM enabled.&lt;/li&gt;
&lt;li&gt;Keep thinking/budget low for any online suggestion path; reserve higher compute for offline prep (job-description summary, question bank).&lt;/li&gt;
&lt;li&gt;Dual independent rings so mic and loopback never mix early.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Local models (faster-whisper, Whisper.cpp, or smaller speech checkpoints) avoid the network hop entirely once the audio has left the engine. Hybrid is common: local for the hot path, cloud for higher accuracy on finalized turns.&lt;/p&gt;

&lt;h2&gt;
  
  
  Trade-offs that remain
&lt;/h2&gt;

&lt;p&gt;Claude Meetings is a server-side participant that sees the already-mixed call. A local Windows overlay sees the raw endpoints and can decide what to send where. The first approach is convenient for teams that already pay for the platform and want shared artifacts. The second is what keeps latency predictable when the goal is personal real-time help during a high-stakes call conducted in another language.&lt;/p&gt;

&lt;p&gt;The 1.0.0-era local inference stacks (llama.cpp, ggml, and the various speech GGUFs) make the model side more stable. They do not move the first millisecond. That millisecond is still spent inside the audio engine.&lt;/p&gt;

&lt;p&gt;I'm building &lt;a href="https://talkaside.com/" rel="noopener noreferrer"&gt;Talkaside&lt;/a&gt;, a Windows app for job interviews and calls in another language.&lt;br&gt;
&lt;em&gt;Drafted with AI assistance from this project's code and logs.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>windows</category>
      <category>audio</category>
      <category>stt</category>
      <category>ai</category>
    </item>
    <item>
      <title>llama.cpp 1.0.0 Stabilizes the Local Inference API — Dual-Track WASAPI Capture Still Owns the Latency Budget</title>
      <dc:creator>devs</dc:creator>
      <pubDate>Sat, 14 Nov 2026 04:15:09 +0000</pubDate>
      <link>https://dev.to/madnessbrains_i/llamacpp-100-stabilizes-the-local-inference-api-dual-track-wasapi-capture-still-owns-the-4a1h</link>
      <guid>https://dev.to/madnessbrains_i/llamacpp-100-stabilizes-the-local-inference-api-dual-track-wasapi-capture-still-owns-the-4a1h</guid>
      <description>&lt;p&gt;On 13 November 2026 the ggml-org team released llama.cpp v1.0.0 and ggml v1.0.0. After 3.5 years the libraries now follow explicit semantic versioning: a MAJOR bump signals a breaking change to the public C API in &lt;code&gt;./include&lt;/code&gt;, MINOR covers backward-compatible features and new model support, PATCH is pure bug fixes. The release removes 35 deprecated functions (plus &lt;code&gt;llama_set_warmup&lt;/code&gt; and the defrag threshold), promotes the NextN speculative API into &lt;code&gt;llama.h&lt;/code&gt;, adds OCP FP8 KV caches, a host-memory MoE expert cache with configurable CPU share, and hardens &lt;code&gt;llama-server&lt;/code&gt; against SSRF, CORS and DNS-rebinding. It also ships drama-3 TTS support and a &lt;code&gt;/v1/audio/speech&lt;/code&gt; endpoint.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fz4l0mmc20xkzoaeeltqj.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fz4l0mmc20xkzoaeeltqj.png" alt="llama.cpp v1.0.0 announcement" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The stability promise matters if you embed local models for post-STT translation polish, short reply generation, or on-device transcription. It does not move the first millisecond of a live Windows overlay. That millisecond is still spent inside the audio engine.&lt;/p&gt;

&lt;h2&gt;
  
  
  The capture path is still the dominant term
&lt;/h2&gt;

&lt;p&gt;A dual-track pipeline for a call or interview assistant separates the local microphone from the remote party:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Capture endpoint (USB headset, Bluetooth HFP, built-in, or exclusive-mode WASAPI).&lt;/li&gt;
&lt;li&gt;Render loopback on the same or another endpoint so speaker labels stay intact.&lt;/li&gt;
&lt;li&gt;Windows Audio Engine (&lt;code&gt;audiodg.exe&lt;/code&gt; + &lt;code&gt;audioeng.dll&lt;/code&gt;) in shared mode. Mixing, communications enhancements and vendor APOs routinely add 10–40+ ms and become non-deterministic under load or Bluetooth.&lt;/li&gt;
&lt;li&gt;Your consumer thread: lock-free SPSC ring copy, float32→int16, 48 kHz→16 kHz resample, energy or Silero-style VAD.&lt;/li&gt;
&lt;li&gt;Streaming STT (local GGUF or remote WebSocket) or direct local model call.&lt;/li&gt;
&lt;li&gt;Overlay paint on a dedicated thread.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Even with FP8 KV and a warm context, the model’s contribution is downstream of steps 1–4. Bluetooth HFP alone can inject another 50–150 ms of buffering that no quantization scheme recovers.&lt;/p&gt;

&lt;p&gt;A minimal shared-mode capture callback that stays under 1 ms and does no allocation:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight c"&gt;&lt;code&gt;&lt;span class="c1"&gt;// WASAPI shared-mode capture — callback must be allocation-free&lt;/span&gt;
&lt;span class="kd"&gt;static&lt;/span&gt; &lt;span class="no"&gt;HRESULT&lt;/span&gt; &lt;span class="no"&gt;STDMETHODCALLTYPE&lt;/span&gt; &lt;span class="nc"&gt;CaptureCallback&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;IAudioCaptureClient&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="no"&gt;UINT32&lt;/span&gt; &lt;span class="n"&gt;packetLength&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;BYTE&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;DWORD&lt;/span&gt; &lt;span class="n"&gt;flags&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;HRESULT&lt;/span&gt; &lt;span class="n"&gt;hr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nc"&gt;GetBuffer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;flags&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;FAILED&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;hr&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;hr&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="c1"&gt;// Writer side of a pre-allocated power-of-two SPSC ring (float32 frames)&lt;/span&gt;
    &lt;span class="nf"&gt;ring_push_float32&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;capture_ring&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="kt"&gt;float&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nc"&gt;ReleaseBuffer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="no"&gt;S_OK&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The consumer owns resampling (libsamplerate or a fixed polyphase), VAD, and the write to the model or WebSocket. Keep the audio thread pure; any lock or malloc here shows up as glitches or higher DPC latency.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyllf2qo6wqjgk1wjim0v.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyllf2qo6wqjgk1wjim0v.png" alt="Basic waveforms illustrating the raw signal before engine processing" width="800" height="684"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Local models and the new speech endpoint
&lt;/h2&gt;

&lt;p&gt;The drama-3 TTS addition and the &lt;code&gt;/v1/audio/speech&lt;/code&gt; endpoint are useful for offline generation or for feeding synthetic reference audio into residual-echo experiments. For the live input path they are irrelevant. If you are running a local STT GGUF or a small translation model after the recognizer, the 1.0.0 API cleanup removes one class of future breakage, but the measured time-to-final on the model side is still measured after the audio has left the shared-mode engine.&lt;/p&gt;

&lt;p&gt;Practical control surfaces that do help:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Keep thinking/budget low for any online suggestion path; reserve higher compute for offline prep (job-description summary, question bank).&lt;/li&gt;
&lt;li&gt;Dual independent rings so mic and loopback never mix early. Mixing destroys speaker identity and forces the recognizer to handle two accents in one beam.&lt;/li&gt;
&lt;li&gt;Request the smallest period the driver will honor (often 10 ms default; some allow 5–8 ms). Below that the engine starts dropping or the APO stack becomes unstable.&lt;/li&gt;
&lt;li&gt;Overlay exclusion so screen-share does not capture the helper window — a Windows-specific problem that no model release solves.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Trade-offs that remain
&lt;/h2&gt;

&lt;p&gt;FP8 KV caches shrink memory and can improve tokens/second on some backends, but the first token of a short rewrite is still gated by network (if remote) or by the CPU/GPU scheduling of the local process. The host-memory MoE cache is interesting for larger local models; for a 1–3 B parameter translation or suggestion model it is usually unnecessary. The real variable cost on a laptop during a call is the audio engine’s interaction with Bluetooth, spatial audio, or vendor “communications” enhancements — none of which appear in a llama.cpp changelog.&lt;/p&gt;

&lt;p&gt;If you are shipping a desktop tool that shows text during a call, measure end-to-end from speech end (or VAD endpoint) to on-screen glyphs. The model’s TTFT is only one line item. The WASAPI period, the ring copy, the resample, and the paint are the others, and they do not get smaller because the inference library reached 1.0.0.&lt;/p&gt;

&lt;p&gt;I'm building &lt;a href="https://talkaside.com/" rel="noopener noreferrer"&gt;Talkaside&lt;/a&gt;, a Windows app for job interviews and calls in another language.&lt;br&gt;
&lt;em&gt;Drafted with AI assistance from this project's code and logs.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>llamacpp</category>
      <category>windows</category>
      <category>audio</category>
      <category>localai</category>
    </item>
    <item>
      <title>Gemini 4 Hydrogen Ships for Agentic Loops — Cloud TTFT Still Loses to Local Capture on Windows Calls</title>
      <dc:creator>devs</dc:creator>
      <pubDate>Fri, 13 Nov 2026 11:17:33 +0000</pubDate>
      <link>https://dev.to/madnessbrains_i/gemini-4-hydrogen-ships-for-agentic-loops-cloud-ttft-still-loses-to-local-capture-on-windows-calls-5ec6</link>
      <guid>https://dev.to/madnessbrains_i/gemini-4-hydrogen-ships-for-agentic-loops-cloud-ttft-still-loses-to-local-capture-on-windows-calls-5ec6</guid>
      <description>&lt;p&gt;Google dropped Gemini 4 Hydrogen on 12 November 2026. It is the speed model in the Gemini 4 family, aimed at rapid agentic execution rather than the long-horizon reasoning of Carbon or Argon. Pricing is flat at $1.50 per million input tokens and $7.50 per million output tokens across the full 1 048 576-token unified context window. Thinking levels are low, medium, high (default) and a new xhigh. Google claims 2.5× faster task completion than Gemini 3.8 Flash at the high setting, a Terminal-Bench 4.1 score of 57 % versus 18.2 %, and more than 5× fewer fabricated claims. It is live in the Gemini API, AI Studio (all tiers), Search, and for Pro/Ultra subscribers in Antigravity.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgxpc1cr1lrwrggo4tkz8.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgxpc1cr1lrwrggo4tkz8.jpg" alt="Gemini 4 Hydrogen key art" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;That is useful for coding agents and tool loops. It is less useful if the first thing the user sees is a 150–300 ms overlay that already burned most of its budget before the model even started thinking.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where the budget actually goes on a Windows call
&lt;/h2&gt;

&lt;p&gt;A practical dual-track pipeline for a live interview or call assistant looks like this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Microphone endpoint (USB headset, Bluetooth HFP, built-in, or exclusive-mode WASAPI).&lt;/li&gt;
&lt;li&gt;Render loopback on the same or another endpoint so you can separate the remote party.&lt;/li&gt;
&lt;li&gt;Windows Audio Engine (audiodg.exe + audioeng.dll) in shared mode: mixing, communications enhancements, vendor APOs. This alone is 10–40+ ms and is fragile under load or with Bluetooth.&lt;/li&gt;
&lt;li&gt;Your capture callback or consumer thread: copy into lock-free SPSC rings, convert float32 → int16, resample 48 kHz → 16 kHz, lightweight VAD.&lt;/li&gt;
&lt;li&gt;Streaming STT WebSocket (20–40 ms chunks).&lt;/li&gt;
&lt;li&gt;Optional LLM call for suggestion, translation polish, or next-question generation.&lt;/li&gt;
&lt;li&gt;Overlay paint on a dedicated thread (&amp;lt;16 ms if you stay off the UI thread).&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Hydrogen’s own time-to-first-token at high or xhigh is still measured in tens to hundreds of milliseconds once you add network RTT, serialization, and the fact that thinking tokens are billed. On a machine that is already running the audio engine, a browser, and the interview platform, the capture path is the larger and more variable term. Bluetooth HFP in particular can add another 50–150 ms of buffering that no model can claw back.&lt;/p&gt;

&lt;p&gt;A minimal capture skeleton that keeps the callback under 1 ms:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight c"&gt;&lt;code&gt;&lt;span class="c1"&gt;// WASAPI shared-mode capture callback — no allocations, no locks&lt;/span&gt;
&lt;span class="kd"&gt;static&lt;/span&gt; &lt;span class="no"&gt;HRESULT&lt;/span&gt; &lt;span class="no"&gt;STDMETHODCALLTYPE&lt;/span&gt; &lt;span class="nc"&gt;CaptureCallback&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;IAudioCaptureClient&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="no"&gt;UINT32&lt;/span&gt; &lt;span class="n"&gt;packetLength&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;BYTE&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;DWORD&lt;/span&gt; &lt;span class="n"&gt;flags&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="no"&gt;HRESULT&lt;/span&gt; &lt;span class="n"&gt;hr&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nc"&gt;GetBuffer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;flags&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;FAILED&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;hr&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;hr&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="c1"&gt;// Push raw float32 frames into pre-allocated power-of-two SPSC ring&lt;/span&gt;
    &lt;span class="c1"&gt;// (writer side only — consumer runs on another thread)&lt;/span&gt;
    &lt;span class="nf"&gt;ring_push_float32&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;capture_ring&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;float&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="n"&gt;pData&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="n"&gt;pCapture&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nc"&gt;ReleaseBuffer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;packetLength&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="no"&gt;S_OK&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The consumer does the heavy lifting: resampling (e.g. libsamplerate or a simple polyphase), energy-based VAD, and the WebSocket write. Keep the audio callback pure.&lt;/p&gt;

&lt;h2&gt;
  
  
  BYOK and thinking levels for the overlay path
&lt;/h2&gt;

&lt;p&gt;Hydrogen’s four thinking levels give a usable control surface. For a live overlay you almost always want low or medium:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;low: fastest, least reasoning, cheapest.&lt;/li&gt;
&lt;li&gt;medium: decent tool-calling and short rewrites.&lt;/li&gt;
&lt;li&gt;high / xhigh: only for offline prep (summarizing a job description, generating question banks, parsing a scraped company page).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Because it is BYOK via the Gemini API, you can route the hot path to a cheaper or faster model and the cold path to Hydrogen xhigh without changing the rest of the pipeline. A simple router:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;choose_model&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;task&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;latency_budget_ms&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;latency_budget_ms&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mi"&gt;200&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;task&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"partial_translate"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"short_suggestion"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="s2"&gt;"gemini-4-hydrogen"&lt;/span&gt;  &lt;span class="c1"&gt;# with thinking_level=low&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;task&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"jd_summary"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"question_bank"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="s2"&gt;"gemini-4-hydrogen"&lt;/span&gt;  &lt;span class="c1"&gt;# thinking_level=xhigh&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="s2"&gt;"local-qwen3-4b"&lt;/span&gt;  &lt;span class="c1"&gt;# fallback when offline or privacy required&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Local models still win for the absolute lowest latency and for anything that must stay on-device. Hydrogen wins when you need the 1 M context or the better tool-calling reliability on multi-step agent loops (scraping a careers page, extracting requirements, generating a tailored answer outline). The tradeoff is network dependency and the fact that every thinking token costs money.&lt;/p&gt;

&lt;h2&gt;
  
  
  Architecture notes and common failure modes
&lt;/h2&gt;

&lt;p&gt;Separate the two audio tracks all the way to the transcript store. Mixing early destroys speaker labels and forces the recognizer to handle two languages or two accents in one beam. On Windows the correct way is still “open an input stream on a render device” via WASAPI for the loopback side.&lt;/p&gt;

&lt;p&gt;Rate-limit and retry logic matters. Hydrogen is new; the API will return 429s under load. Exponential backoff with jitter, plus a local queue of the last N partials, keeps the overlay from freezing. If the WebSocket dies, fall back to the last good local model or simply show the raw STT partials.&lt;/p&gt;

&lt;p&gt;Capture-side pitfalls that still bite:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Default shared-mode period of 10 ms is fine; requesting 5 ms or lower is driver-dependent and often fails on Bluetooth.&lt;/li&gt;
&lt;li&gt;Vendor audio enhancements (noise suppression, AEC) fight your own AEC and add variable latency. Disable them in the endpoint properties or via IAudioEndpointVolume / APO control when possible.&lt;/li&gt;
&lt;li&gt;DPC latency spikes from bad drivers still show up in LatencyMon or Process Lasso’s LatencyTest. They turn a 40 ms buffer into 80–120 ms of jitter.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fsfbq6dv9mdpn25iwd83m.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fsfbq6dv9mdpn25iwd83m.jpg" alt="Abstract generative art used as visual stand-in for model architecture" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Hydrogen is a genuine step forward for agentic coding and tool use. On the live call path the model is rarely the bottleneck. The dual-track capture, the Windows audio engine, and the decision of which thinking level to pay for still dominate. Treat the LLM as a controllable, optional stage after a solid low-latency audio foundation and the overlay stays responsive.&lt;/p&gt;

&lt;p&gt;I'm building &lt;a href="https://talkaside.com/" rel="noopener noreferrer"&gt;Talkaside&lt;/a&gt;, a Windows app for job interviews and calls in another language.&lt;br&gt;
&lt;em&gt;Drafted with AI assistance from this project's code and logs.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>windows</category>
      <category>audio</category>
      <category>llm</category>
    </item>
    <item>
      <title>Google Open-Sources XSE: 6 Million Queries a Second Without Python in the Path</title>
      <dc:creator>devs</dc:creator>
      <pubDate>Fri, 13 Nov 2026 04:16:08 +0000</pubDate>
      <link>https://dev.to/madnessbrains_i/google-open-sources-xse-6-million-queries-a-second-without-python-in-the-path-3g92</link>
      <guid>https://dev.to/madnessbrains_i/google-open-sources-xse-6-million-queries-a-second-without-python-in-the-path-3g92</guid>
      <description>&lt;p&gt;On 12 November 2026 Google open-sourced XSE, the XLA Serving Engine that powers six million queries a second across Search, Ads and YouTube Shorts. The point of the project is simple and brutal: take the Python interpreter completely out of the serving path so the request never touches CPython at all.&lt;/p&gt;

&lt;p&gt;That is the correct direction for any system that lives under a hard latency budget. It does not solve the problem when the input is live audio on a Windows desktop and the output is a private translation overlay that must stay invisible to screen share.&lt;/p&gt;

&lt;h2&gt;
  
  
  What XSE actually removes
&lt;/h2&gt;

&lt;p&gt;XSE sits under the model server and replaces the Python runtime that normally owns the request loop. The published material says it handles six million queries a second in production. The engineering claim is that once the model graph is compiled, the interpreter is pure overhead—reference counting, GIL, object allocation, and the general mess of a dynamic language sitting on the hot path.&lt;/p&gt;

&lt;p&gt;For a cloud serving fleet that is true. For a single Windows machine that needs to capture two audio streams, run a local or hybrid STT model, and paint a translation that the other side of the call never sees, the Python interpreter is not the dominant cost. The WASAPI shared-mode engine, the capture callback, and the time spent loading or warming a local model are.&lt;/p&gt;

&lt;h2&gt;
  
  
  The capture path that still owns the budget
&lt;/h2&gt;

&lt;p&gt;A practical overlay for interviews or support calls still starts with dual-track WASAPI:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;One &lt;code&gt;IAudioClient&lt;/code&gt; in shared mode on the default render endpoint (or the endpoint the call app is using) with &lt;code&gt;AUDCLNT_STREAMFLAGS_LOOPBACK&lt;/code&gt;. This is the remote party.&lt;/li&gt;
&lt;li&gt;One &lt;code&gt;IAudioClient&lt;/code&gt; in shared mode on the preferred capture endpoint. This is your microphone.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Both clients request a format the engine can actually mix (usually 48 kHz float32 or whatever &lt;code&gt;GetMixFormat&lt;/code&gt; returns). Periodicity around 10 ms is a reasonable target; the engine will quantize to the mix format anyway.&lt;/p&gt;

&lt;p&gt;The rule that matters is that &lt;code&gt;IAudioCaptureClient::GetBuffer&lt;/code&gt; must return immediately. Copy the packet into a lock-free ring buffer and signal a worker. Any allocation, logging, or model call inside the callback adds jitter that shows up as gaps or increased end-to-end latency.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F8puvd24gcfwmwowyqqtz.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F8puvd24gcfwmwowyqqtz.png" alt="PyTorch logo" width="800" height="160"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Align the two streams by device clock or a shared monotonic timestamp. Drift is inevitable; you correct it by dropping or duplicating a few frames on the slower side rather than resampling in the hot path unless you have measured that the quality difference is worth the cost.&lt;/p&gt;

&lt;p&gt;From there the remote track goes to STT. The mic track can go to a separate stream, to a local model for your own speech, or simply be used for energy-based barge-in detection. The translation result is painted on a layered window excluded from screen capture with &lt;code&gt;SetWindowDisplayAffinity(WDA_EXCLUDEFROMCAPTURE)&lt;/code&gt; on Windows 10+.&lt;/p&gt;

&lt;h2&gt;
  
  
  Local models versus cloud BYOK
&lt;/h2&gt;

&lt;p&gt;Local models (faster-whisper, Whisper.cpp, Ollama with LFM3-audio or similar small audio checkpoints) remove the network hop and keep the audio on the machine. The tradeoff is load time, VRAM or RAM pressure, and accuracy on accented or noisy speech. Cloud endpoints (BYOK to whatever provider you already pay for) give better accuracy and diarization on clean audio, but they add RTT plus the model’s own time-to-final, and they stop working when the network is hostile or when you simply refuse to ship the audio off-box.&lt;/p&gt;

&lt;p&gt;A practical hybrid keeps a small local model warm for the common case and falls back to a cloud endpoint only when the local confidence drops or when the user explicitly chooses the higher-quality path. Rate limits on the cloud side become a real problem once you are doing continuous streaming rather than occasional requests; the usual mitigations are token-bucket pacing, request coalescing, and having more than one provider key so a single 429 does not kill the overlay.&lt;/p&gt;

&lt;h2&gt;
  
  
  Latency budget that actually matters
&lt;/h2&gt;

&lt;p&gt;A 150 ms cloud time-to-final is interesting only after you have already spent:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;10–40 ms in the shared-mode WASAPI engine and buffering (measured, not guessed).&lt;/li&gt;
&lt;li&gt;Packet alignment and any small resampling.&lt;/li&gt;
&lt;li&gt;Network RTT + model compute for the cloud path.&lt;/li&gt;
&lt;li&gt;Translation and UI paint.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Local models trade the network hop for a longer first-token time and higher memory use. The right number is the one you measure on the machine you actually ship, with the same audio devices and the same process priority as production.&lt;/p&gt;

&lt;h2&gt;
  
  
  Practical notes from shipping this
&lt;/h2&gt;

&lt;p&gt;Keep the capture callback empty. Use a single-producer single-consumer ring buffer (or two, one per track). Timestamp every packet with a shared QPC or device clock. Prefer shared mode so the rest of the machine keeps working; exclusive mode gives lower latency but breaks every other app that wants to play sound. Test with the actual call app (Teams, Zoom, Meet, Discord) because process-loopback capture behaves differently across them.&lt;/p&gt;

&lt;p&gt;The XSE news is a reminder that removing interpreter overhead is valuable. On a desktop the interpreter is not the first thing that burns your latency budget. The capture path and the model load are.&lt;/p&gt;

&lt;p&gt;I'm building &lt;a href="https://talkaside.com/" rel="noopener noreferrer"&gt;Talkaside&lt;/a&gt;, a Windows app for job interviews and calls in another language.&lt;br&gt;
&lt;em&gt;Drafted with AI assistance from this project's code and logs.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>windows</category>
      <category>performance</category>
      <category>ai</category>
      <category>audio</category>
    </item>
    <item>
      <title>ElevenLabs Scribe v3 Live Cuts Semantic Errors 43% — Dual-Track WASAPI Still Sets the Latency Floor</title>
      <dc:creator>devs</dc:creator>
      <pubDate>Thu, 12 Nov 2026 07:20:00 +0000</pubDate>
      <link>https://dev.to/madnessbrains_i/elevenlabs-scribe-v3-live-cuts-semantic-errors-43-dual-track-wasapi-still-sets-the-latency-floor-2311</link>
      <guid>https://dev.to/madnessbrains_i/elevenlabs-scribe-v3-live-cuts-semantic-errors-43-dual-track-wasapi-still-sets-the-latency-floor-2311</guid>
      <description>&lt;p&gt;On 10 November 2026 ElevenLabs shipped Scribe v3 and Scribe v3 Live. The batch model claims 27% fewer word errors than Scribe v2 across 102 languages. The realtime variant, Scribe v3 Live, claims 43% lower semantic word-error rate versus the previous realtime model, first-time realtime speaker diarization, context-aware turn detection instead of pure silence VAD, and mid-sentence language switching. Pricing stayed flat at roughly $0.22 per audio hour for the live endpoint. Model IDs are &lt;code&gt;scribe_v3&lt;/code&gt; and &lt;code&gt;scribe_v3_live&lt;/code&gt;; the WebSocket lives at the usual realtime speech-to-text endpoint. Latency numbers in the public material hover around the 150 ms class that has become the marketing floor for cloud streaming STT.&lt;/p&gt;

&lt;p&gt;Those gains are real once clean PCM is already at the model. They do not move the capture budget on a Windows desktop when you need a private translation overlay for an interview or support call.&lt;/p&gt;

&lt;h2&gt;
  
  
  What a better cloud STT still cannot see
&lt;/h2&gt;

&lt;p&gt;Scribe v3 Live (and every other hosted streaming model) receives the stream you send it. It does not sit on your local audio endpoints. It does not give you a second, private track of your own microphone that never enters the call. It does not let you paint a translation overlay that is invisible to screen share. And it does not run when the network is hostile or when you simply refuse to ship the audio off-box.&lt;/p&gt;

&lt;p&gt;The practical path for a language-aid overlay is still dual-track WASAPI capture: one client on the loopback/render device that hears the remote party, one client on the capture device that hears you, both in shared mode so the rest of the machine keeps working, both with the callback kept empty so you do not add scheduling jitter.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fz94ggw0de0w6zy63lxfg.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fz94ggw0de0w6zy63lxfg.png" alt="Frequency and amplitude diagram" width="551" height="328"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Dual-track capture architecture
&lt;/h2&gt;

&lt;p&gt;Open two &lt;code&gt;IAudioClient&lt;/code&gt; instances:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Loopback on the default render endpoint (or the endpoint the call app is using). This is the remote audio.&lt;/li&gt;
&lt;li&gt;Capture on the default (or preferred) input endpoint. This is your microphone.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Both in shared mode (&lt;code&gt;AUDCLNT_SHAREMODE_SHARED&lt;/code&gt;). Periodicity around 10 ms is a reasonable target; the engine will quantize to the mix format. Use &lt;code&gt;AUDCLNT_STREAMFLAGS_LOOPBACK&lt;/code&gt; only on the render client. Request a format you can actually deliver (usually 48 kHz float32 or the device mix format via &lt;code&gt;GetMixFormat&lt;/code&gt;).&lt;/p&gt;

&lt;p&gt;The capture callback (&lt;code&gt;IAudioCaptureClient::GetBuffer&lt;/code&gt;) must return immediately. Copy the packet into a lock-free ring buffer and signal a worker. Any work inside the callback—allocation, logging, STT submission—adds jitter that shows up as gaps or increased latency.&lt;/p&gt;

&lt;p&gt;Align the two streams by device clock or by a shared monotonic timestamp. Drift is inevitable; you correct it by dropping or duplicating a few frames on the slower side rather than resampling in the hot path unless you have a measured need.&lt;/p&gt;

&lt;p&gt;From there the remote track goes to STT (cloud or local). The mic track can go to a separate STT stream, to a local model for your own speech, or simply be used for energy-based barge-in detection. The translation result is painted on a layered window that is excluded from screen capture (&lt;code&gt;SetWindowDisplayAffinity&lt;/code&gt; with &lt;code&gt;WDA_EXCLUDEFROMCAPTURE&lt;/code&gt; on Windows 10+).&lt;/p&gt;

&lt;h2&gt;
  
  
  Latency budget that actually matters
&lt;/h2&gt;

&lt;p&gt;A 150 ms cloud STT time-to-final is interesting only after you have already spent:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;10–40 ms in the shared-mode WASAPI engine and buffering (measured, not guessed).&lt;/li&gt;
&lt;li&gt;Packet alignment and any small resampling.&lt;/li&gt;
&lt;li&gt;Network RTT + model compute for the cloud path.&lt;/li&gt;
&lt;li&gt;Translation and UI paint.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Local models (faster-whisper, Whisper.cpp, Qwen3-ASR GGUF variants, LFM3-audio through Ollama, etc.) remove the network hop and keep the audio on-device. They trade accuracy on hard accents or domain terms for lower and more predictable latency. Hybrid is common: local for the hot path and partials, cloud for finalized turns when the network is clean.&lt;/p&gt;

&lt;p&gt;Scribe v3 Live’s new context-aware turn detection and realtime diarization are useful when you are already sending a single mixed stream. They do not replace the ability to keep the two tracks separate so you can bias one stream with the remote party’s language and the other with yours, or so you can run different models on each side.&lt;/p&gt;

&lt;h2&gt;
  
  
  Code sketch (WASAPI loopback + capture)
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cpp"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Pseudocode outline — real code needs COM init, error handling, format negotiation&lt;/span&gt;
&lt;span class="nc"&gt;IAudioClient&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;loopbackClient&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;nullptr&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nc"&gt;IAudioClient&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;captureClient&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;nullptr&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="c1"&gt;// ... ActivateAudioInterface / GetDefaultAudioEndpoint ...&lt;/span&gt;

&lt;span class="c1"&gt;// Loopback&lt;/span&gt;
&lt;span class="n"&gt;loopbackClient&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nc"&gt;Initialize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="no"&gt;AUDCLNT_SHAREMODE_SHARED&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="no"&gt;AUDCLNT_STREAMFLAGS_LOOPBACK&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="no"&gt;AUDCLNT_STREAMFLAGS_EVENTCALLBACK&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;hnsPeriod&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;mixFormat&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nullptr&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;// Capture&lt;/span&gt;
&lt;span class="n"&gt;captureClient&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nc"&gt;Initialize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="no"&gt;AUDCLNT_SHAREMODE_SHARED&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="no"&gt;AUDCLNT_STREAMFLAGS_EVENTCALLBACK&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;hnsPeriod&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;mixFormat&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nullptr&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;// Start both, then in the event callback:&lt;/span&gt;
&lt;span class="c1"&gt;// GetBuffer -&amp;gt; copy to ring -&amp;gt; ReleaseBuffer&lt;/span&gt;
&lt;span class="c1"&gt;// Signal worker thread that owns STT submission&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Measure the actual period with &lt;code&gt;IAudioClient::GetCurrentPadding&lt;/code&gt; and ETW (&lt;code&gt;Microsoft-Windows-Audio&lt;/code&gt;) or a simple high-resolution timer around the callback. Process Lasso’s LatencyTest or raw ETW still surface the real floor.&lt;/p&gt;

&lt;h2&gt;
  
  
  Trade-offs that still matter
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Cloud STT vs local.&lt;/strong&gt; Scribe v3 Live (or Soniox, Deepgram Nova, etc.) wins on accuracy for noisy, accented, or domain-heavy speech once the audio is clean. Local checkpoints remove the network variable and the data-egress question. Most production overlays do both.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Shared-mode vs exclusive.&lt;/strong&gt; Shared mode is what users already have open. Exclusive mode can drop latency but breaks other apps and is brittle across driver updates. Production overlays stay in shared mode and live with the 10–40 ms floor.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Single mixed stream vs dual track.&lt;/strong&gt; A single stream is simpler for pure cloud agents. Dual track gives you private mic audio that never enters the call, independent biasing, and the ability to overlay only the remote-side translation.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Overlay visibility.&lt;/strong&gt; Paint on a layered window excluded from capture. That is the difference between a private language aid and a bot that everyone on the call can see.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Scribe v3 Live is a solid accuracy step for anyone already in the ElevenLabs realtime path. For the developer who needs the language barrier gone on their own machine, without joining a bot and without leaking the overlay, the work is still the same: open two WASAPI clients, keep the callback empty, align the tracks, and feed a tight STT/translation path. The model improvements keep arriving. The capture budget does not move until you measure and tighten it yourself.&lt;/p&gt;

&lt;p&gt;I'm building &lt;a href="https://talkaside.com/" rel="noopener noreferrer"&gt;Talkaside&lt;/a&gt;, a Windows app for job interviews and calls in another language.&lt;br&gt;
&lt;em&gt;Drafted with AI assistance from this project's code and logs.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>windows</category>
      <category>audio</category>
      <category>stt</category>
      <category>ai</category>
    </item>
    <item>
      <title>Fish Audio Live 3 Claims 186 ms Full-Duplex Reactions — Dual-Track WASAPI Still Sets the Real Floor</title>
      <dc:creator>devs</dc:creator>
      <pubDate>Thu, 12 Nov 2026 04:14:43 +0000</pubDate>
      <link>https://dev.to/madnessbrains_i/fish-audio-live-3-claims-186-ms-full-duplex-reactions-dual-track-wasapi-still-sets-the-real-floor-3a3e</link>
      <guid>https://dev.to/madnessbrains_i/fish-audio-live-3-claims-186-ms-full-duplex-reactions-dual-track-wasapi-still-sets-the-real-floor-3a3e</guid>
      <description>&lt;p&gt;On 10 November 2026 Fish Audio shipped Live 3, a full-duplex voice model aimed at agents. It keeps listening while it speaks, so a pause, a quick “mm-hmm”, or a mid-sentence correction can change the next output. The company says it can decide to emit a short reaction in as little as 186 ms. Turn-taking is learned inside the model from real conversations rather than bolted on as VAD thresholds and silence timers. English only at launch, available on their agent platform, API, and on-prem for enterprises.&lt;/p&gt;

&lt;p&gt;Those numbers look good once clean PCM is already at the model. They do not change the capture budget on a Windows desktop when you need a private translation overlay for an interview or support call.&lt;/p&gt;

&lt;h2&gt;
  
  
  What a full-duplex cloud model still cannot see
&lt;/h2&gt;

&lt;p&gt;Live 3 (and every other hosted duplex or streaming model) receives the stream you send it. It does not sit on your local audio endpoints. It does not give you a second, private track of your own microphone that never enters the call. It does not let you paint a translation overlay that is invisible to screen share. And it does not run when you prefer not to send audio off-box at all, or when the platform you are on has not been integrated.&lt;/p&gt;

&lt;p&gt;For a developer who needs real-time speech-to-text and translation during a technical interview, a sales call, or a support session in another language, those constraints matter. The useful path is still local capture of two tracks, local or BYOK processing, and an overlay that never appears in the shared desktop.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyllf2qo6wqjgk1wjim0v.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyllf2qo6wqjgk1wjim0v.png" alt="Multiple waveforms" width="800" height="684"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Multiple waveforms. Dual-track capture keeps the mic and the remote side separate so each can be biased, timed, and translated independently.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Dual-track capture is still the practical floor
&lt;/h2&gt;

&lt;p&gt;A single mixed stream loses speaker identity. For live translation you need:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Mic track — your voice (private, never sent to the remote side unless you choose to)&lt;/li&gt;
&lt;li&gt;Loopback track — everything the other person says (WASAPI loopback of the render device)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That separation lets the STT engine produce clean transcripts for each side and the UI show them side-by-side without injecting a bot into the meeting.&lt;/p&gt;

&lt;p&gt;On modern Windows the practical stack is:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Endpoint device (USB headset, Bluetooth HFP/A2DP, built-in mic, or WASAPI loopback).&lt;/li&gt;
&lt;li&gt;Windows Audio Engine (audiodg.exe + audioeng.dll) in shared mode: mixing, any “communications” enhancements, spatial audio, vendor APOs. This routinely adds 10–40+ ms and is fragile under load or with Bluetooth.&lt;/li&gt;
&lt;li&gt;Your capture client buffer (10–40 ms is the common practical range; lower values drop under DPC spikes or heavy system load).&lt;/li&gt;
&lt;li&gt;Format conversion and ring-buffer copy off the audio callback (float32 → int16, 48 kHz → 16 kHz).&lt;/li&gt;
&lt;li&gt;VAD or simple energy gate.&lt;/li&gt;
&lt;li&gt;STT (local faster-whisper / Whisper.cpp / Qwen3-ASR GGUF, or a BYOK cloud endpoint) + optional translation.&lt;/li&gt;
&lt;li&gt;Overlay paint on a dedicated thread (&amp;lt;16 ms if you stay off the UI thread).&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A “186 ms reaction” cloud model is real once it has clean PCM. It is almost irrelevant if the samples have already been delayed or dropped inside the audio engine. End-to-end partials on a good day land in the 80–150 ms range; Bluetooth or a busy shared-mode session pushes that higher.&lt;/p&gt;

&lt;h2&gt;
  
  
  Minimal capture sketch
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Pseudocode shape of a dual-track WASAPI client&lt;/span&gt;
&lt;span class="c1"&gt;// Real code uses the Windows Audio Session API or a thin wrapper&lt;/span&gt;

&lt;span class="k"&gt;struct&lt;/span&gt; &lt;span class="nc"&gt;DualCapture&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;mic&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;WasapiCapture&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;      &lt;span class="c1"&gt;// exclusive or shared, low-latency&lt;/span&gt;
    &lt;span class="n"&gt;loopback&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;WasapiCapture&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="c1"&gt;// render-device loopback&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;impl&lt;/span&gt; &lt;span class="nc"&gt;DualCapture&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;fn&lt;/span&gt; &lt;span class="nf"&gt;start&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="kd"&gt;mut&lt;/span&gt; &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;period_ms&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;u32&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="c1"&gt;// Request smallest stable period the driver will honor&lt;/span&gt;
        &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;mic&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;start&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;period_ms&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;loopback&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;start&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;period_ms&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="c1"&gt;// Callback or event-driven pull into lock-free rings&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;fn&lt;/span&gt; &lt;span class="nf"&gt;pull_frames&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="kd"&gt;mut&lt;/span&gt; &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;Option&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;Vec&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;i16&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;Vec&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;i16&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="n"&gt;mic&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;mic&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;read_converted&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;16000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="n"&gt;remote&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;loopback&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;read_converted&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;16000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="kc"&gt;Some&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mic&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;remote&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Keep the audio callback tiny. Do the resampling and any light VAD off-thread. Feed the two streams independently into your STT path so you can apply different biases, timeouts, or even different models per side.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fz94ggw0de0w6zy63lxfg.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fz94ggw0de0w6zy63lxfg.png" alt="Labeled sine wave" width="551" height="328"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Basic frequency and amplitude labels. Real capture graphs get messier once the Windows audio engine, Bluetooth, and vendor APOs enter the path.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Tradeoffs that actually show up in the wild
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Exclusive mode on the mic gives the lowest period but breaks other apps and some headsets. Shared mode is the pragmatic default for an overlay that must coexist with the call app.&lt;/li&gt;
&lt;li&gt;Bluetooth HFP is still a latency and quality tax. Prefer USB or the PC’s own mic + speakers when you can.&lt;/li&gt;
&lt;li&gt;Loopback of the render device captures whatever the call app is playing, including its own notifications. You still want a way to gate or bias that stream.&lt;/li&gt;
&lt;li&gt;Local models (Whisper.cpp, faster-whisper, small GGUF ASR) keep audio on-box and avoid network RTT. Cloud BYOK endpoints win on accuracy and language coverage when the link is clean.&lt;/li&gt;
&lt;li&gt;Overlay must be click-through and excluded from screen share. Paint on its own thread so a stalled UI never blocks the audio callback.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The 186 ms number is useful marketing once the samples have already left the machine. The floor you actually ship against is still the dual-track WASAPI path, the engine’s processing, and the buffer you choose.&lt;/p&gt;

&lt;p&gt;I'm building &lt;a href="https://talkaside.com/" rel="noopener noreferrer"&gt;Talkaside&lt;/a&gt;, a Windows app for job interviews and calls in another language.&lt;br&gt;
&lt;em&gt;Drafted with AI assistance from this project's code and logs.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>windows</category>
      <category>audio</category>
      <category>stt</category>
      <category>ai</category>
    </item>
    <item>
      <title>ElevenLabs Scribe v3 Live Lands at 93 ms TTFS — Dual-Track WASAPI Capture Still Sets the Overlay Floor</title>
      <dc:creator>devs</dc:creator>
      <pubDate>Wed, 11 Nov 2026 14:23:46 +0000</pubDate>
      <link>https://dev.to/madnessbrains_i/elevenlabs-scribe-v3-live-lands-at-93-ms-ttfs-dual-track-wasapi-capture-still-sets-the-overlay-4jli</link>
      <guid>https://dev.to/madnessbrains_i/elevenlabs-scribe-v3-live-lands-at-93-ms-ttfs-dual-track-wasapi-capture-still-sets-the-overlay-4jli</guid>
      <description>&lt;p&gt;On 10 November 2026 ElevenLabs shipped Scribe v3 and its realtime sibling Scribe v3 Live. The batch model claims 27 % fewer word errors than Scribe v2 across 102 languages. The live model cuts semantic word-error rate 43 % versus Scribe v2 Realtime, adds speaker diarization for the first time in their streaming path, and does turn detection from conversation context rather than silence alone. Coval’s latest numbers put Scribe v3 Live at roughly 93 ms mean time-to-final-segment and 2.2 % WER.&lt;/p&gt;

&lt;p&gt;Those are strong numbers once clean PCM is already at the model. They do not change the capture budget on a Windows desktop when you need a private translation overlay for an interview or support call.&lt;/p&gt;

&lt;h2&gt;
  
  
  What a cloud STT endpoint cannot see
&lt;/h2&gt;

&lt;p&gt;Scribe v3 Live (and every other hosted realtime model) receives the stream you send it. It does not sit on your local audio endpoints. It does not give you a second, private track of your own microphone that never enters the call. It does not let you paint a translation overlay that is invisible to screen share. And it does not run when you prefer not to send audio off-box at all, or when the platform you are on has not been integrated.&lt;/p&gt;

&lt;p&gt;For a developer who needs real-time speech-to-text and translation during a technical interview, a sales call, or a support session in another language, those constraints matter. The useful path is still local capture of two tracks, local or BYOK processing, and an overlay that never appears in the shared desktop.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyllf2qo6wqjgk1wjim0v.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyllf2qo6wqjgk1wjim0v.png" alt="Multiple waveforms" width="800" height="684"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Multiple waveforms. Dual-track capture keeps the mic and the remote side separate so each can be biased, timed, and translated independently.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Dual-track capture is still the practical floor
&lt;/h2&gt;

&lt;p&gt;A single mixed stream loses speaker identity. For live translation you need:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Mic track — your voice (private, never sent to the remote side unless you choose to)&lt;/li&gt;
&lt;li&gt;Loopback track — everything the other person says (WASAPI loopback of the render device)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That separation lets the STT engine produce clean transcripts for each side and the UI show them side-by-side without injecting a bot into the meeting.&lt;/p&gt;

&lt;p&gt;On modern Windows the practical stack is:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Endpoint device (USB headset, Bluetooth HFP/A2DP, built-in mic, or WASAPI loopback).&lt;/li&gt;
&lt;li&gt;Windows Audio Engine (audiodg.exe + audioeng.dll) in shared mode: mixing, any “communications” enhancements, spatial audio, vendor APOs. This routinely adds 10–40+ ms and is fragile under load or with Bluetooth.&lt;/li&gt;
&lt;li&gt;Your capture client buffer (10–40 ms is the common practical range; lower values drop under DPC spikes or heavy system load).&lt;/li&gt;
&lt;li&gt;Format conversion and ring-buffer copy off the audio callback (float32 → int16, 48 kHz → 16 kHz).&lt;/li&gt;
&lt;li&gt;VAD or simple energy gate.&lt;/li&gt;
&lt;li&gt;STT (local faster-whisper / Whisper.cpp / Qwen3-ASR GGUF, or a BYOK cloud endpoint such as Scribe v3 Live) + optional translation.&lt;/li&gt;
&lt;li&gt;Overlay paint on a dedicated thread (&amp;lt;16 ms if you stay off the UI thread).&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A “sub-100 ms time-to-final” cloud model is real once it has clean PCM. It is almost irrelevant if the samples have already been delayed or dropped inside the audio engine. End-to-end partials on a good day land in the 80–150 ms range; Bluetooth or a busy shared-mode session pushes that higher.&lt;/p&gt;

&lt;h2&gt;
  
  
  Minimal capture sketch
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Pseudocode shape of a dual-track WASAPI client&lt;/span&gt;
&lt;span class="c1"&gt;// Real code uses the Windows Audio Session API or a thin wrapper&lt;/span&gt;

&lt;span class="k"&gt;struct&lt;/span&gt; &lt;span class="nc"&gt;DualCapture&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;mic&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;WasapiCapture&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;      &lt;span class="c1"&gt;// exclusive or shared, low-latency&lt;/span&gt;
    &lt;span class="n"&gt;loopback&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;WasapiCapture&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="c1"&gt;// render-device loopback&lt;/span&gt;
    &lt;span class="n"&gt;mic_ring&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;RingBuffer&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;i16&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;lb_ring&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;RingBuffer&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;i16&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;impl&lt;/span&gt; &lt;span class="nc"&gt;DualCapture&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;fn&lt;/span&gt; &lt;span class="nf"&gt;on_mic_callback&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="kd"&gt;mut&lt;/span&gt; &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;f32&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="c1"&gt;// convert, resample if needed, push to ring&lt;/span&gt;
        &lt;span class="c1"&gt;// never block; drop or expand under pressure&lt;/span&gt;
        &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;mic_ring&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;push_pcm&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;fn&lt;/span&gt; &lt;span class="nf"&gt;on_loopback_callback&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="kd"&gt;mut&lt;/span&gt; &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;f32&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;lb_ring&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;push_pcm&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;fn&lt;/span&gt; &lt;span class="nf"&gt;drain_for_stt&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="kd"&gt;mut&lt;/span&gt; &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;Option&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;Vec&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;i16&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;Vec&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;i16&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="c1"&gt;// pull fixed frames (e.g. 20–40 ms) from both rings&lt;/span&gt;
        &lt;span class="c1"&gt;// align by timestamp if the devices are independent&lt;/span&gt;
        &lt;span class="kc"&gt;Some&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;mic_ring&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;drain_frame&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;lb_ring&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;drain_frame&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The callback must stay short. Anything heavier (resampling, VAD, network send) belongs on a worker. Shared-mode sessions inherit the system mix and any communications enhancements; exclusive mode can drop the latency but breaks system sounds and requires careful format negotiation.&lt;/p&gt;

&lt;h2&gt;
  
  
  Trade-offs that still matter
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Cloud vs local.&lt;/strong&gt; Scribe v3 Live (or Soniox, Deepgram Nova-3, etc.) gives you the latest accuracy and diarization without shipping a GPU. Local GGUF models (Qwen3-ASR 1.7B Q4_K_M at ~1.3 GB, faster-whisper, Whisper.cpp) keep the audio on-box and remove the network RTT. Hybrid (local VAD + cloud STT only on speech) is the practical middle.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Bluetooth.&lt;/strong&gt; HFP is still a latency and quality tax. Prefer USB or the built-in mic + loopback of the speakers when you can.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Overlay thread.&lt;/strong&gt; Painting the translation on the UI thread will hitch. Keep the overlay on its own thread and only push text deltas.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Speaker identity.&lt;/strong&gt; Even with realtime diarization on a mixed stream, a private dual-track setup gives you hard separation without relying on the model’s clustering.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Practical path for interview and call overlays
&lt;/h2&gt;

&lt;p&gt;Keep the mic and the remote side separate. Run VAD locally. Send each track (or only the remote track) to the STT of your choice—local or BYOK cloud. Render the result in an overlay that is excluded from screen share. The model latency numbers look great in isolation; the Windows audio graph still sets the floor you actually feel.&lt;/p&gt;

&lt;p&gt;I'm building &lt;a href="https://talkaside.com/" rel="noopener noreferrer"&gt;Talkaside&lt;/a&gt;, a Windows app for job interviews and calls in another language.&lt;br&gt;
&lt;em&gt;Drafted with AI assistance from this project's code and logs.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>windows</category>
      <category>audio</category>
      <category>stt</category>
      <category>ai</category>
    </item>
  </channel>
</rss>
