DEV Community: Apoorv Darshan

How I built Live Voice: one tool contract, six realtime voice providers

Apoorv Darshan — Sat, 24 Oct 2026 13:46:57 +0000

Fud AI 8.0 is out on iPhone and iPad (Android is in Google Play review) with Live Voice: you talk to the app, and it logs meals, scans barcodes, checks your macros and edits your diary while you watch it happen on screen.

Here's how it works under the hood. Fud AI is open source (MIT), so every file below is public.

One tool contract, three codebases

The assistant can only act through tools. They live in one file, shared/voice-assistant/tools.json: 25 actions like open_camera, capture_photo, scan_barcode, log_food, log_water, start_fast, log_workout, update_setting and end_conversation.

The iOS app, the Android app and the Cloudflare Worker all follow that contract. Both apps bundle an identical copy, and a CI script fails the build if either copy drifts. I change the JSON first, then each platform.

Every tool also declares its effect: read, navigate, write or destructive. The UI behaviour comes from that:

Writes show a card with Undo for 8 seconds.
Destructive actions always need confirmation.
With "Confirm before saving" on (the default), a write without user_confirmed: true changes nothing. It returns confirmation_required, the app shows a Confirm/Cancel card, and the model has to ask you before calling again.

The model is also told never to invent numbers. Calories and macros come from the data tools, not from its memory.

Six realtime providers, one experience

Live Voice runs on whichever provider you choose:

Provider	Key
Gemini Live	Your own
OpenAI Realtime	Your own
OpenAI GPT-Live	Your own
xAI Grok Voice	Your own
Qwen Omni Realtime	Your own
Fud AI hosted (Gemini Live via my Worker)	Plus/Pro plan

They differ more than you'd expect:

Audio: Gemini, Qwen and hosted take 16 kHz PCM in and send 24 kHz out. OpenAI and xAI use 24 kHz both ways.
Tool calls: Gemini sends toolCall.functionCalls and wants toolResponse. OpenAI, xAI and Qwen send response.function_call_arguments.done and want a function_call_output item followed by response.create.
Vision: Gemini, Qwen and hosted get one camera frame per second while the camera is open. OpenAI gets the latest frame with each turn. xAI and GPT-Live get none, so the model asks you to say when the food is in frame.

Each provider gets a thin adapter, and everything above it (tools, cards, confirmation, undo) is shared. Own keys go in headers (x-goog-api-key or Authorization: Bearer), never in URLs.

The hosted relay

For people who don't want to manage an API key, the Plus/Pro plan connects to a WebSocket relay on Cloudflare Workers that forwards to Gemini Live. A relay like that could easily turn into a free general-purpose Gemini key, so it's strict:

Before the upgrade, the Worker checks the user id, rate limits, the RevenueCat plan and the remaining voice balance.
The first message must be setup. The Worker rebuilds it from an allow-list: the model is pinned server-side, tools are function declarations only, and instructions are size-limited.
After that, only realtimeInput, clientContent and toolResponse are accepted, each 1 MiB at most, with a bounded input rate.
Billing is by time: 15-second blocks, each charged when it starts. Sessions end at 15 minutes.
Nothing you say or show is stored. Logs carry structured codes and a hashed id prefix only.

The orb

The glowing orb is a Metal shader on iOS and an AGSL port on Android 13+, with a Canvas fallback on older versions. It's driven by the loudness of whoever is talking, so you can see when it's listening, thinking or speaking.

You can start a session from the orb on Home, Siri, the Action Button or Control Center on iPhone, or a Quick Settings tile, shortcut or widget on Android.

Try it

Fud AI is free on iOS and Android, and Live Voice reaches Android as soon as Google approves 8.0. Use your own AI key, or the optional Plus/Pro plan if you'd rather not. No account needed.

Website: https://fud-ai.app
Source: https://github.com/apoorvdarshan/fud-ai

I'd love feedback on the voice experience, and PRs are welcome.

Five switches. Two app stores. How I built Fud AI’s release pipeline

Apoorv Darshan — Sun, 11 Oct 2026 06:18:20 +0000

Building the app was only half the release. I still had store listings, screenshots, notes, and subscriptions to deal with.

For Fud AI, I moved those steps into GitHub Actions. Five switches decide what runs; a version tag starts the workflow.

The controls

The diagram shows the five repository variables. Listing updates and screenshots can run independently of production rollout or review submission.

Unset means off. Enabled switches stay enabled for future releases. On iOS, listing updates include What’s New; Play release notes require a binary upload.

Android: tag, build, then decide whether to publish

An android-v* tag runs checks, builds signed AAB/APK files, adds signatures and provenance, and creates a GitHub Release.

Production rollout is optional. When enabled, it uploads the AAB to Play’s production track with status completed, subject to Google’s processing and review. There’s no testing-track selector.

I still increment versionCode before tagging.

iOS: two systems share the release

An ios-v* tag starts two paths: Xcode Cloud archives and uploads the binary; GitHub Actions handles checks, metadata, review submission, and the GitHub Release.

The matching App Store version must already exist, with the right build selected. Actions doesn’t yet wait for Xcode Cloud or select that build. I’m planning that handoff for the next update.

Submitting still means waiting for Apple’s review.

Subscriptions are part of the release

The submission script includes eligible subscriptions and in-app purchases from the repository’s catalog. Subscriptions use their subscription-version relationship.

If a required product can’t be attached, app submission stops. Retries reuse the draft and skip products already added.

Keep the content alongside the code

Listing copy lives in APPSTORE.md and PLAYSTORE.md. Notes and screenshots also come from the repo. A dry run prepares the files and validates the catalog before any store API calls.

Credentials stay in GitHub Secrets; switches are repository variables. RevenueCat catalog writes aren’t implemented yet, so enabling that sync fails explicitly.

Where it stands

Currently, iOS listing updates and review submission are enabled. Android rollout, screenshots, and Play release notes are off.

The next step is the iOS build handoff. If you’re building something similar, start with a dry run, then enable store actions as each release is ready.

Which part of your mobile release still makes you open the store console?

Implementation: https://github.com/apoorvdarshan/fud-ai/tree/main/store

Originally published on X.

I built a cookie bridge, then changed my entire agent setup

Apoorv Darshan — Sun, 11 Oct 2026 06:18:19 +0000

I wanted my coding agents to use the websites I was already logged into. That small problem ended up changing my whole setup.

Chrome, Brave, and a cookie bridge

I started with the Codex desktop app and its in-app browser. Importing my Chrome session was convenient: the agent could work with sites where I was already signed in.

Then I switched to Brave, and the import option I needed was missing. My accounts were signed in over there while the agent was working somewhere else.

So I built Browser Cookie Bridge, a macOS app that started by moving Brave cookies into Codex and grew to support more Chromium browsers.

When I tried Grok Bot later, I used Cookie Bridge there too, exporting a cookie bundle into its browser. I kept wanting the same thing: let the agent start from the session I already have.

The code: https://github.com/apoorvdarshan/browser-cookie-bridge

Moving into cmux

The RAM usage of the Codex desktop app eventually pushed me toward terminal interfaces. I started using cmux, which also had an in-app browser with session import built in.

That helped, but I still had to sync often. Some Google sessions did not come across properly, and some passkey flows did not work as expected.

I would start a task and end up spending time getting the browser signed in again. Cookie transfers helped, but they could not carry everything, including passkeys.

The rule I use now

Eventually I tried agent-browser from Vercel Labs and added this rule to my AGENTS.md:

Use the agent-browser MCP for web browsing by default (headless, logged out). If a site needs my login, relaunch it with --profile Default to reuse my Chrome logins.

Most browsing starts without my accounts. When a task needs a login, the agent uses the Chrome profile flow. That flag launches a temporary copy of the profile, so session reuse becomes part of the workflow instead of another import I have to arrange by hand.

I still handle a fresh login or passkey approval when needed. But now I can use Claude Code, Codex or OpenCode in cmux with the same browser rule.

I shared the setup on October 9:

// Detect dark theme var iframe = document.getElementById('tweet-2108313693696188526-473'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2108313693696188526&theme=dark" }

Guillermo Rauch replied, "This is the way. more to come!" A nice surprise after all that fiddling with browser sessions.

// Detect dark theme var iframe = document.getElementById('tweet-2108314389170516471-245'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2108314389170516471&theme=dark" }

It became part of my daily work

I checked my saved agent threads before writing this. Across October 8 to 11, the audit found 616 recorded agent-browser calls and command batches in 11 threads. That includes retries and batches with multiple actions, so it is a usage count, not 616 completed tasks.

Those threads covered the Omi website and leaderboard, a ringtone editor, my portfolio and resume, marketing, and research. I knew I was using it a lot. The history made that obvious.

This was my follow-up:

// Detect dark theme var iframe = document.getElementById('tweet-2109099239699890312-532'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2109099239699890312&theme=dark" }

Browser Cookie Bridge still helps when I want sessions inside an in-app browser. For everyday work, I now prefer terminal agents with a browser tool they can call when needed.

Thanks @vercel, @v0 and @rauchg for the work you're putting into tools for builders. agent-browser has saved me a lot of time. I get to spend more of it actually shipping.

I wanted to build apps. I did not expect browser logins to shape so much of how I build them.

Try it: https://agent-browser.dev. Chrome profile details: https://agent-browser.dev/sessions.

Originally published on X.

I built a free ringtone maker with Web Audio and FFmpeg.wasm

Apoorv Darshan — Sun, 11 Oct 2026 04:42:45 +0000

I built Ringtone Cropper, a free, open-source browser tool for turning a short section of audio into a ringtone for Android or iPhone.

Try the editor · Source on GitHub

Here's the workflow in about 30 seconds:

What it does

Upload an audio or video file, select a moment on the waveform, preview it, and export. The selection defaults to exactly 30 seconds, with 15/30/40-second presets and custom lengths.

There are fade-in and fade-out controls, looped preview, optional loudness normalization, and MP3, M4A, M4R, or WAV exports. The interface works on mobile and desktop, with light, dark, and system themes. No signup is needed.

The site creates the audio file; your phone's apps or settings apply it as a ringtone. It includes setup instructions, including a GarageBand workflow for iPhone.

What runs where

The frontend is plain HTML, CSS, and JavaScript. Web Audio handles decoding and preview, while ffmpeg.wasm handles export in the browser.

For an uploaded file, the flow is:

Local file → browser decoding + waveform
           → selection + preview
           → browser export → download

Local files stay on your device.

Optional YouTube imports take a different path: a Cloudflare Worker validates and rate-limits the request, then a container running yt-dlp retrieves the audio. Trimming and export still happen in the browser. These imports are for audio you're permitted to download.

A small detail: keeping 30 seconds exactly 30 seconds

Selection boundaries use a 0.1-second grid. A subtle edge case is a source that ends between grid points: rounding after clamping can push a selection past the actual end.

The selection helper floors the available duration to that grid first, fits the clip length, and then clamps its start. The same helper keeps the presets and editable boundaries consistent.

Here's the selection helper.

Audio quality has a ceiling

YouTube imports request the best available audio stream without an intermediate lossy conversion. Exports use the original source, with 320 kb/s MP3 or AAC options and uncompressed WAV.

A higher export bitrate can't recover detail missing from the source. WAV avoids another lossy encode, but doesn't magically improve the recording.

I'd love feedback on the waveform controls and export experience—especially on a phone. Is it easy to select the exact moment you want?