DEV Community: Phillip Gray

I Used an AI Agent to Make a Product Video. The Cost Was $0, But There's a Catch.

Phillip Gray — Wed, 01 Jul 2026 05:14:02 +0000

Can an AI Agent Make a Professional Explainer Video for Pocket Change?

The promise of projects like OpenMontage, an open-source agentic video production system, is seductive: describe a video in plain English, and an AI agent handles the rest. Research, scripting, asset generation, editing, rendering—the full stack. To put this to a real-world test, I tasked an agent with producing a 75-second explainer video on a non-trivial topic: local-first software. The goal was a hybrid of animated diagrams, abstract b-roll, and clean text overlays—a classic tech explainer. The question was simple: could the agent deliver a professional product, and what would it really cost?

The Plan: Smooth Sailing and a Reference Point

Getting started with OpenMontage was a breeze. Aside from installing FFmpeg, a well-documented prerequisite, the make setup command handled all Python and Node dependencies cleanly. The journey began not with a blank prompt, but with a reference video I chose for its polish and recency: Linear's "Introducing Linear Agent." OpenMontage's video analyzer ingested the YouTube URL and returned something far more insightful than a mere transcript. It produced a five-aspect cinematographic breakdown—subject, motion, scene, framing, camera—and even inferred why the style worked, noting the "high-contrast dark palette... to make plain text look incredibly premium."

This single prompt took me from a URL to a creative brief with a full estimate of API costs, complete with two distinct concepts grounded in research on the local-first ecosystem. The agent was off to a flying start.

The Build: An Agent at the Helm

The agent returned with a STORYBOARD.md, a detailed 75-second timeline mapping script beats to assets and motion design. It looked solid, so I gave it the green light. A key insight emerged mid-build: OpenMontage isn't an agent itself. As the README states, "Your AI coding assistant IS the orchestrator." The project is a powerful toolkit of pipelines, tools, and skills designed to be wielded by an external agent—in my case, the Antigravity CLI (agy). This reframed the experiment: I wasn't testing a monolithic product, but how well its tools and instructions could steer my chosen agent.

The first render attempt was a mixed bag. The agent correctly re-timed the video from 75s to 53s, accounting for the faster-than-estimated narration from the local Piper TTS model. But it also hit a wall. The storyboard had vaguely described a "dedicated subtitle track generated via our transcriber." The result? The entire script was dumped on-screen at once, complete with raw transcriber tokens like [_BEG_] and awkward word splits (CR DT). It was a perfect lesson in agentic workflows: the vaguest line in the plan is precisely where it will break.

First Look: Art School Project, Not Polished Product

After stripping the broken caption track, I watched the first complete version. The verdict was immediate: it was cohesive and on-theme, but it looked more like an art school video project than a professional explainer. The problems were substantive:

Missing Information: The video mentioned CRDTs—the core technical concept—but never explained or diagrammed them. A crucial GCP billing issue earlier in the process had blocked access to Google's Imagen, and the fallback visuals never filled this explanatory gap.
Sync Issues: On-screen text and bullet points were frequently out of sync with the narration, sometimes appearing seconds early or transposed.
Amateur Aesthetics: The typography felt like a PowerPoint slide, and the b-roll clips, while individually fine, were repeated randomly.

The agent had nailed the vibe but whiffed on the substance. It had assembled atmosphere where I needed explanation.

Iteration and a Quality Plateau

I tasked the agent with a targeted revision: fix the timing and add a diagram explaining CRDTs. It successfully corrected the sync issues and generated a Mermaid flowchart for the diagram. This was a definite improvement, but it also highlighted the agent's limits. The Mermaid diagram, while technically correct, had the aesthetic of a corporate IT presentation—the wrong register entirely for a polished product video. The output had hit a quality plateau. We could iterate on the details, but the fundamental feel remained amateurish. This was the stopping point.

Friction, Surprises, and the Real Cost

The process also revealed several fascinating and instructive points of friction. At one point, the agent silently stalled, seemingly stuck in a long thought process. It turned out to be waiting for a hidden permission prompt, a reminder to check an agent's underlying processes. More surprisingly, the agent autonomously patched OpenMontage's source code to fix a bug in how it loaded API keys from the .env file.

The most critical lesson, however, was about cost. The final bill for the video was effectively $0. The handful of calls to Google's Imagen API fell within the free tier. But getting there wasn't free. A single GOOGLE_API_KEY doesn't unlock all of Google's services; the key for Gemini doesn't work for Cloud Text-to-Speech or Imagen out of the box. Unlocking Imagen required enabling billing on my Google Cloud project, which involved a ten-minute detour to set up a new billing account with a $10 minimum prepay.

This is the crucial asterisk. The marginal cost of making the video was pennies, but the floor to enter was $10 and a bureaucratic setup process. The pocket change promise is real, but the on-ramp isn't free.

The Verdict: A Powerful Tool with a Ceiling

So, would I use OpenMontage again? Absolutely. It delivered on its core promise, orchestrating a complex production pipeline from a simple prompt for virtually no cost. The economic advantage over cloud video tools is staggering. Coming from the world of manual video editing, the ability to iterate on a script or timing with a single prompt feels like a superpower.

But the output has a ceiling. The final product never felt professional. A key workflow gap is the inability to easily review visual assets before they're baked into the final composition. For a truly polished result, I'd need a more hands-on approach, using the agent for heavy lifting but manually guiding the script, visual selection, and final composition.

OpenMontage proves that agent-driven video production is not only possible but incredibly cost-effective. It can get you 80% of the way there for 1% of the cost. Closing that final 20% gap, however, still requires a human hand at the wheel.

Building a Local-First Voice Copilot for the Shell with HoldSpeak and Ollama

Phillip Gray — Sat, 27 Jun 2026 03:04:40 +0000

The Promise: A Private, Voice-Activated Shell

The dream of a voice-activated command line is compelling: speak a command, see it executed. But for many developers, piping terminal input through a cloud-based API is a non-starter. This is the promise of a project like karolswdev/HoldSpeak, a cross-platform tool for local voice typing. Could it be the core of a truly local-first, push-to-talk shell assistant? I paired it with Ollama and a local llama3.2 model to find out.

The goal was simple: hold a key, speak a command like "list files by size," release the key, and have the correct shell command appear, gated by a final confirmation prompt. This project turned out to be a tale of two stacks: one for voice that was surprisingly clean, and one for language that revealed the sharp edges of the local-first promise.

Building the Demo

To test this idea, I built a small Python script to tie these components together. You can find the complete code for this experiment, including the prompt engineering, in my demo project on GitHub: voice-activated-shell-demo.

Setup Instructions

Recreating this local-first voice assistant involves a few distinct steps:

Install HoldSpeak from Source: Since we need to use it as a library, clone the repository and install it in editable mode.
```
git clone https://github.com/karolswdev/HoldSpeak.git
cd HoldSpeak
pip3 install -e .
```
Install and Run Ollama: Use Homebrew (on macOS) to install the Ollama CLI, then start the server.
```
brew install ollama
ollama serve
```
Pull a Local LLM: In a separate terminal, pull a small, capable model. I used llama3.2.
```
ollama pull llama3.2
```
Grant Permissions (macOS): To allow the hotkey listener to work, your terminal application (e.g., iTerm, Terminal.app) must be given Accessibility permissions in System Settings > Privacy & Security > Accessibility.
Run the Demo Script: With the setup complete, you can run the final Python script that integrates all these components.

Finding the Seams in HoldSpeak

HoldSpeak presents itself as an application, but my goal was to use it as a library. The first step, installation, was a simple pip3 install -e .. The second step was hitting a wall: import holdspeak exports no documented, usable API. The path forward was to grep the source code.

This source-diving quickly revealed the clean seams I needed:

AudioRecorder (holdspeak/audio.py): A simple class to capture audio from the default microphone.
Transcriber (holdspeak/transcribe.py): A wrapper that intelligently selects a local backend (in my case, MLX Whisper on macOS) to convert audio into text.

The core components were there, but finding them was the tax. For a project with extensive product documentation, it provided no documented API.

Assembling the Local Stack

With the voice components identified, I set up the other half of the local stack: the Large Language Model. Standing up a local LLM with Ollama is one of the easiest parts of the modern AI landscape: brew install ollama, ollama serve, and ollama pull llama3.2. No accounts, no API keys.

The friction appeared immediately, not in setup, but in correctness. A smoke test asking for "list files sorted by size" returned ls -lh | sort -k1, a plausible but incorrect command that sorts by file permissions, not size. The correct command is ls -lhS. This early result established the central tension of the project: the local LLM was easy to install but unreliable out of the box.

In contrast, wiring up HoldSpeak's AudioRecorder and Transcriber was almost anticlimactic. Twelve lines of Python were all it took to capture and transcribe audio. When I spoke "list files by size," Whisper returned the exact text, 'List files by size', flawlessly. The local stack had two very different halves: speech-to-text was crisp and effortless; text-to-command was already the weak link.

Adding Push-to-Talk and a Safety Gate

More source-diving uncovered a third seam: HotkeyListener (holdspeak/hotkey.py). This class provided a clean callback API for push-to-talk functionality, defaulting to the Right Option key. Integrating it was trivial, but it surfaced an undocumented OS-level hurdle: on macOS, my terminal (iTerm) needed Accessibility permissions. The listener failed silently until I granted the permission and completely restarted the application.

With the hotkey, recorder, transcriber, and Ollama all wired together, the final piece was a confirmation loop. Before executing any command with subprocess, a simple input("run? [y/N]") provided a critical safety gate.

The very first real run proved why it was so necessary. I held Right Option and said, "show me the 5 largest files." The LLM returned ls -lhS | tail -n 5. It looked correct, so I hit y.

It was wrong. ls -lhS sorts files from largest to smallest. tail -n 5 therefore returns the 5 smallest files from that list. The confirmation prompt protected me from malicious commands, but not from plausible but subtly incorrect ones. I had approved my own bug.

Evaluation: A Great Voice Library, A Fragile Copilot

Should you build a voice CLI copilot on HoldSpeak today? Yes for the voice layer, but with major caveats about relying on a small, local LLM for autonomy.

HoldSpeak as a Library

I would use HoldSpeak again. Once you find the entry points—AudioRecorder, Transcriber, and HotkeyListener—they are well-shaped components that compose into a working push-to-talk loop in under 100 lines. The local Whisper transcription was fast, accurate, and required zero configuration to use the MLX backend on Apple Silicon. The single biggest improvement HoldSpeak could make is documenting this programmatic path. A short "using HoldSpeak as a library" guide would have erased most of this project's friction.

The Local-First Tradeoff

This demo perfectly illustrates the tradeoff of a fully local AI stack. The "no cloud" setup is fast, cheap, and private. But that convenience comes at the cost of correctness. The llama3.2 model produced plausible-but-wrong commands repeatedly, and while prompt engineering can fix specific failure cases you identify in advance, it's a process of patching over specific failures rather than building generalized reliability.

The confirm-before-execute gate is mandatory. It makes the tool safe, but it does not make it correct. It protects you from commands you recognize as wrong, but does nothing for the ones that look right, which is where small models often fail. A voice interface, by its nature, encourages speed over the careful inspection of an LLM's work.

HoldSpeak is a recommendable, high-quality voice library waiting to be discovered. The local-first shell copilot I built on top of it, however, remains a great demo but not yet a shippable tool. To make it real, you'd need a larger, more capable model (defeating some of the purpose of a small local setup) or a UI that explains why a command was chosen, giving the user a better chance to catch the subtle bugs a confirmation prompt can't.