DEV Community: Nick Valencia

Making AI-Generated Code Fail Gracefully

Nick Valencia — Sun, 31 May 2026 02:55:55 +0000

Making AI-Generated Code Fail Gracefully

If your app generates code with an LLM and executes it, you already know the dirty secret: it fails a lot. Not catastrophically — just wrong method names, bad assumptions about state, off-by-one stuff. The kind of errors a human would fix in 10 seconds.

The question is what your user sees when that happens.

The Problem

Version 1 of my app showed users raw Python tracebacks when a generated script failed. Something like:

Script execution failed:
Traceback (most recent call last):
File "", line 3, in
items = timeline.GetItemsInTrack("video", 1)
AttributeError: 'Timeline' object has no attribute 'GetItemsInTrack'
The LLM got the method name wrong — it's GetItemListInTrack, not GetItemsInTrack. An easy fix. But my users are video editors, not Python developers. That traceback means nothing to them except "it broke."

The Fix: Silent Self-Correction

Instead of showing the error, I send it back to the LLM with context:

"The previous script failed with: AttributeError: 'Timeline' object has no attribute 'GetItemsInTrack'. Generate a corrected script."

The LLM sees its own mistake, fixes the method name, and the corrected script runs. The user sees:

"Hmm, let me try that a different way..."

Then 2 seconds later:

"✓ Set opacity to 50% on 12 clips"

They never see the error. It just works on the second attempt.

The Implementation (High Level)

The retry loop is simple:

LLM generates a script
Script fails (validation or execution)
Send the error message back to the LLM as a new prompt
LLM generates a corrected script
Try again (up to 2 retries)
If all retries fail, show a friendly message suggesting simpler commands
The key insight: LLMs are surprisingly good at fixing their own mistakes when you show them the exact error. The success rate on retry is much higher than the first attempt because the error message narrows the solution space.

Friendly Validation Messages
Not all failures are execution errors. Some scripts get rejected before they run because they violate sandbox rules (my app runs generated code in a restricted environment). Instead of showing "Script contains blocked import: 'os'", the user sees:

"That operation would need external libraries that aren't available. Try rephrasing — most operations work with the built-in tools."

Different failure modes get different messages. The user gets guidance on what to try next, not a technical explanation of why it broke.

What I Learned

Users don't care about errors — they care about results. If you can fix it silently, fix it silently.

LLMs are good debuggers of their own output. Feeding the error back works 70-80% of the time on the first retry.

Three retries is the sweet spot. One isn't enough (sometimes the fix introduces a new error). Two catches most errors. Three is for that last 10-20% that need complex logic reevaluations.

Friendly messages need to be actionable. "Something went wrong" is useless. "Try a simpler version of your request" gives the user a next step.

The QThread signal collision was the real bug. I spent hours debugging why retries weren't working before realizing Qt's built-in finished signal was shadowing my custom one in packaged builds. Renamed it and everything clicked. If you're subclassing QThread — don't name your signals finished or started.

The UX Difference

Before: 30% of commands showed a traceback. Users assumed the app was broken.

After: 90%+ of commands succeed (including retries). The 10% that fail get a conversational message. Users assume the app is smart but has limits — which is exactly right.

Building Cutting Room AI — natural language video editing for DaVinci Resolve Studio.

Available now FOR FREE: NickValenciaTech.com

Cutting Room Now Available on macOS

Nick Valencia — Sat, 16 May 2026 01:49:16 +0000

Cutting Room AI now runs on macOS. If you edit in DaVinci Resolve Studio on a Mac, you can control your timeline with plain English.

What It Does
You type something like "set opacity to 50% on the third clip" or "add markers at every cut point on track 1" — and it happens. No scripting knowledge required. No copy-pasting code from forums. Describe what you want, confirm, done.

It connects to your running Resolve instance and executes operations against your live timeline in real time.

Why I Built It
I kept running into the same repetitive timeline tasks — batch property changes, organizing clips, setting up renders with specific settings. Resolve's scripting API can do all of this, but writing Python scripts for every little thing breaks your creative flow.

So I built a layer on top: natural language in, timeline changes out. The AI understands Resolve's API and your current timeline state, generates the right operations, and runs them.

What's New
The Mac version is the same app, same capabilities — just built natively for macOS. Signed and notarized, so it opens cleanly without Gatekeeper friction.

What It Can Do (v1)
Clip properties: opacity, zoom, pan, tilt, rotation, flip, crop, composite mode
Track operations: add, delete, enable/disable, lock, rename
Markers: add, delete, list, color-code
Clip organization: colors, flags
Timeline navigation: jump to timecode, switch pages
Media pool: list clips, create bins and timelines
Rendering: configure settings, add jobs, start/stop renders
Batch operations across entire tracks

Requirements
DaVinci Resolve Studio (external scripting requires the paid version)
Claude API key from Anthropic (~$0.01 per command)
macOS 12+ or Windows 10+

Get It:

Personal Site - https://nickvalenciatech.com
Gumroad — https://nickvalenciatech.gumroad.com/l/cuttingroomai

$39 one-time purchase, both platforms included. You bring your own API key.

Built by a video editor who got tired of the gap between "I know what I want to do" and "now I have to figure out how to script it."

NLP Video Editing Copilot

Nick Valencia — Thu, 14 May 2026 22:17:46 +0000

Hey y'all!
I'm very excited to announce the launch of Cutting Room AI, a natural language video editing copilot. Here's what it is...

Cutting Room AI is a standalone Windows desktop app that lets DaVinci Resolve Studio users control their timeline with plain English. Type what you want — "set opacity to 50% on all clips on track 2" or "add a red marker at the current timecode" — and the AI generates and executes the scripting API calls against your live Resolve session. No scripting knowledge required.

Key Features

Natural language commands executed against your live DaVinci Resolve timeline

Clip properties: opacity, zoom, pan, rotation, crop, composite mode, retime

Track operations: add, delete, enable/disable, lock, rename
Markers, clip colors, flags, media pool queries, and rendering control

Sandboxed script execution with AST-level validation and restricted builtins

Prompt library with pre-written commands to get started fast

Your feedback on the product will be greatly valued and appreciated!

https://nickvalenciatech.com/apps/cutting-room-ai