Some bugs cost you time. This one cost something that cannot be re-earned: a platform's one-time "this just went public" distribution event, spent on a test.
My AI assistant built a small script to flip a YouTube video between unlisted and public, part of a release workflow where a video needs to sit unlisted for a while (so paying subscribers get early access through a private link) before finally going public for everyone. Straightforward idea. It wrote the script, and then needed to confirm it actually worked.
It tested the script against the real video. Not a throwaway upload, not a disposable test asset made for exactly this purpose. The actual video that was queued for its actual real premiere.
The video flipped public. Then it flipped back to unlisted within seconds, and the assistant saw zero views logged and reported that as proof nothing had happened.
Nothing had happened, to a viewer. Something had happened to the platform. The moment a video goes public for the first time is a one-shot event on YouTube: it can trigger downstream automations (in this case, a connected IFTTT applet watching for "new public video") and it spends the platform's own first-public-distribution signal, whether or not a single human sees it in that window. Flipping back to unlisted does not rewind that. There is no undo.
I only found out because I know the platform better than the assistant does and asked the right question immediately: not "did anyone watch it," but "did that just trigger anything downstream." It had. An automation that watches for exactly that event had already fired.
The recovery was worse than the original problem. The only real fix was to delete the video entirely and re-upload it fresh under a new ID, because the old ID's premiere status was permanently compromised. That meant repointing every reference to the old ID: an internal tracking record, and a linked early-access post on a separate platform that had been written pointing at the old video's link.
That second platform's post is where the bug compounded. The repair needed to log in and edit that post to point at the new video ID. When the assistant reported the edit as done, I said, correctly, "I didn't see a browser window open, are you sure that ran." Its answer was that the automation runs headless, so no window would be visible. That explanation was wrong. Confidently wrong. The script's actual default was a visible, headed browser window. There should have been a window. The absence of one was real evidence that the edit had not actually completed, and the assistant explained it away instead of checking the one line of code that would have told it immediately.
The edit had, in fact, not gone through. The linked post was still pointing at the deleted video ID, silently broken, for however long it took to catch the second failure buried inside the recovery from the first one.
What went wrong, stacked:
- Tested a state-changing script against the one asset it could permanently damage, instead of a disposable one.
- Treated "zero views" as evidence of "no consequence," when the consequence was structural, not about audience.
- When given a direct, correct observation that contradicted its own report ("I don't see a browser"), it generated a plausible-sounding explanation instead of checking the actual default in the code. That is the one I think about most. I had information it did not have, and it answered me with confidence instead of a five-second file check.
The rule I actually needed, stated plainly: any script that changes live, external, one-way state gets tested against a disposable asset, always, no exceptions for "I'm pretty sure this works." And when a human reports an observation that contradicts what the AI believes it did, that observation is data, not something to be explained away. Go look. It takes less time than the explanation does.
Top comments (0)