YouTube Data API: duration comes back P0D and processingStatus never leaves processing

#api #python #debugging #googlecloud

I lost about 18 hours to this one and the fix turned out to be boring, so I'm writing it down for the next person who googles the error string.

Setup: I have an automated pipeline that renders a video and uploads it to YouTube through the Data API v3. No human in the loop past the approval step. The upload call returned fine. The video ID came back. I logged it, marked the job published, and went to bed.

Next morning the video was still processing. Not "processing, nearly done." Just processing, indefinitely. The Studio page showed the thumbnail and title, so clearly something had arrived. But videos.list with part=contentDetails,status gave me this:

"contentDetails": {
  "duration": "P0D"
},
"status": {
  "uploadStatus": "uploaded",
  "processingStatus": "processing"
}

P0D is ISO 8601 for a zero length duration. Zero days, zero everything. The video was 8 minutes 56 seconds.

It sat there for 18 hours. It never errored. It never finished. Nothing in the API ever told me the file was bad.

What actually happened

The upload got killed partway through. Network hiccup, my own process, doesn't really matter which. YouTube kept the bytes it had received and accepted the video record anyway. What it was left holding was a truncated MP4 that the transcoder could never parse, so processing never completed and the duration never resolved past zero.

The part that cost me the time

My pipeline had a check for exactly this. It just checked the wrong field.

I was reading fileDetails.fileSize and comparing it against the size of the file on disk. They matched. I concluded the upload was complete. It was not.

fileDetails.fileSize is not the number of bytes YouTube received. It is the size declared when the resumable session is opened, before a single byte of content is sent. It is your own number handed back to you. It will match your local file whether you uploaded all of it or four percent of it. As a completeness check it is worthless, and it fails in the worst possible direction, which is that it reports success.

What to check instead

contentDetails.duration. If it parses to a real runtime, the file arrived intact and the transcoder read it. If it is P0D, the file is broken and it is not going to recover on its own.

import re

def upload_is_complete(video_resource):
    """P0D means the transcoder never got a parseable file."""
    d = video_resource.get("contentDetails", {}).get("duration", "")
    return d and d != "P0D" and re.search(r"\d", d.lstrip("P")) is not None

Rough timing check: my re-upload of the same 8m56s file processed in about two minutes. If yours has been sitting at P0D for an hour, it's not slow. It's dead.

Recovery

You cannot repair the broken video in place. Re-upload the file as a new video and set the old ID to private.

Worth planning for: you get a new video ID. In my case three Shorts had descriptions pointing at the dead ID and were scheduled to go public the next day. I had to rewrite all three before they published. If anything downstream stores that ID, it needs to be updated as part of recovery, not after someone notices.

What I changed

Two things.

First, I switched from a one shot upload to a chunked resumable one, so an interrupted transfer can resume instead of leaving a corpse behind.

Second, and this is the one that mattered, the pipeline no longer treats "the API returned a video ID" as proof of anything. It polls contentDetails.duration and only marks the job complete when a real runtime comes back.

The general shape of this bug is one I keep running into. I was verifying against a field that gets populated by my own request rather than by the thing I actually wanted to confirm. It looked like verification. It was an echo.

DEV Community