The YouTube analytics cron ran without incident on August 10. Health checks were green. Then I looked at the channel manually and counted 79 videos where I expected 89. Ten were gone. A quick oEmbed check on every tracked ID confirmed eleven total deletions — the YouTube Data API v3 omitted them entirely, which means deleted rather than unlisted or private.
One of the eleven was published by this pipeline at 21:47 UTC on August 10. By 01:14 UTC on August 11 it was gone. The pipeline that uploaded it never noticed.
This article is about the blind spot that caused that, what the implementation looks like, and a subtle bug in the first version that I would have hit the next day.
The blind spot: health checks only watch the input side
The pipeline health monitor I built in July checks whether videos have been queued, generated, and uploaded according to schedule. It opens GitHub Issues when a cron doesn't fire, when the queue goes stale, or when an upload timestamp is missing.
What it doesn't do — and what I never thought to add — is ask YouTube whether the videos it previously published still exist. The monitor is entirely input-side: local files, local timestamps, queue state. If something on YouTube's end deletes a video, every local artifact still looks valid. The upload timestamp is real. The video ID is in the queue's history. All green.
This is the same category of problem I wrote about in fixing survivorship bias in my analytics — a measurement that looks at what you produced, not at the current state of what exists. Silent failure detection in this pipeline has always been a coverage problem: the tests check output, not outcome.
The fix needs to work the other direction: take today's live fetch from the YouTube Data API and compare it against what was known to exist yesterday. Anything absent is a candidate deletion.
Building detect_disappeared() using JSONL history
The analytics script already maintained a JSONL history file — one row per (date, video_id) — because fixing survivorship bias required having all-time fleet data rather than just the current top/bottom snapshot. That existing history is the comparison baseline.
def previous_snapshot(history_path: Path = HISTORY_PATH) -> tuple[str, dict[str, dict]]:
"""(date, {video_id: row}) of the most recent day already in history."""
by_date: dict[str, dict[str, dict]] = defaultdict(dict)
if history_path.exists():
for line in history_path.read_text().splitlines():
try:
d = json.loads(line)
except ValueError:
continue
if d.get("date") and d.get("video_id") and d.get("status") != "disappeared":
by_date[d["date"]][d["video_id"]] = d
today = datetime.now(timezone.utc).strftime("%Y-%m-%d")
prior = sorted(d for d in by_date if d != today)
return (prior[-1], by_date[prior[-1]]) if prior else ("", {})
def detect_disappeared(stats_videos: list[dict], history_path: Path = HISTORY_PATH) -> list[dict]:
prev_date, prev_rows = previous_snapshot(history_path)
if not prev_rows:
return []
live = {v.get("id") for v in stats_videos if v.get("id")}
return [
{"video_id": vid, "last_seen": prev_date, "title": row.get("title", ""),
"published_at": row.get("published_at", ""), "views": row.get("views", 0)}
for vid, row in sorted(prev_rows.items())
if vid not in live
]
stats_videos is the live fetch from videos.list — everything the Data API returns for the channel today. previous_snapshot() reads the most recent date already written to history (not today). The diff is a set subtraction: IDs in the snapshot that are absent from the live set.
The result goes two places. First, the run prints a ::warning:: annotation that surfaces in the GitHub Actions step summary:
::warning::11 video(s) disappeared from YouTube since 2026-08-10: abc123(24v), def456(0v), ...
Second, the daily report gains a "Disappeared from YouTube" table with each ID, its last-known view count, and its publish date. Both outputs are loud by design — this is the one failure mode the input-side health checks structurally cannot catch.
The subtle bug: disappearance markers that look like presence
The first implementation had a defect I would have hit on the very next run.
When detect_disappeared() finds deletions, they get appended to the history file as rows with status: "disappeared". That keeps a permanent record of the deletion and lets day7_by_archetype() exclude those videos from median calculations.
The bug: previous_snapshot() read every row in the history file to build the "what existed yesterday" map. A status: "disappeared" row has a valid date and video_id — so it was being counted as "this video was present on that date." The next day's run would read the disappeared marker for a video, conclude that video was present yesterday, not find it in the live fetch (because it was deleted), and report it as disappeared again. The same 11 videos would fire every single day with a drifting last_seen.
The fix is one filter condition:
if d.get("date") and d.get("video_id") and d.get("status") != "disappeared":
by_date[d["date"]][d["video_id"]] = d
Disappeared markers record an absence, not a presence. Including them when building the comparison snapshot inverts their meaning. The test that covers this:
def test_disappearance_is_reported_once_not_every_day(self):
# Day 1: three alive. Day 2: one vanishes and is marked.
# Day 3: the marker must not read as "present on day 2" and re-fire.
...
self.assertEqual(run.detect_disappeared(live, self.path), [])
I built this test after the fix rather than before, but it locks in the correct semantics.
Excluding deleted videos from archetype medians
A secondary issue: if a deleted video's status: "disappeared" row were included in the day-7 median calculation, it would drag medians down. The 11 videos that were deleted all sat at 0-25 views. A product_findindiegame Short at 12 views included in the archetype's day-7 median would look like a weak performer, when actually it never had a chance to accumulate views before disappearing.
This is survivorship bias in the reverse direction: not over-counting winners by excluding losers, but over-penalizing the archetype by including phantom underperformers.
day7_by_archetype() already has a guard for this:
if d.get("status") == "disappeared":
continue # deleted from YouTube — must not count toward any median
The comparison pipeline now looks like this:
| Data path | What it tracks | Disappeared handling |
|---|---|---|
fetch_uploads() |
Local upload records (timestamps, IDs) | Unaware of YouTube deletions |
fetch_stats() |
Live YouTube Data API response | Only shows surviving videos |
append_history() |
JSONL per (date, video_id) | Records status: "disappeared" rows |
detect_disappeared() |
Diffs live vs snapshot | Fires :⚠️: and report table |
day7_by_archetype() |
Day-7 view medians by archetype | Skips status: "disappeared" rows |
What I'd change about the original design
Adding output-side monitoring from day one would have caught this. The simplest form is a daily count check: fetch channel.statistics.videoCount from the Data API and compare it against your own count of known-uploaded IDs. A drop of more than 1-2 between runs is an anomaly worth alerting on.
The auto-tuner's archetype directive system runs daily and produces a directive that the video generation routine consumes. That directive is built on the day-7 median — so a corrupted median directly produces a wrong archetype bias. I'd caught one survivorship-bias version of this problem two days earlier. This was the inverse arriving from an unexpected direction.
Input-side health checks give you confidence that your automation ran correctly. Output-side checks give you confidence that what you produced still exists. For a publishing pipeline, you need both. The pipeline health checks I had were comprehensive on the input side and completely blind on the output side. That's the structural gap this detection closes.
FAQ
Does this distinguish between deleted and unlisted/private?
In this case, yes — the YouTube Data API omits videos entirely when they're deleted. Private videos owned by the authenticated account appear in response if you're using OAuth; public videos that go private return no results on a public key request, same as deleted. For this pipeline the distinction didn't matter: all 11 were confirmed deleted via oEmbed returning 404 and the Data API omitting them with a public key.
What if the API itself has an outage and returns fewer videos?
detect_disappeared() doesn't distinguish between a partial API response and a real deletion. If the API returns 60 of 79 videos, 19 will be flagged as disappeared. For now I treat this as an acceptable false positive rate — the :⚠️: annotation requires manual confirmation, and a full-channel API outage is obvious from the report. Adding a sanity check on the returned video count (is it within 10% of yesterday's) would reduce noise.
Should the disappeared rows ever be pruned from history?
Not for now. They're useful as an audit trail and the file is small. If the history grows past a few MB, pruning rows older than 90 days while keeping disappeared markers would be the right call.
What happened with the 11 deleted videos?
Both publish schedules are paused pending a YouTube Studio check for a policy notice. The pipeline never received one. The channel itself is healthy (public, 79 videos). I don't have a confirmed explanation yet — I'll post an update when I do.
Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.
Top comments (0)