I've been running an automated YouTube channel since April — a two-host VTuber pipeline that generates daily scripts, renders them overnight with ffmpeg, and uploads automatically via a single CI workflow. The analytics are surprisingly clear: product_findindiegame shorts — game A vs game B comparisons — get around 20× the median views of every other format I've tried.
The problem: until yesterday, my AI script generation routine kept ignoring that.
The original design and why it drifted
My analytics script runs daily at 09:30 JST, fetches the last 30 videos via the YouTube Data API v3, ranks archetypes by median views, and writes bias hints back to docs/yt-knowhow-bank-en.md. The generation routine reads that document before each session.
The knowhow bank had clear language: "product_findindiegame is the proven winner — prefer it." But "prefer" in a 400-line advisory document competes with everything else in the document, plus whatever the model brings from training. The routine kept generating recap and build_in_public shorts — both fine formats, both consistently underperforming in my data.
I made the language stronger. Still drifted. I added a dedicated strategy section. Still drifted. The issue isn't whether the model follows instructions — it does. The issue is that advisory guidance in a long reference document is categorically different from a mandatory step that executes first.
Extracting the decision into a shared function
The first change was refactoring scripts/yt-analytics/run.py to separate the ranking logic from the document rendering. Before, the "which archetype should we prefer?" computation was inline in the knowhow-bank formatter. I extracted it:
def archetype_bias(videos: list[dict], high: list[dict]):
arch_views: dict[str, list[int]] = defaultdict(list)
for v in videos:
a = v.get("_archetype", "unknown")
if a == "unknown":
continue
views = int((v.get("statistics") or {}).get("viewCount", 0))
arch_views[a].append(views)
ranked = sorted(
((a, statistics.median(vs), len(vs)) for a, vs in arch_views.items()),
key=lambda x: (x[1], x[2]),
reverse=True,
)
pref_arch = ranked[0][0] if ranked else "—"
avoid_arch = (
ranked[-1][0]
if len(ranked) >= 2 and ranked[-1][1] < ranked[0][1]
else "—"
)
return pref_arch, avoid_arch, ranked
The "unknown" exclusion matters. When the publish step fails silently and no archetype .json file gets committed, recent videos have no archetype metadata. "Unknown" would otherwise top the ranking by sample size and produce contradictory guidance ("prefer unknown"). I hit that edge case the first week.
Now both the knowhow bank and the new directive call archetype_bias() and always agree on the ranking. Before the refactor, it was possible for them to drift if I modified one formatter but not the other.
The directive file: one screen, imperative, committed
render_directive() generates docs/yt-today-directive.md — a short, mandatory document the generation routine reads before anything else:
# YT — Today's generation directive (2026-06-22)
> Machine-generated by `scripts/yt-analytics/run.py` from live view data.
> The Auto-Script routine MUST read this FIRST and generate EXACTLY the
> archetype named here. This **overrides** the day-of-week default and any
> hardcoded habit. Do not pick a weak archetype "for variety".
## SHORT (daily)
- **archetype = product_findindiegame** ← generate this today
- open with the **numeric** hook pattern
- do NOT generate archetype: **recap** (worst performer)
## Why (live ranking, median views)
- product_findindiegame=194v(n=2), technical=67v(n=1), build_in_public=31v(n=2)...
Three design choices drove the format:
One screen. The document is short by design. An AI routine can skim a long document and de-emphasize sections; it can't miss the first section of a short one. The critical instruction is on line 4, not line 48.
Imperative, not advisory. "MUST", "EXACTLY", "overrides". I avoided "prefer", "consider", and "try to". The document is a directive, not a suggestion.
Explains itself. The "Why" block shows the live performance data that drove the decision. Partly for my own debugging — I can read the committed directive to understand why a given day's archetype was chosen — but also so the routine understands the reasoning, not just the conclusion.
Committed to the repo. This was the fix that made everything else work. The yt-analytics.yml workflow now includes docs/yt-today-directive.md in its commit step. Before: the file was written during the run and then discarded. Tonight's generation routine would run without it. The directive existed for about 30 seconds on the runner's filesystem and then was gone.
The streak guard in code
If product_findindiegame consistently wins by a wide margin, every video in the queue converges on game comparisons. That's probably too narrow — you need some variety to discover the next format that might overtake it.
The knowhow bank had a prose note about variety. That note was being de-emphasized in exactly the same way as the archetype preference.
The streak guard is now in render_directive(), applied before the directive is written:
def recent_uploaded_archetypes(n: int = 2) -> list[str]:
rows: list[tuple[str, str]] = []
if UPLOADED_DIR.is_dir():
for f in UPLOADED_DIR.glob("*.json"):
try:
d = json.loads(f.read_text())
except (ValueError, OSError):
continue
order = d.get("uploaded_at") or f.name[:16]
rows.append((order, d.get("archetype", "unknown")))
rows.sort(reverse=True)
return [a for _, a in rows[:n]]
# In render_directive():
recent = recent_uploaded_archetypes(2)
if len(recent) == 2 and recent[0] == recent[1] == target:
alt = next((a for a, _m, _n in ranked if a != target), None)
if alt:
switched_from, target = target, alt
If the last two uploaded videos were both product_findindiegame, the directive names the next-best ranked archetype instead. The directive itself says "← generate this today (final pick; the 3-in-a-row guard is already applied)" so the generation routine doesn't need to re-evaluate — the decision is already final when it reads the file.
This is the change I feel best about. The streak guard is a predicate on a sorted list of JSON files in content/yt-queue/uploaded/. It's testable and deterministic. I can write a unit test for it. I could not write a unit test for "don't do the same format three times in a row" as prose in a document.
Updating the spec to make this mandatory
The generation routine reads docs/yt-script-spec-en.md as its operating manual. I updated section R5 to add a mandatory first step:
**STEP 0 (mandatory, overrides everything else): open `docs/yt-today-directive.md`
and generate EXACTLY the archetype its "SHORT (daily)" line names.** That file is
machine-generated from live view data and is the single source of truth for
today's pick — it already applies the ranking, the avoid-list, and the 3-in-a-row
guard for you.
The previous section started with "The day-of-week mapping is only a default for variety. Before writing, you MUST let live performance steer the pick:" and then described the computation in prose. The routine was re-deriving the answer rather than reading a pre-computed answer. The new section makes the directive file the single source of truth and demotes everything else to fallback or explanation.
What I'm still uncertain about: whether the generation routine reliably opens the directive file, or whether it reads the spec but skips file reads when the context is long. I'll know by looking at what archetype gets queued each morning. If it's product_findindiegame, it's working. If it's recap, the routine is reading the spec but not following through on the file read, and I need to make that step more explicit — perhaps as a separate pre-check in the workflow that injects the directive content into the prompt directly.
The longform "動画の先生" dialogue format is preserved as a weekly directive in the same file, separate from the daily SHORT pick. That format is different enough architecturally — A/B speaker JSON, 8–12 minute runtime — that it can't compete with shorts on the same ranking.
What I'd do differently
Start with the directive pattern. I spent two weeks making increasingly emphatic prose in the knowhow bank before concluding the problem was category, not emphasis. Advisory guidance and mandatory steps belong in different documents.
Commit the file immediately. The analytics workflow was writing yt-today-directive.md but not including it in the commit. I caught this because the feature appeared to have no effect the first day. git log -- docs/yt-today-directive.md showed zero commits. The file was written on the runner and discarded.
Put the streak guard first. The streak guard is the piece most likely to affect long-term output quality. Best archetype dominating every slot is almost as bad as worst archetype dominating every slot — you need some distribution to find the next breakout format. I added it last, after the other pieces were working.
Concrete numbers: I'll have them in 30 days
The directive is two days old. I don't have enough data to say whether the archetype distribution has shifted. What I have is the pre-directive baseline: product_findindiegame appeared in roughly 20% of the queue despite ranking first by a wide margin. In 30 days I'll publish the post-directive distribution.
The YouTube slide renderer and title card pipeline are the same upstream — the directive change doesn't affect how videos are rendered, only which template gets invoked. So the production cost stays constant while (hopefully) the output shifts toward higher-performing formats.
FAQ
Why not hardcode the archetype in the generation prompt?
"Best archetype" changes as the channel grows and audience composition shifts. Hardcoding product_findindiegame today means editing a prompt I can't change from code when the data eventually shows something else winning. The directive pattern keeps the decision in version-controlled Python; the prompt stays generic.
Why a file instead of an environment variable?
The generation routine is a scheduled Claude Code session, not a script I invoke with arguments. It reads repo files as context. Files are the natural interface. An env var would require workflow changes every time the decision changes; a committed file updates automatically on each analytics run.
Could the streak guard misfire on stale data?
Yes, in a benign direction. If old archive files persist, the streak guard might switch archetypes unnecessarily. I addressed this by using the uploaded_at timestamp field in the JSON for sort order, rather than mtime — so even if stale files persist, they sort to the bottom and don't influence the "last two" check.
What's the fallback if the directive file is missing?
The spec says "day-of-week mapping is fallback only". That mapping produces reasonable variety across the week. A missing directive means the routine reverts to pre-directive behavior — not ideal, but not broken. The analytics workflow commits the file daily, so it should only be missing the first time or after a workflow failure.
Is this over-engineering for a channel at near-zero scale?
Probably, for now. But the pattern generalizes: compute a decision in code, commit it as a machine-readable file, have an AI session read it rather than recompute it. The Bluesky post queue uses the same principle — state lives in the repo, not in an external service. I'm likely to use this pattern again elsewhere in the shared-prompt-caching ETL layer whenever I need to pass a pre-computed decision into a generation context.
Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.
Top comments (0)