Four patterns that keep my YouTube longform JSON queue from going stale

I manage the YouTube longform queue for my BuilderStack channel as JSON files in content/yt-longform-queue/. A spec file lands there when a generator script commits a new dialogue; the publish workflow picks the file, renders it to MP4, uploads it, then moves the file to uploaded/. No external queue service, no database rows, no management dashboard.

This has worked for three months without a major incident. Four patterns kept it from collapsing.

Archetype-priority picking, not FIFO

First-in, first-out publishing breaks when you have product walkthrough videos, educational deep-dives, and weekly recap specs all in the queue simultaneously. A recap spec committed yesterday would block a product walkthrough from two weeks ago if the queue ran FIFO — and the product content is what actually grows the channel.

The picker uses an explicit priority rank:

RANK = {
    "product_findindiegame": 0,
    "product_ossfind": 1,
    "hidden-gem": 1,
    "build_in_public": 2,
    "technical": 3,
    "curated": 4,
    "meta": 4,
    "contrarian": 6,
    "recap": 7,
    "ai_tools": 7,
}
DEFAULT_RANK = 5

Archetypes not in the dict fall to DEFAULT_RANK = 5 — the middle, not the bottom. New formats I haven't classified yet still air rather than sitting perpetually at the end. Within each rank tier, files sort by filename (oldest-first). The archetype value comes from the spec JSON's top-level archetype field, falling back to a prefix match on the filename for older files that predate the field.

One consequence: adding a new archetype name to the dict can reorder the queue overnight. I've done this intentionally to let a backlogged product video jump ahead of a stale recap.

21-day stale expiry

Queue files include a date prefix: YYYY-MM-DD-<slug>.json. The picker removes files whose date is more than 21 days old before selecting what to publish:

MAX_AGE_DAYS="${QUEUE_MAX_AGE_DAYS:-21}"
CUTOFF=$(date -u -d "${MAX_AGE_DAYS} days ago" +%Y-%m-%d)
for f in content/yt-longform-queue/*.json; do
  D=$(basename "$f" | grep -oE '^[0-9]{4}-[0-9]{2}-[0-9]{2}' || true)
  if [[ -n "$D" && "$D" < "$CUTOFF" ]]; then
    rm -f "$f"
  fi
done

A spec written three weeks ago may cite GitHub star counts, model download numbers, or pricing tiers that have shifted. Publishing it as-is misleads viewers. The 21-day window is wide enough to survive multi-week publishing gaps without mass expiry, while still catching specs that accumulated during a pause in generator output.

The practical implication: if you pause publishing for longer than 21 days, specs expire silently and nothing airs. The fix is to re-date the spec (rename the file to today's date) after reviewing its content. Re-dating signals you've checked it and re-approved it for the current context. I learned this the hard way when I added AI-generated keyart thumbnails to two older specs and both were expired by the next picker run before publishing.

The MAX_AGE_DAYS env var lets me override the threshold via GitHub Actions secret without touching the workflow file.

Clean-skip on empty queue

The publish workflow fires on two triggers: a push event when a new spec file is committed (for immediate publication), and a 3x/week cron (Tue/Thu/Sat 23:00 UTC) to drain backlogged specs on days when no new content was generated.

The cron trigger fires whether or not anything is in the queue. When the queue is empty, the picker returns an empty string. The workflow must exit 0 — not fail — or every empty-queue cron run would show red in GitHub Actions history:

if [ -z "$FILE" ] || [ ! -f "$FILE" ]; then
  echo "No long-form queue file, exiting cleanly"
  echo "skip=true" >> "$GITHUB_OUTPUT"
  exit 0
fi

Every subsequent step checks if: steps.pick.outputs.skip != 'true'. The workflow succeeds in about 30 seconds without running the render or upload steps.

Without this pattern, the pipeline health monitor would flag "no video published in 36 hours" as an anomaly even during healthy but quiet periods. A clean-skip exit keeps the failure signal meaningful — a real failure is a non-zero exit, not an empty queue.

Concurrency group with cancel-in-progress: false

The push trigger fires immediately when a generator script commits a new spec. If a render is already running — renders take 12-18 minutes — a second push could start a concurrent run, and two runs might pick the same file.

concurrency:
  group: yt-publish-longform
  cancel-in-progress: false

cancel-in-progress: false queues the new run rather than canceling the in-progress one. The in-flight render completes, moves the picked file to uploaded/, then the queued run starts fresh and picks the next file in priority order. If cancel-in-progress: true, the queued run would cancel the expensive render mid-flight — wasted CI minutes — and then sometimes pick the same file again if the move step hadn't completed before cancellation.

I used cancel-in-progress: true initially because it matched the Shorts pipeline behavior. Shorts render in under two minutes, so cancellation is cheap. A 15-minute longform render is not interchangeable.

The concurrency group name yt-publish-longform is also used by the workflow_dispatch trigger, which lets me manually specify a file to publish. All three trigger types share the group, so a manual dispatch queues behind any running render rather than racing it. GitHub's concurrency documentation covers the cancel-in-progress behavior in detail — the default for omitted cancel-in-progress is false, but I find explicit is clearer than relying on the default.

For comparison, the Bluesky JSONL queue runs daily at a fixed time with no push trigger, so concurrency isn't an issue there. The push-plus-schedule hybrid is specific to workflows where content lands at irregular intervals and you want both immediate publish (on push) and catch-up drain (on schedule).

---

Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.