YT Poller — Build a fast, cache-first YouTube feed (PHP + JS)

By Mayank Chawdhari — a practical guide to setup, deploy, and embed a production-ready YouTube RSS poller.

TL;DR: YT Poller fetches a YouTube channel RSS feed, parses it, stores a JSON cache (atomic writes + locking), and exposes a small HTTP API (/api/get_videos.php). A themeable frontend (index.php) and a documentation/instructions page (setup.php) make integration trivial. Use a cron job to pre-warm cache and serve instant results to visitors. This post explains why this architecture is fast and SEO-friendly, how the pieces fit together, and step-by-step instructions to get it running and hosted (including static GitHub Pages usage for the docs/UI).

---

Why YT Poller (short, practical reasons)

• Avoids on-demand network fetches to YouTube when visitors arrive — reduced latency and increased reliability.
• Pre-warmed cache + small JSON payload = fast pages and better Core Web Vitals.
• No API keys required (uses YouTube RSS), so it’s easy to deploy and maintain.
• The API returns JSON only (no HTML errors), so frontends behave predictably.
• Easy to embed on any site: client-side snippet or server-side pre-warm via cron.

---

Prerequisites

• PHP 7.4+ / 8.x with curl and simplexml enabled.
• Web server (Apache / Nginx / PHP-FPM) or a host that runs PHP.
• Ability to run cron jobs (or other scheduler).
• Optional: GitHub account (for Pages), Docker (if you want containerization).

---

Project overview — structure & responsibilities

/
├─ index.php           # themeable UI (client)
├─ setup.php           # documentation & integration generator (keep this as your docs page)
├─ api/get_videos.php  # API gateway (validate, cache, fetch fallback)
├─ includes/
│  ├─ config.php       # settings: default TTL, cache_dir, allowed_origins, default_channel (optional)
│  ├─ cache.php        # atomic file cache helper (read/write)
│  └─ fetcher.php      # channel resolver, feed builder, fetcher, RSS parser
└─ cache/               # JSON cache files and runtime logs

---

How it works (brief)

1. Frontend requests /api/get_videos.php?channel=...&limit=...&ttl=....
2. API validates input and checks cache. If cache is fresh, return it.
3. If stale/forced, server uses fetcher.php to resolve channel (UC id), build a reliable RSS URL, fetch feed via cURL, parse with SimpleXML, then write JSON to cache atomically.
4. API always returns JSON; on fetch failure it falls back to previous cache (if present) or returns a clear fetch_failed error with HTTP status.
5. Optionally run a server-side cron (prime_cache.php) to force fetch and pre-warm cache.

---

Key implementation notes (you can copy these into docs or README)

includes/cache.php — atomic JSON writes (safe)

• Sanitizes channel id for filename.
• Writes to a .tmp then rename() (atomic on POSIX).
• Uses flock() for exclusive lock during write.

Important behavior: robust to partial writes and concurrent access.

---

includes/fetcher.php — channel resolution + RSS parsing

• Accepts many channel forms: UC..., full channel URL, @handle, /c/username etc.
• Resolves to the canonical UC... id (recommended approach: use YouTube oEmbed or author_url detection server-side).
• Builds RSS URL: https://www.youtube.com/feeds/videos.xml?channel_id=UC... (this is the reliable canonical RSS endpoint).
• Uses cURL with sensible defaults (timeout, follow redirects, user agent) and simplexml_load_string() to parse RSS.
• Extracts videoId, title, published, thumbnails, etc., into a simple array suitable for JSON.

---

api/get_videos.php — the small API gateway

• Central responsibilities:

  • Input validation and sanitization
  • CORS header support (configured via includes/config.php)
  • Cache TTL logic and server-side minimum fetch interval (throttling)
  • Safe error handling (exceptions/errors converted to JSON payloads, not HTML 502 pages)
  • Fallback to cached payload if fetch fails

Production tips:

• Set YTPOLLER_DEV to false in production to hide stack traces.
• Limit allowed_origins to your domain in includes/config.php to avoid open CORS in prod.

---

Quick install (5–10 minutes)

1. Clone repo:

git clone https://github.com/BOSS294/yt-poller.git
cd yt-poller

1. Ensure PHP + extensions:

php -m | grep -E "curl|simplexml"

1. Create and secure cache dir:

mkdir -p cache
chown www-data:www-data cache        # adapt to your server user
chmod 750 cache

1. Configure (optional): edit includes/config.php:

return (object)[
  'default_ttl' => 300,
  'max_ttl' => 86400,
  'cache_dir' => __DIR__ . '/../cache',
  'allowed_origins' => ['https://yourdomain.com'],
  'user_agent' => 'YT-Poller/1.0 (+https://yourdomain.example)',
  'min_fetch_interval' => 10,
  'default_limit' => 20,
  // optional default channel:
  // 'default_channel' => 'UC_xxxxxxxxxxxxx',
];

1. Upload to PHP host or run locally for testing:

php -S 0.0.0.0:8000
# Visit http://localhost:8000/setup.php to read the docs page and generate snippets

---

Recommended cron: pre-warm cache

Create cron/prime_cache.php (or use the snippet in setup.php) and add to crontab:

cron/prime_cache.php:

<?php
$channel = "UC_xxxxxxxxx";   // your channel
$api = "https://yourdomain.com/api/get_videos.php";
$url = $api.'?channel='.urlencode($channel).'&ttl=3600&limit=20&force=1';
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_exec($ch);
curl_close($ch);
echo "Done\n";

Crontab entry (every 15 minutes):

*/15 * * * * /usr/bin/php /path/to/cron/prime_cache.php >/dev/null 2>&1

Notes:

• Choose TTL and cron frequency sensibly. For low-frequency channels, longer TTL (1h–24h) might be OK. For active channels, 5–15 minutes is common.
• Use force=1 in cron so it bypasses TTL and fetches fresh.

GITHUB REPO : https://github.com/BOSS294/yt-poller