DEV Community

ahmet gedik
ahmet gedik

Posted on

API Rate Limiting Strategies for YouTube Data API

#php #api #youtube #webdev

The YouTube Data API v3 gives you 10,000 quota units per day by default. That sounds like a lot until you start fetching trending videos from 8 regions every 2 hours. Here are the quota management strategies I've developed for DailyWatch.

Understanding Quota Costs

Not all API calls cost the same:

Operation Cost
videos.list (chart=mostPopular) 1 unit
videos.list (by ID, up to 50) 1 unit
search.list 100 units
videoCategories.list 1 unit
channels.list 1 unit

Notice that search.list costs 100x more than videos.list. This is the most important thing to internalize.

Strategy 1: Avoid Search API

Instead of using the expensive search.list endpoint, use videos.list with chart=mostPopular and filter client-side:

// Expensive: 100 units per call
// DON'T DO THIS for browsing
$searchUrl = 'https://www.googleapis.com/youtube/v3/search?' . http_build_query([
    'part' => 'snippet',
    'q' => 'trending music',
    'type' => 'video',
    'key' => $apiKey,
]);

// Cheap: 1 unit per call
// DO THIS instead
$trendingUrl = 'https://www.googleapis.com/youtube/v3/videos?' . http_build_query([
    'part' => 'snippet,statistics,contentDetails',
    'chart' => 'mostPopular',
    'videoCategoryId' => '10', // Music
    'regionCode' => 'US',
    'maxResults' => 50,
    'key' => $apiKey,
]);
Enter fullscreen mode Exit fullscreen mode

Strategy 2: Quota Tracking 

class QuotaTracker {
    private const QUOTA_FILE = 'data/quota_usage.json';
    private const DAILY_LIMIT = 10000;
    private const SAFETY_MARGIN = 500; // Reserve for emergencies

    public function record(int $units, string $operation): void {
        $usage = $this->loadUsage();
        $today = date('Y-m-d');

        if (!isset($usage[$today])) {
            $usage[$today] = ['total' => 0, 'operations' => []];
        }

        $usage[$today]['total'] += $units;
        $usage[$today]['operations'][] = [
            'time' => date('H:i:s'),
            'units' => $units,
            'operation' => $operation,
        ];

        // Keep only last 7 days
        $usage = array_slice($usage, -7, 7, true);

        file_put_contents(self::QUOTA_FILE, json_encode($usage, JSON_PRETTY_PRINT));
    }

    public function canSpend(int $units): bool {
        $todayUsage = $this->getTodayUsage();
        return ($todayUsage + $units) <= (self::DAILY_LIMIT - self::SAFETY_MARGIN);
    }

    public function getTodayUsage(): int {
        $usage = $this->loadUsage();
        return $usage[date('Y-m-d')]['total'] ?? 0;
    }

    public function getRemaining(): int {
        return max(0, self::DAILY_LIMIT - self::SAFETY_MARGIN - $this->getTodayUsage());
    }

    private function loadUsage(): array {
        if (!file_exists(self::QUOTA_FILE)) return [];
        return json_decode(file_get_contents(self::QUOTA_FILE), true) ?: [];
    }
}
Enter fullscreen mode Exit fullscreen mode

Strategy 3: Key Rotation 

class ApiKeyRotator {
    private array $keys;
    private int $currentIndex = 0;

    public function __construct(array $keys) {
        $this->keys = array_values(array_filter($keys));
        if (empty($this->keys)) {
            throw new \RuntimeException('No API keys provided');
        }
    }

    public function getKey(): string {
        return $this->keys[$this->currentIndex];
    }

    public function rotate(): string {
        $this->currentIndex = ($this->currentIndex + 1) % count($this->keys);
        return $this->getKey();
    }

    public function handleError(int $httpCode): ?string {
        if ($httpCode === 403 || $httpCode === 429) {
            // Quota exceeded or rate limited, try next key
            if ($this->currentIndex < count($this->keys) - 1) {
                return $this->rotate();
            }
        }
        return null; // No more keys to try
    }
}
Enter fullscreen mode Exit fullscreen mode

Strategy 4: Batch Video Details

Instead of fetching video details one by one, batch them:

// Bad: 50 units for 50 videos
foreach ($videoIds as $id) {
    fetchVideoDetails($id, $apiKey); // 1 unit each
}

// Good: 1 unit for 50 videos
$batchIds = implode(',', array_slice($videoIds, 0, 50));
$url = 'https://www.googleapis.com/youtube/v3/videos?' . http_build_query([
    'part' => 'statistics',
    'id' => $batchIds,
    'key' => $apiKey,
]);
// Returns all 50 videos in one response, costs 1 unit
Enter fullscreen mode Exit fullscreen mode

Strategy 5: Smart Caching 

function fetchWithCache(string $url, int $cacheTtl = 3600): ?array {
    $cacheKey = md5($url);
    $cacheFile = "data/cache/api/{$cacheKey}.json";

    // Return cached response if fresh
    if (file_exists($cacheFile)) {
        $age = time() - filemtime($cacheFile);
        if ($age < $cacheTtl) {
            return json_decode(file_get_contents($cacheFile), true);
        }
    }

    // Fetch from API
    $response = @file_get_contents($url);
    if ($response === false) return null;

    $data = json_decode($response, true);

    // Cache the response
    $dir = dirname($cacheFile);
    if (!is_dir($dir)) mkdir($dir, 0755, true);
    file_put_contents($cacheFile, $response);

    return $data;
}
Enter fullscreen mode Exit fullscreen mode

Daily Budget at DailyWatch

Here's our actual quota budget at dailywatch.video:

Operation Runs/day Cost/run Daily total
Trending (8 regions) 12 8 96
Categories 12 1 12
Stale refresh 12 1 12
Total 120 units

That's just 1.2% of the daily quota, leaving plenty of headroom for growth and unexpected needs. The key is avoiding the search.list endpoint and batching everything possible.

Top comments (0)