DEV Community

Cover image for Track competitor YouTube uploads every week and send them to Slack or Google Sheets
Joshua Smith
Joshua Smith

Posted on

Track competitor YouTube uploads every week and send them to Slack or Google Sheets

Competitor tracking is usually a habit you skip: open a dozen channel pages, scroll to find what is new, guess how it did. A weekly job fixes that: once a week, fetch only the videos each channel published in the last seven days, store them, and compare with last week. Below: what to measure, how a date filter keeps cost low, a comparison script, and the limits.

What to track each week

Each video record from YouTube's official Data API carries enough for four signals:

  • Upload cadence. New videos per channel per week, from channelTitle and publishedAt.
  • Title patterns. Length, numbers, question marks and "how to" openers from title. One week proves little; a few months of rows show habits.
  • Length. durationSeconds. Shorts appear in channel uploads like any other video, so split formats before taking a median.
  • Views after N days. viewCount next to publishedAt and scrapedAt, so every number comes with the video's age when it was read.

Why the date filter keeps a weekly job cheap

A channel's uploads can be read newest first, so "what is new this week" means reading until you reach a video older than your cutoff, then stopping. Reading uploads costs about 1 quota unit per 50 videos. A search page costs 100, so do not build this on search.

Best Damn YouTube Scraper does that through its publishedAfter input. Paging stops as soon as older videos appear, and billing is per delivered video, so the back catalogue is never delivered and never billed. Without it, a weekly run at the default maxResults of 50 re-delivers each channel's latest 50 videos, mostly ones you already have, and bills them again.

The weekly input

Use this input and swap in your competitors' @handles, channel URLs or channel IDs:

{
    "startUrls": [
        "https://www.youtube.com/@YouTube",
        "https://www.youtube.com/@jawed"
    ],
    "publishedAfter": "7 days",
    "maxResults": 0
}
Enter fullscreen mode Exit fullscreen mode

maxResults: 0 means no cap, which is safe because the date filter does the limiting. publishedAfter also takes a date such as 2026-01-31 or a period such as 2 weeks. Save the input as a task, put it on a weekly schedule in Apify, and add your own YouTube API key in apiKey (see limits).

From code

For your own cron, one call returns the dataset:

curl -X POST "https://api.apify.com/v2/acts/josh99smith~youtube-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "startUrls": ["https://www.youtube.com/@YouTube", "https://www.youtube.com/@jawed"], "publishedAfter": "7 days", "maxResults": 0 }' \
  > "week-$(date +%F).json"
Enter fullscreen mode Exit fullscreen mode

A small week-over-week comparison

From week two, compare the two latest files with python compare.py this-week.json last-week.json:

import json, sys
from collections import defaultdict
from datetime import datetime
from statistics import mean, median

def ts(s):
    return datetime.fromisoformat(s.replace("Z", "+00:00"))

def load(path):
    rows = json.load(open(path, encoding="utf-8"))
    good = [r for r in rows if r.get("success") and r.get("liveStatus") != "upcoming"]
    return good, {r.get("errorType") for r in rows if not r.get("success")}

def summarise(rows):
    by_channel = defaultdict(list)
    for r in rows:
        by_channel[r["channelTitle"]].append(r)
    return {ch: {
        "uploads": len(vs),
        "minutes": round(median(v["durationSeconds"] for v in vs) / 60, 1),
        "title_chars": round(mean(len(v["title"]) for v in vs)),
        "age_days": round(median((ts(v["scrapedAt"]) - ts(v["publishedAt"])).days for v in vs)),
        "views": median(v["viewCount"] or 0 for v in vs),
    } for ch, vs in by_channel.items()}

this_rows, errors = load(sys.argv[1])
last_rows, _ = load(sys.argv[2])
if errors:
    print("WARNING: failed records", errors, "- this week may be incomplete")

now, before = summarise(this_rows), summarise(last_rows)
for ch in sorted(set(now) | set(before)):
    n, b = now.get(ch, {}), before.get(ch, {})
    print(ch, "uploads", b.get("uploads", 0), "->", n.get("uploads", 0), n)
Enter fullscreen mode Exit fullscreen mode

It keeps only success: true records and skips liveStatus: "upcoming" streams, which have no views yet. It warns when any record failed: a quota error produces free rate-limited records instead of a failed run, so an empty week would otherwise look like a quiet one. Median age sits beside median views so you can see what the views belong to.

Compare like with like. A video one day old has fewer views than one six days old, so a change in median views mostly reflects when the channel posted. For a fairer number, take a second reading: bare video IDs are valid startUrls entries, so a week later you can send last week's IDs and read the same videos again. Each re-read is billed at the usual per-video price.

Send it to Slack or a sheet

Apify integrations move finished results onward without a download. The Actor's README lists Make, Zapier, n8n, Slack, Google Drive and Airbyte, with Slack posts and Google Sheets rows as example uses.

  • Slack: the Slack integration guide covers sending a task's results to a channel. Good for noticing new videos, poor for history.
  • Google Sheets: append rows every week so the history builds. Treat videoId as the key and drop duplicates, because a delayed run can overlap the previous window.
  • Your own endpoint: webhooks fire when a run succeeds, fails or times out. Point one at a small service that runs the comparison and posts the summary.

Cost and limits

$0.001 per video delivered, so 1,000 videos cost $1.00. Private, deleted and unavailable videos, invalid inputs and quota errors cost nothing, there is no start-up charge, and a run stops at the maximum cost you set.

Worked example, with assumed numbers: 20 channels publish 60 videos between them in a week. At 60 x $0.001, a weekly run costs $0.06, or $3.12 over 52 weeks. Without the filter, 20 channels at 50 videos each return up to 1,000 videos per run: $1.00 a week, $52.00 a year, nearly all of it videos you already have. Re-reading last week's 60 videos for the second reading adds another $0.06 a run.

Limits:

  • Quota. By default the Actor uses a built-in key whose 10,000 daily units are shared by all its users. For a scheduled workload, create your own free key in the Google Cloud Console with the YouTube Data API v3 enabled, and paste it into apiKey. When a quota runs out, the run stops cleanly and returns free rate-limited records; quotas reset at midnight Pacific Time. The SUMMARY record in the key-value store lists failures and quota units used.
  • Young videos. Early view counts are small and noisy. Compare videos of similar age.
  • Public data only. No login is used, so nothing from a channel owner's private analytics is available, and private, unlisted and members-only videos are not included. likeCount and commentCount are null when hidden, and YouTube rounds subscriber counts.

If this saves you time, a review on the Store page helps other people find it, and a star on GitHub helps too.

Disclosure: I built this Actor. Source: github.com/josh99smith/youtube-scraper. YouTube is a trademark of Google, and the Actor is not affiliated with Google.

Top comments (0)

Some comments may only be visible to logged-in visitors. Sign in to view all comments.