YouTube Video Scraper API: How to Extract Public Video Metadata and Stats with Python

The fastest way to turn a single public YouTube video URL into a stable, JSON-shaped metadata record — title, description, duration, view count, like count, tags, upload date, channel name, and thumbnail URL — is to pair an open-source video scraper repository with a small Python normalization layer that flattens fields, coerces types, and appends each result to a JSONL file. This article is for research engineers, content analytics teams, dataset curators, digital archivists, and AI-agent builders who need a per-video primitive they can call repeatedly without re-writing parsing logic each time.

The data-scrape/youtube-video-scraper-api repository provides a Python reference for extracting a single public video's metadata. Combine it with data-scrape/youtube-channel-scraper when your workflow also needs to walk every video on a channel and aggregate stats at the creator level.

TL;DR

• Use data-scrape/youtube-video-scraper-api to fetch a single public YouTube video's metadata into a structured JSON record.
• Define a narrow input: one video URL or ID per call. Batch jobs iterate the URL list externally so the scraper stays a one-record primitive.
• Normalize each record into a stable schema with video_url, video_id, title, description, channel_name, channel_id, duration_seconds, view_count, like_count, comment_count, tags, upload_date, thumbnail_url, scraped_at, and source_repo.
• Append each normalized record to a JSONL file so the dataset grows by line and can stream into pandas, DuckDB, or a downstream queue.
• Switch to data-scrape/youtube-channel-scraper when you need to enumerate every video on a channel before extracting per-video metadata.
• Capture one representative sample first, validate the field schema against your assumptions, and only then automate at scale.

Why per-video metadata is the right primitive

Most YouTube analysis eventually distills into one question: what does this specific video say, who posted it, and what is the engagement snapshot right now? A scraper that answers that question for a single URL becomes the building block for every higher-level workflow — a content audit, a competitive benchmarking study, a research dataset, an archival snapshot, or a context bundle for an AI agent.

A handful of problems recur once you start building:

• Schema drift. Duration may arrive as an ISO 8601 string, a colon-delimited string, or raw seconds. View and like counts may be locale-formatted. Tags may arrive as a comma-separated string or a list. Raw JSON rarely has the field names a downstream consumer expects.
• Missing fields. Not every public video has a description, tags, or a like count. Some videos disable comments. Your schema must handle nulls gracefully instead of crashing on a missing key.
• ID format variability. YouTube video IDs are 11 characters, but URLs can arrive as watch?v=, youtu.be/, embed/, or shorts/ variants. A normalizer that extracts the canonical ID up front prevents duplicate records.
• Thumbnail resolution. YouTube provides multiple thumbnail resolutions per video. Pick the highest available and store the URL so the record remains self-describing.
• Rate of platform change. YouTube ships front-end changes frequently. Anything that consumes raw HTML or relies on selector paths needs periodic re-validation.
• Compliance scope. Public video URLs are in scope. Private videos, member-only content, and live chat replays behind access controls are not. Always respect YouTube's Terms of Service and applicable law when collecting and storing public data.

This article takes the middle path: start with an open-source per-video reference, then add a small, auditable Python layer for normalization and JSONL append.

What the verified repositories provide

data-scrape/youtube-video-scraper-api is a Python reference built around extracting a single public video's metadata into a structured document. It exposes a scraper.py entry point, a requirements.txt, an examples/ folder, and a README. Read the README for the current setup steps and supported arguments before running it — the names of CLI flags and the shape of the response may differ from the illustrative examples in this article.

data-scrape/youtube-channel-scraper is a sibling repository that targets channel-level enumeration rather than single-video resolution. When your analysis needs to iterate over every video on a channel before extracting per-video metadata, you keep the same normalization layer and only swap the source adapter.

The data-scrape profile hosts the rest of the open-source repositories that share a similar pattern, including per-record JSONL normalization pipelines for TikTok, Instagram, X, Zillow, and other domains.

Pipeline design

A maintainable per-video extraction pipeline has five stages: capture raw JSON for a single public video URL, normalize the response into a stable field schema, coerce numeric and date fields into typed columns, append the record to a JSONL file, and surface a small validation job that flags schema drift. Keep the raw JSON as an artifact alongside the normalized record so any future schema change can be replayed against the original payload.

Setup and configuration

Clone the repository and install dependencies inside a virtual environment so packages do not collide with system Python:

git clone https://github.com/data-scrape/youtube-video-scraper-api.git
cd youtube-video-scraper-api
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install -r requirements.txt

Verify the scraper runs against a single public video URL before building the normalization layer:

python scraper.py --url "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

The command above is illustrative. Check the repository README for the exact CLI flags and output path the current version uses.

Runnable Python: normalize and append

The following script wraps the scraper's raw JSON output with a normalization layer. It accepts a single video URL or a text file of URLs (one per line), extracts the canonical video ID, normalizes fields into a stable schema, and appends each record to a JSONL file:

"""
Normalize YouTube video metadata into a stable JSONL record.
Wraps the raw output from data-scrape/youtube-video-scraper-api.
"""
import json
import re
import sys
import subprocess
from datetime import datetime, timezone
from pathlib import Path

# ── Configuration ──────────────────────────────────────────────
SCRAPER_SCRIPT = "scraper.py"          # entry point in the cloned repo
OUTPUT_DIR = Path("output")
RAW_DIR = OUTPUT_DIR / "raw"
JSONL_PATH = OUTPUT_DIR / "videos.jsonl"
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
RAW_DIR.mkdir(parents=True, exist_ok=True)

# ── ID extraction ───────────────────────────────────────────────
VIDEO_ID_RE = re.compile(r"(?:watch\?v=|youtu\.be/|embed/|shorts/)([A-Za-z0-9_-]{11})")

def extract_video_id(url: str) -> str | None:
    """Pull the 11-character video ID from any common YouTube URL variant."""
    match = VIDEO_ID_RE.search(url)
    return match.group(1) if match else None

# ── Field coercion ─────────────────────────────────────────────
def coerce_int(value) -> int | None:
    """Convert locale-formatted count strings like '1,234,567' to int."""
    if value is None:
        return None
    if isinstance(value, int):
        return value
    cleaned = re.sub(r"[^\d]", "", str(value))
    return int(cleaned) if cleaned else None

def coerce_duration(value) -> int | None:
    """Accept ISO 8601 duration, colon-delimited string, or raw seconds."""
    if value is None:
        return None
    if isinstance(value, (int, float)):
        return int(value)
    text = str(value).strip()
    # ISO 8601: PT1M30S
    iso_match = re.match(r"PT(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)S)?", text)
    if iso_match:
        h = int(iso_match.group(1) or 0)
        m = int(iso_match.group(2) or 0)
        s = int(iso_match.group(3) or 0)
        return h * 3600 + m * 60 + s
    # Colon-delimited: 1:30 or 1:02:30
    parts = text.split(":")
    if len(parts) in (2, 3) and all(p.isdigit() for p in parts):
        return sum(int(p) * 60 ** (len(parts) - 1 - i) for i, p in enumerate(parts))
    return None

def coerce_tags(value) -> list[str]:
    """Accept a list or a comma-separated string; return a clean list."""
    if value is None:
        return []
    if isinstance(value, list):
        return [str(t).strip() for t in value if t]
    return [t.strip() for t in str(value).split(",") if t.strip()]

def pick_thumbnail(thumbnails) -> str | None:
    """Pick the highest-resolution thumbnail URL from the raw payload."""
    if not thumbnails:
        return None
    if isinstance(thumbnails, dict):
        return thumbnails.get("url") or thumbnails.get("high", {}).get("url")
    if isinstance(thumbnails, list) and thumbnails:
        last = thumbnails[-1]
        if isinstance(last, dict):
            return last.get("url")
    return None

# ── Normalization ──────────────────────────────────────────────
def normalize(raw: dict, video_url: str) -> dict:
    video_id = raw.get("video_id") or extract_video_id(video_url)
    return {
        "video_url": video_url,
        "video_id": video_id,
        "title": raw.get("title"),
        "description": raw.get("description"),
        "channel_name": raw.get("channel_name") or raw.get("author"),
        "channel_id": raw.get("channel_id"),
        "duration_seconds": coerce_duration(
            raw.get("duration") or raw.get("length")
        ),
        "view_count": coerce_int(raw.get("view_count") or raw.get("views")),
        "like_count": coerce_int(raw.get("like_count") or raw.get("likes")),
        "comment_count": coerce_int(raw.get("comment_count") or raw.get("comments")),
        "tags": coerce_tags(raw.get("tags")),
        "upload_date": raw.get("upload_date") or raw.get("publish_date"),
        "thumbnail_url": pick_thumbnail(raw.get("thumbnails")),
        "scraped_at": datetime.now(timezone.utc).isoformat(),
        "source_repo": "data-scrape/youtube-video-scraper-api",
    }

# ── Runner ─────────────────────────────────────────────────────
def scrape_one(url: str) -> dict:
    """Run the scraper for one URL and return raw JSON."""
    result = subprocess.run(
        [sys.executable, SCRAPER_SCRIPT, "--url", url],
        capture_output=True, text=True, timeout=60,
    )
    if result.returncode != 0:
        raise RuntimeError(f"Scraper failed for {url}: {result.stderr[:500]}")
    raw_path = RAW_DIR / f"{extract_video_id(url)}.json"
    raw_path.write_text(result.stdout, encoding="utf-8")
    return json.loads(result.stdout)

def append_jsonl(record: dict) -> None:
    with open(JSONL_PATH, "a", encoding="utf-8") as f:
        f.write(json.dumps(record, ensure_ascii=False) + "\n")

def run(url: str) -> dict:
    raw = scrape_one(url)
    record = normalize(raw, url)
    append_jsonl(record)
    print(f"OK  {record['video_id']}  views={record['view_count']}  "
          f"duration={record['duration_seconds']}s")
    return record

if __name__ == "__main__":
    if len(sys.argv) < 2:
        print("Usage: python normalize.py <video_url | urls_file.txt>")
        sys.exit(1)
    arg = sys.argv[1]
    if arg.endswith(".txt"):
        urls = [l.strip() for l in Path(arg).read_text().splitlines() if l.strip()]
        for u in urls:
            try:
                run(u)
            except Exception as e:
                print(f"ERR {u}: {e}")
    else:
        run(arg)

Run it for a single video:

python normalize.py "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

Or batch-process from a URL list:

python normalize.py video_urls.txt

The script writes raw JSON to output/raw/<video_id>.json and normalized records to output/videos.jsonl. Adjust SCRAPER_SCRIPT to match the actual entry-point filename documented in the repository README.

Representative output

After running the normalizer, each line in videos.jsonl looks like this:

{
  "video_url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
  "video_id": "dQw4w9WgXcQ",
  "title": "Rick Astley - Never Gonna Give You Up (Official Video) (4K Remaster)",
  "description": "The official video for Never Gonna Give You Up by Rick Astley...",
  "channel_name": "Rick Astley",
  "channel_id": "UCuAXFkgswmhWqot6PqM4aQ",
  "duration_seconds": 213,
  "view_count": 1600000000,
  "like_count": 1700000,
  "comment_count": 250000,
  "tags": ["Rick Astley", "Never Gonna Give You Up", "pop"],
  "upload_date": "2009-10-25",
  "thumbnail_url": "https://i.ytimg.com/vi/dQw4w9WgXcQ/maxresdefault.jpg",
  "scraped_at": "2026-09-11T10:15:00+00:00",
  "source_repo": "data-scrape/youtube-video-scraper-api"
}

The numbers above are representative of the field shape, not a live snapshot. View, like, and comment counts change continuously. Always treat scraped counts as point-in-time observations and record scraped_at so the observation window is auditable.

Use cases

• Content archival. Capture metadata snapshots for videos that may be edited, deleted, or made private. The JSONL file plus raw JSON artifacts form a reproducible audit trail.
• Competitive content analysis. Collect per-video stats for a competitor's recent uploads, then compare view velocity, engagement ratios, and tag strategies.
• Research datasets. Build a labeled dataset of public video records. Each record carries its source URL and extraction timestamp so the dataset remains traceable.
• AI agent context. Feed normalized video metadata as structured context into an LLM pipeline that summarizes or classifies content without re-scraping on every call.
• SEO and keyword research. Extract titles, descriptions, and tags from ranking videos for a topic cluster, then feed the structured text into a keyword analysis workflow.
• Creator benchmarking. Combine the per-video primitive with data-scrape/youtube-channel-scraper to aggregate stats at the channel level for creator comparison.

Comparison: per-video vs channel-level scraping

Dimension	Per-video (this article)	Channel-level
Primary repo	youtube-video-scraper-api	youtube-channel-scraper
Input	One video URL or ID	One channel URL or handle
Output	Single metadata record per video	List of video URLs + channel stats
Best for	Archival, point-in-time snapshot, AI context	Creator audits, upload cadence, benchmarking
Scaling pattern	External URL list → loop	Single channel → enumerate → loop
Normalization	Same schema, same JSONL target	Same schema, same JSONL target

The two repositories are complementary. A typical research workflow uses the channel scraper to discover the video URL list, then feeds each URL into the per-video scraper for deep metadata extraction.

Limitations and compliance

• Point-in-time data. View, like, and comment counts are live counters. A record scraped at 10:00 will differ from one scraped at 10:05. Always store scraped_at and never present a scraped count as a fixed value.
• No private data. This workflow targets public video URLs only. Member-only videos, private videos, and live chat replays behind access controls are out of scope.
• Rate and pacing. Scraping at high concurrency risks IP-level blocking and may violate YouTube's Terms of Service. Add delays between requests and cap concurrency. If you need large-scale collection, verify current rate limits and terms before production use.
• Layout changes. YouTube's front-end evolves. If the scraper depends on HTML structure or selector paths, a platform update can break extraction silently. Run a validation job that checks field presence on a known sample after each run.
• Thumbnail availability. maxresdefault.jpg is not available for all videos. Fall back to hqdefault.jpg or another resolution if the highest-quality thumbnail returns 404.
• Description truncation. YouTube may truncate long descriptions in certain views. Compare the scraped description against the full watch page to verify completeness.
• Terms and law. Respect YouTube's Terms of Service, applicable data protection laws, and privacy requirements when collecting, storing, and sharing public video metadata. Do not redistribute scraped content in ways that exceed fair use or platform terms.

Maintenance checklist

• [ ] Pin Python and dependency versions in the virtual environment.
• [ ] Store raw JSON alongside normalized records for replay.
• [ ] Run a schema-validation job after each batch — flag records with missing title, video_id, or duration_seconds.
• [ ] Monitor scraper exit codes and log failures with the source URL for retry.
• [ ] Re-validate against a known sample video after YouTube ships a front-end change.
• [ ] Cap concurrency and add inter-request delays to respect platform rate expectations.
• [ ] Document the scraped_at observation window in any downstream report or dashboard.

FAQ

Is there an official YouTube Data API?
Yes. YouTube provides an official Data API v3 with quotas for video, channel, and search endpoints. The open-source repository in this article is an independent reference for extracting public metadata without an API key. Evaluate both options against your quota needs, compliance requirements, and maintenance capacity.

What metadata fields can I extract from a public video?
Typical fields include title, description, channel_name, channel_id, duration_seconds, view_count, like_count, comment_count, tags, upload_date, and thumbnail_url. The exact fields depend on what the scraper returns and may vary by video type. Always validate against a sample.

How often should I re-scrape a video?
For archival, a single snapshot is sufficient. For engagement tracking, daily or weekly re-scrapes with a stored scraped_at timestamp let you compute deltas. Avoid high-frequency scraping — it adds load without meaningful signal for most metrics.

What happens when YouTube changes its layout?
If the scraper relies on page structure, a front-end update can break extraction. Keep raw JSON artifacts, run a validation job on a known sample after each run, and monitor for empty or null fields that signal a silent failure.

Can this connect to pandas, DuckDB, or an AI agent pipeline?
Yes. The JSONL output is line-delimited JSON, which loads directly into pandas via pd.read_json(..., lines=True), imports into DuckDB as a table, and streams into LLM context as structured documents. The per-record schema is intentionally flat to keep these integrations simple.

What should I verify before production use?
Confirm that the scraper's current CLI flags and output format match the README, test against a diverse set of video URLs (standard, Shorts, live replays, older uploads), validate field completeness, and verify that your request pacing respects YouTube's Terms of Service and rate expectations.

Can I use this for videos on private channels or member-only content?
No. This workflow targets public video URLs only. Private videos, member-only content, and any resource behind access controls are out of scope. Always respect YouTube's Terms of Service, applicable law, and privacy requirements.

Next steps

Start with a single public video URL, run the normalizer, and inspect the JSONL output. When you are ready to scale to channel-wide collection, use data-scrape/youtube-channel-scraper to enumerate video URLs, then feed each URL back into the per-video pipeline. Browse the data-scrape profile for adjacent repositories that share the same per-record JSONL pattern across TikTok, Instagram, X, Zillow, and other domains.