DEV Community

Cover image for Key-Less YouTube Scraping in Python: Meet ytscrape 🚀
Vasyl Smutok
Vasyl Smutok

Posted on

Key-Less YouTube Scraping in Python: Meet ytscrape 🚀

#python #opensource #webdev #webscraping

If you've ever built a data pipeline, ML dataset, or analytics tool that relies on YouTube data, you've probably faced the same two headaches:

  1. The Official YouTube Data API: Strict daily quotas (10,000 units disappear fast when searching or retrieving comments), required billing setup, and API key management.
  2. Headless Browsers (Selenium / Playwright): Heavy memory footprint, complex setup, and slow execution speed.

To solve this, I built ytscrape — a free, lightweight, and typed Python package that communicates directly with YouTube's internal InnerTube API (the exact same endpoints used by the official web client).

No API keys. No quota limits. No browser overhead. Pure HTTP performance. ⚡

💡 What Makes ytscrape Different?

  • 🔑 Zero Configuration: No API keys, no Google Cloud Console setup, no cost.
  • 🧊 Pure HTTP requests: Powered by requests — extremely lightweight and easy to deploy in containers, serverless environments, or CLI tools.
  • 🧩 Fully Typed Dataclasses: Get standard, frozen dataclasses (Video, Channel, Comment, etc.) with autocomplete support instead of parsing chaotic, deeply nested JSON responses.
  • 📄 Transparent Pagination: Simply iterate over results using standard Python loops — continuation tokens are fetched under the hood automatically.
  • 💬 Deep Comment Scraping: Fetch every single comment and reply, with full support for sorting (such as Newest First, which prevents YouTube from hiding "potential spam" or less relevant comments).
  • 🌍 Full Localization: Native support for interface languages (hl) and content regions (gl) validated via pycountry.

⚡ Quick Showcase

1. Simple Search & Pagination

Iterate through video, channel, or playlist search results without managing pagination tokens:

from ytscrape import YouTube, SearchFilter

with YouTube(language="en", region="US") as yt:
    # Search for videos
    for video in yt.search("python tutorial", filter=SearchFilter.VIDEOS, max_results=10):
        print(f"📹 {video.title} ({video.duration}) -> {video.url}")
Enter fullscreen mode Exit fullscreen mode

2. Extract Video Details & Captions / Transcripts

Fetch rich metadata along with auto-generated or manual caption tracks:

from ytscrape import YouTube

with YouTube() as yt:
    # Fetch details
    video = yt.video("[https://youtu.be/dQw4w9WgXcQ](https://youtu.be/dQw4w9WgXcQ)")
    print(f"Title: {video.title} | Views: {video.views:,}")

    # Extract captions
    transcript = yt.transcript("dQw4w9WgXcQ", languages=["en", "uk"])
    for line in transcript[:5]:
        print(f"[{line.start:.1f}s] {line.text}")
Enter fullscreen mode Exit fullscreen mode

3. Collect Every Comment & Reply

Extract comments sequentially, including author info, likes, creator hearts, and nested replies:

from ytscrape import YouTube, CommentSort

with YouTube() as yt:
    # Use CommentSort.NEWEST to make sure YouTube doesn't hide comments
    comments = yt.comments(
        "dQw4w9WgXcQ",
        include_replies=True,
        sort=CommentSort.NEWEST,
        max_results=50,
    )

    for comment in comments:
        prefix = "  ↳ Reply:" if comment.is_reply else "💬 Comment:"
        print(f"{prefix} {comment.author}: {comment.text}")
Enter fullscreen mode Exit fullscreen mode

📊 Quick Comparison

Feature ytscrape YouTube Data API yt-dlp Headless Browser
API Key Needed ❌ No ✅ Yes ❌ No ❌ No
Daily Quota ❌ None ⚠️ Strict ❌ None ❌ None
Browser Required ❌ No ❌ No ❌ No ✅ Yes
Typed Python Models ✅ Yes ❌ No ❌ No ❌ No
Download Media ❌ No ❌ No ✅ Yes ✅ Yes
Setup Size 🪶 Tiny 📦 Medium 📦 Large 🐘 Massive

Rule of thumb: Use yt-dlp when you need to download video/audio files, use the Official API for enterprise ToS compliance, and use ytscrape when you need fast, structured Python access to metadata, search, transcripts, and comments.

🛠️ Installation & Usage

Install via pip or uv:

pip install ytscrape
# or with uv
uv add ytscrape
Enter fullscreen mode Exit fullscreen mode

It also comes with an out-of-the-box CLI tool:

ytscrape search "python scraping" --max 10
ytscrape comments "[https://www.youtube.com/watch?v=dQw4w9WgXcQ](https://www.youtube.com/watch?v=dQw4w9WgXcQ)" --replies --sort newest
Enter fullscreen mode Exit fullscreen mode

🗺️ What's Next? (Roadmap & Open Source)

ytscrape is actively maintained and open source. The roadmap includes:

  • ⚡ Async API (asyncio / httpx integration)
  • 📺 Channel Tab Scraping (Videos, Shorts, Live streams, Community posts)
  • 🎵 Playlist extraction & item pagination
  • 🔗 Related videos & Trending feeds

🤝 How You Can Support or Contribute

If you find this project helpful for your projects, data collection pipelines, or research:

  1. ⭐ Give it a Star on GitHub: github.com/vsmutok/ytscrape
  2. 📦 Check it out on PyPI: pypi.org/project/ytscrape
  3. 💬 Feedback & PRs: Bug reports, feature suggestions, and contributions are extremely welcome!

Happy scraping! 🐍

Top comments (0)