DEV Community

Cover image for Scrape YouTube videos in Python
SerpApi profile image hil
hil for SerpApi

Posted on • Originally published at serpapi.com

Scrape YouTube videos in Python

#webscraping #youtube

Scraping YouTube videos enables developers and businesses to extract detailed YouTube video metadata at scale, including titles, descriptions, view counts, thumbnails, channel names, related videos, and comments/replies. It streamlines what would otherwise require complex scraping and anti‑blocking measures.

Get your API Key

First, ensure to register at SerpApi to get your API Key. You can get 250 free searches per month. Use this API Key to access all of our APIs, including the YouTube Video API.

Available parameters

In addition to running the basic search, you can view all YouTube Video API parameters here.

How to scrape YouTube video data with Python

  • Create a new main.py file
  • Install requests with: 
pip install requests
Enter fullscreen mode Exit fullscreen mode

Here is what the basic setup looks like:

import requests
SERPAPI_API_KEY = "YOUR_REAL_SERPAPI_API_KEY"

params = {
    "api_key": SERPAPI_API_KEY, #replace with the actual API Key
    # soon
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()
print(response)
Enter fullscreen mode Exit fullscreen mode

With these few lines of code, we can access all of the search engines available at SerpApi, including the YouTube Video API.

import requests
SERPAPI_API_KEY = "YOUR_SERPAPI_API_KEY"

params = {
    "api_key": SERPAPI_API_KEY, 
    "engine": "youtube_video",
    "v": "j3YXfsMPKjQ" # YouTube video ID
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()
print(response)
Enter fullscreen mode Exit fullscreen mode

To make it easier to see the response, let's add indentation.

import json

# ...
# ...
# all previous code

print(json.dumps(response, indent=2))
Enter fullscreen mode Exit fullscreen mode

Here is the result:

YouTube Video API response example

Here is how to scrape the comments on a video.

From the request we performed previously, you should be able to see this in the response.

comment page token response

We can scrape the "Top comments" or "Newest first" comments using the token that is available for each. We can put this token in the next_page_token parameter.

Here is an example:

params = {
    "api_key": SERPAPI_API_KEY, 
    "engine": "youtube_video",
    "v": "j3YXfsMPKjQ", # YouTube video ID
    "next_page_token": "Eg0SC2ozWVhmc01QS2pRGAYyOCIRIgtqM1lYZnNNUEtqUTABeAIwAUIhZW5nYWdlbWVudC1wYW5lbC1jb21tZW50cy1zZWN0aW9u"
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()
print(json.dumps(response, indent=2))
Enter fullscreen mode Exit fullscreen mode

Here is the result:

Scraping YouTube comments example

If you're interested in scraping the reply on the comment, you can repeat the same action, but this time using the replies_next_page_token.

Bonus

You can also scrape YouTube search results using our YouTube Search API. Here is how to scrape YouTube search results using Python.

Why Use It?

  • Real‑time data: Fetch up‑to‑date video details.
  • Rich metadata access: Retrieve comments and nested replies, related videos, and viewer stats in one request.
  • Automated proxy and captcha handling: SerpApi manages the complexities of scraping so you can focus on analysis.

Key Features

  • Feature: Comprehensive video metadata
    • Benefit: Includes title, description, duration, chapters, views, publishing date, channel, thumbnails
  • Feature: Comments & replies
    • Benefit: Access nested comments for engagement analysis
  • Feature: Related videos
    • Benefit: Discover context and competition around a video

Top comments (0)