DEV Community

Tyr Xargaar
Tyr Xargaar

Posted on

Four pay-per-result YouTube and Google News scrapers on Apify (no API key)

#apify #youtube #python #webscraping

I built four small data tools on Apify that return YouTube and Google News data as JSON, without an API key or quota. You pay per result delivered, and nothing else. Here is what each one does and how to call it from Python.

The four tools

  1. YouTube Transcript Extractor - timestamped transcripts for videos, playlists and channels, with optional chunks sized for RAG. Videos without captions are reported and not charged. $0.004 per transcript. https://apify.com/xargaar_tools/youtube-transcript-extractor-batch
  2. YouTube Comments Scraper - comments (and optionally replies) with author, likes, reply count and publish time. $0.0015 per comment. https://apify.com/xargaar_tools/youtube-comments-scraper-fast
  3. YouTube Search Scraper - search results with title, channel, views, duration, publish date and thumbnail, with upload date and duration filters. $0.002 per video. https://apify.com/xargaar_tools/youtube-search-video-scraper
  4. Google News Scraper with Full Article Text - news results for a keyword, language and country, with the full article text pulled from the publisher page when it can be extracted. $0.003 per headline, $0.01 per article with text. https://apify.com/xargaar_tools/google-news-scraper-article-text

YouTube blocks datacenter IPs, so the YouTube tools run through a residential proxy by default. That cost is already in the per-result price.

Calling one from Python 

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("xargaar_tools/youtube-comments-scraper-fast").call(run_input={
    "startUrls": ["https://www.youtube.com/watch?v=dQw4w9WgXcQ"],
    "maxCommentsPerVideo": 50,
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["likes"], item["author"], item["text"][:80])
Enter fullscreen mode Exit fullscreen mode

Swap the Actor name and input to use the others. Every Actor has an input form in the Apify console if you prefer clicking to coding.

What I tested

I ran each Actor on Apify's platform before publishing. Transcripts: 11 of 12 test videos returned a transcript, a private video was reported cleanly instead of failing the run. Comments: 45 comments from one video, replies mode, and a channel URL across two videos. Search: two queries returned 80 videos, and the date and duration filters changed the results as expected. News: 10 results, 6 with full article text.

Known limits

  • Comments come back in YouTube's top-comments order. A "newest first" sort is not supported yet.
  • The search tool applies one filter per query (upload date or duration).
  • News articles behind paywalls or bot protection return the headline data without full text.

If something breaks or you need a field that is missing, open an issue on the Actor page and I will look at it.

Xargaar Tools builds small, pay-per-use data tools.

Top comments (0)