DEV Community

loading...
ScrapeHunt

Scrape ALL latest news on any company/person within 3 lines of Python code

Laura Smith
・1 min read

Step 1. Install pygooglenews

$ pip install pygooglenews
Enter fullscreen mode Exit fullscreen mode

Step 2. Three lines to get news data

from pygooglenews import GoogleNews

gn = GoogleNews(lang = 'en', country = 'US')

# latest news on Amazon that got published over the last hour
news = gn.search('Amazon', when = '1h')
Enter fullscreen mode Exit fullscreen mode

Why is it cool?

  1. You get data from Google — the best search engine
  2. pygooglenews hits RSS feed URL (not the UI URL). So, you don't get blocked by scraping Google

  3. Google’s RSS can have up to 100 articles. But, Google allows you to specify that you want to see only the articles from the past hour.

Make such a request a few times an hour, and you will not miss a single news article that mentions your company/person of interest.

Discussion (0)

Forem Open with the Forem app