DEV Community

Laura Smith for ScrapeHunt

Posted on

Scrape ALL latest news on any company/person within 3 lines of Python code

Step 1. Install pygooglenews

$ pip install pygooglenews
Enter fullscreen mode Exit fullscreen mode

Step 2. Three lines to get news data

from pygooglenews import GoogleNews

gn = GoogleNews(lang = 'en', country = 'US')

# latest news on Amazon that got published over the last hour
news = gn.search('Amazon', when = '1h')
Enter fullscreen mode Exit fullscreen mode

Why is it cool?

  1. You get data from Google โ€” the best search engine
  2. pygooglenews hits RSS feed URL (not the UI URL). So, you don't get blocked by scraping Google

  3. Googleโ€™s RSS can have up to 100 articles. But, Google allows you to specify that you want to see only the articles from the past hour.

Make such a request a few times an hour, and you will not miss a single news article that mentions your company/person of interest.

Discussion (0)