Build your news aggregator with Apify and Twitter

Tech News is everywhere, and many of us feel overwhelmed by the sheer volume of exciting updates, especially around AI and technology.

I've always wished that I could gather the most interesting ones - especially groundbreaking announcements on Twitter - into a single pane of glass so that I could digest them without being distracted by comments and other visual elements when I'm on Twitter (or Facebook or Reddit or whatever).
Sometimes the real power is in simplicity.

After many experiments, I came up with a super easy way to do just that usingApify.

What is Apify

Apify is a web scraping and automation platform that lets users extract data from websites and automate browser tasks. It provides tools to build, run and share web scrapers (called "actors"), handle proxies, manage datasets, and process results via API. Users can either use pre-built actors or create custom ones using JavaScript/Node.js.

The Idea:

Automatically scrape the latest twitter (x.com) posts from companies I follow, e.g., Anthropic and OpenAI
Summarize the results using GPT
Enjoy the concise summary
(Optional) Send it to myself by email daily (not covered in this tutorial)

Sounds interesting? Here is how.

Step 1: Configure the Twitter Scraper

First, set up the Twitter scraper with your favorite brands. Here, we'll use OpenAI and Anthropic.

Configure the Twitter Scraper

{
    "filter:blue_verified": false,
    "filter:consumer_video": false,
    "filter:has_engagement": false,
    "filter:hashtags": false,
    "filter:images": false,
    "filter:links": false,
    "filter:media": false,
    "filter:mentions": false,
    "filter:native_video": false,
    "filter:nativeretweets": false,
    "filter:news": false,
    "filter:pro_video": false,
    "filter:quote": false,
    "filter:replies": false,
    "filter:safe": false,
    "filter:spaces": false,
    "filter:twimg": false,
    "filter:verified": false,
    "filter:videos": false,
    "filter:vine": false,
    "from": "elonmusk",
    "include:nativeretweets": false,
    "lang": "en",
    "maxItems": 30,
    "searchTerms": [
        "from:openai",
        "from:anthropic"
    ],
    "since": "2024-01-01_23:59:59_UTC"
}