DEV Community

Himanshu Agrawal
Himanshu Agrawal

Posted on

How to Scrape Tweets from X (Twitter) Without Paying for Empty or Fake Results

Disclosure: I built the X Twitter Scraper used in this guide and earn from its usage on Apify.

If you have ever paid for a batch of scraped tweets and opened the file to find half of it empty, duplicated or filled with sample rows, this guide is for you. I will show how to collect tweets from X (Twitter) by keyword, profile, link or replies, how to keep the numbers exact, and how to check that every row you paid for is a real tweet.

We will use the X Twitter Scraper on Apify. You need a free Apify account and nothing else: no X account, no developer app, no keys from X.

Why scraped tweet data goes wrong

Before the steps, it helps to know the usual failure points:

  • Placeholder rows. Data sources sometimes return demo or placeholder items when a search has no real results. If a tool passes them through, you pay for rows that never existed.
  • Loose limits. You ask for 100 tweets and get 140, because pages come in fixed sizes. The extra 40 are on your bill.
  • Date filters that are only hints. A query with since: and until: can still return tweets outside the range, and you pay for those too.
  • Duplicates. Two search terms that overlap return the same tweet twice.
  • Silent stops. A run ends with 12 rows and no reason, so you cannot tell if there was no data or something failed.

The scraper in this guide handles each of these in the Actor itself: placeholder rows are dropped and counted, Max rows per input item is exact, dates and filters are checked on every tweet, duplicates are saved once, and a SUMMARY record says why each input stopped. Rows that are dropped or skipped are not charged.

Step 1: Open the Actor in Apify Console

  1. Go to the X Twitter Scraper Store page.
  2. Click Try for free and sign in or create an Apify account.
  3. You land on the Input tab in Apify Console.

Step 2: Pick your sources

You can mix several sources in one run:

  • Search terms: any X search, one per line. Advanced operators work, so openai lang:en -filter:replies min_faves:20 is valid.
  • Profiles: nasa, @nasa or https://x.com/nasa to get that account's tweets. Turn on Include the profile's own replies if you need them.
  • Tweet links or IDs: fetch specific tweets to refresh their likes and views.
  • Replies of tweets: paste a tweet link to get the replies under it. This works best on the first tweet of a thread.
  • Followers of / Following of: handles whose followers or followed accounts you want, one row per user.

For this example, enter one search term: "customer support" (from:zendesk OR from:intercom). Replace it with your own topic.

Step 3: Set exact limits and filters

  • Max rows per input item: for example 200. This is per search term, profile or handle, and you never get or pay for more.
  • Max rows for the whole run: a firm total across all inputs, handy when you run many search terms.
  • From date / To date: inclusive, in UTC. The dates are also sent to the data source, so fewer requests are needed than paging back by hand.
  • Min likes, retweets, replies, views: skip low-engagement tweets.
  • Language, Exclude retweets, Exclude replies, Only tweets with media, Only verified authors: narrow it further.

Tweets that fail a filter are skipped and not charged.

Step 4: Run and read the SUMMARY

Click Start. Rows appear in the Output tab as the run works. When it finishes, open the Storage tab, then the key-value store, and open the SUMMARY record. For each input it lists:

  • rows delivered
  • rows skipped by dates or filters
  • duplicates
  • placeholder rows dropped
  • why it stopped: limit reached, no more results, provider error, spending limit or free plan limit

If a number looks low, this record tells you whether the search simply had fewer tweets or a filter removed them.

Step 5: Export

From the Output tab, export as JSON, CSV or Excel. Each tweet row has text, date, language, likes, retweets, replies, quotes, bookmarks, views, media with video links, quoted and retweeted tweets, and author details. IDs are strings, so long tweet IDs do not get rounded in Excel or JavaScript. When X shows no view count, views is null rather than a fake 0.

What it costs

$0.25 per 1,000 tweets from searches, profiles and replies; $2 per 1,000 tweets fetched by link ($3.50 from 12 November 2026, because each one is a separate request); $0.25 per 1,000 followers or following. There is no start fee or per-run fee, the price is the same on every Apify plan, and the free Apify plan allows up to 100 results per run (10 tweets fetched by link) so you can test it.

Run it from the API with curl

Get your Apify token from Console settings and keep it in an environment variable. This call starts a run and returns the rows when it finishes:

curl -X POST \
  "https://api.apify.com/v2/acts/apisamurai~x-twitter-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchTerms": ["openai lang:en -filter:replies"], "maxItems": 50, "startDate": "2026-10-01"}'
Enter fullscreen mode Exit fullscreen mode

Run it from Python

Install the client with pip install apify-client:

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("apisamurai/x-twitter-scraper").call(run_input={
    "profiles": ["nasa"],
    "maxItems": 100,
    "excludeRetweets": True,
    "minLikes": 50,
})

for tweet in client.dataset(run.default_dataset_id).iterate_items():
    print(tweet["createdAt"], tweet["likes"], tweet["text"][:80])

summary = client.key_value_store(run.default_key_value_store_id).get_record("SUMMARY")
print(summary["value"])
Enter fullscreen mode Exit fullscreen mode

The last two lines print the same SUMMARY you saw in Console, so a script can log why a run returned fewer rows.

Integration idea: a daily mentions sheet with n8n

  1. In Apify Console, save a task with your brand as the search term, Search sort set to Latest and Max rows per input item set to 100, then add a Schedule that runs it every morning.
  2. In n8n, add the Apify node with the trigger "Actor run finished" for this Actor.
  3. Add a step that reads the dataset items of that run.
  4. Append each row to Google Sheets with author, text, likes, views and URL.
  5. Optionally add a filter step that sends tweets above 1,000 views to Slack.

The same flow works in Make or Zapier with their Apify apps.

Checklist before a big run

  • Test the query on x.com search first. If X shows nothing, the scraper will not either.
  • Use dates instead of paging through old tweets.
  • For one account's long history, search from:handle with a date range. Profile timelines on X only go back about 3,200 tweets.
  • Set a spending limit on the run. When it is reached, the run stops cleanly.
  • Read the SUMMARY after every run until you trust your setup.

A note on rules

Collect only public data, follow X's terms and the laws where you live, and take care with personal data such as usernames under rules like GDPR. This is not legal advice.


I publish this and other pay-per-result data tools as API Samurai. The full catalog, with prices and code examples for each tool, is at apisamurai.com.

Top comments (0)