Ever since Elon Musk killed off free API access in early 2023, scraping Twitter (now X) has become one of the most frustrating challenges in web scraping. The platform that was once the easiest social network to pull data from is now one of the hardest.
In this guide, I'll walk through every practical method available in 2026 — what works, what doesn't, what it costs, and where the traps are.
The Current State of Twitter/X Data Access
Let's be blunt: there is no free, reliable, officially-supported way to get Twitter data at scale in 2026.
Here's what happened:
- February 2023: Twitter shut down free API v1.1 access
- March 2023: API v2 moved to paid tiers ($100/month minimum for basic access)
- Late 2023-2024: Aggressive anti-scraping measures rolled out (rate limiting, bot detection, IP bans)
- 2025-2026: Continued tightening — even logged-in browser sessions get throttled after moderate usage
If anyone tells you "just use this free Python library and you're good," they're either outdated or lying. Every method has trade-offs.
Method 1: Twitter API v2 (Official, Paid)
The official API is the most reliable option, but it's expensive.
Pricing Tiers (as of 2026)
| Tier | Cost | Tweet reads/month | Write access |
|---|---|---|---|
| Free | $0 | 1 app, limited | Write only (1,500 tweets/month) |
| Basic | $100/mo | 10,000 reads | Yes |
| Pro | $5,000/mo | 1,000,000 reads | Yes |
| Enterprise | Custom | Unlimited | Full firehose |
Basic Python Example
# Implementation is proprietary (that IS the moat).
# Skip the build — use our ready-made Apify actor:
# see the CTA below for the link (fpr=yw6md3).
Pros: Reliable, structured data, legal
Cons: $100/month minimum for reads, strict rate limits, limited historical data on Basic tier
When to Use the Official API
- You need real-time data (streaming endpoints)
- You're building a product that depends on Twitter data
- Compliance matters (academic research, business use)
- You only need a small volume (< 10K tweets/month)
Method 2: Alternative Frontends (Nitter-Style)
Nitter was an open-source alternative Twitter frontend that let you view tweets without JavaScript or a Twitter account. At its peak, dozens of public instances existed.
Current Status (2026)
Most public Nitter instances are dead or unreliable. Twitter aggressively blocks the guest API endpoints that Nitter depended on. Some self-hosted instances still work intermittently, but:
- They break every few weeks when Twitter changes something
- Rate limits are severe
- No guarantee of data completeness
- Running your own instance requires constant maintenance
# Implementation is proprietary (that IS the moat).
# Skip the build — use our ready-made Apify actor:
# see the CTA below for the link (fpr=yw6md3).
Verdict: Don't build anything that depends on Nitter in 2026. It's a cat-and-mouse game you'll lose.
Method 3: Browser Automation (Selenium/Playwright)
This approach uses a real browser to load Twitter, scroll through content, and extract data from the rendered page.
# Implementation is proprietary (that IS the moat).
# Skip the build — use our ready-made Apify actor:
# see the CTA below for the link (fpr=yw6md3).
The Honest Truth About Browser Scraping Twitter
- You need to be logged in — Twitter shows almost nothing to logged-out visitors
- Account bans are common — Twitter detects automation patterns and suspends accounts
- It's slow — each page load takes seconds, scrolling takes more seconds
- Anti-bot detection — Twitter uses sophisticated fingerprinting
- Selectors break constantly — Twitter uses obfuscated class names that change regularly
Method 4: Proxy-Based Scraping Services
If you're serious about scraping Twitter at any scale, you'll need residential proxies to avoid IP bans.
ThorData offers residential proxy networks that rotate IPs automatically, which is essential when Twitter rate-limits per IP. For a more managed approach, ScraperAPI handles proxy rotation and browser rendering in one API call.
# Implementation is proprietary (that IS the moat).
# Skip the build — use our ready-made Apify actor:
# see the CTA below for the link (fpr=yw6md3).
Method 5: Pre-Built Scraping Actors
If you don't want to build and maintain your own scraper, platforms like Apify offer pre-built actors that handle the complexity for you. I maintain some data collection actors on Apify that handle the infrastructure side — proxy rotation, browser management, anti-detection, and data formatting.
The advantage here is you get structured JSON output without dealing with any of the browser automation headaches.
What Method Should You Choose?
Here's my honest recommendation:
| Scenario | Best Method | Monthly Cost |
|---|---|---|
| Small volume, need compliance | Twitter API v2 Basic | $100 |
| Academic research | Twitter API v2 (Academic tier) | Apply for access |
| One-off data collection | Pre-built actors (Apify) | Pay per use |
| Large-scale monitoring | Twitter API Pro + proxies | $5,000+ |
| Quick prototype/experiment | Browser automation + proxies | $30-50 (proxy costs) |
Legal Considerations
This is not legal advice, but you should know:
- Twitter's Terms of Service prohibit scraping
- The 2022 hiQ v. LinkedIn ruling established that scraping publicly accessible data is not a CFAA violation — but Twitter's data isn't fully "publicly accessible" anymore (login walls)
- The EU's GDPR applies to personal data regardless of how you collected it
- If you're using scraped data commercially, consult a lawyer
Key Takeaways
- There's no free lunch. Every method costs either money, time, or both.
- The official API is the safest bet if you can afford it and the rate limits work for your use case.
- Browser automation works but requires constant maintenance and proxy infrastructure.
- Pre-built solutions save massive amounts of development time.
- Budget realistically — between proxies like ThorData, API costs, and infrastructure, plan for $50-200/month minimum for any serious data collection.
The days of import tweepy; api.search() for free are gone. But with the right approach and realistic expectations, you can still get the Twitter data you need.
Have questions about scraping Twitter/X? Drop a comment below — I'll share what's worked (and what hasn't) from my experience.
Top comments (0)