DEV Community

Cover image for How to Scrape Twitter/X Without an API Key in Python (2026 Guide)
Yassine Ait Jeddi
Yassine Ait Jeddi

Posted on

How to Scrape Twitter/X Without an API Key in Python (2026 Guide)

#python #webscraping #twitter #tutorial

Twitter's official API now costs $200/month minimum just to read tweets (source). The free tier is write-only — you can post, but you can't search, pull timelines, or read anything except your own profile (source).

Here's the alternative: use your browser cookies to call the same GraphQL API that twitter.com uses internally. No API key. No developer account. No approval process.

If you're in a hurry, this is all it takes:

pip install scweet

from Scweet import Scweet

s = Scweet(auth_token="your_auth_token", proxy="http://user:pass@host:port")
tweets = s.search("python programming", since="2025-01-01", limit=100)
print(f"Got {len(tweets)} tweets")
print(tweets[0])
Enter fullscreen mode Exit fullscreen mode

Why Not the Official API?

X restructured its API pricing in February 2023, then raised prices again in October 2024. Here's what it looks like now:

Tier Price What you get
Free $0 Write-only. 500 posts/month. No search. No read access. Only endpoint: GET /2/users/me
Basic $200/month 15,000 read requests/month. 7 days of search history
Pro $5,000/month 1 million tweets. Full archive search
Enterprise $42,000+/month Custom. Compliance streams

Sources: TechCrunch, X Developer Community, xpoz.ai pricing breakdown

X also launched a pay-as-you-go model in February 2026, but plugging equivalent Basic-tier usage into it comes out to ~$575/month — not actually cheaper.

The approach in this guide costs $0 if you run locally, or about $0.30 per 1,000 tweets if you use the hosted cloud version.

How This Works

When you scroll your Twitter feed in a browser, the web app makes GraphQL API calls to X's backend. These calls are authenticated with cookies set when you log in — specifically, auth_token and ct0.

Scweet replays those same calls from Python. It sends the exact same requests the browser would, using curl_cffi to match the browser's TLS fingerprint. From X's perspective, it looks like normal browser activity.

No headless browser. No Selenium. No Playwright. Just HTTP requests with the right cookies.

What You'll Need

  • A Twitter/X account (free)
  • Python 3.9+
  • A proxy (strongly recommended — reduces rate limit hits and ban risk)

That's it.

Step 1: Install Scweet 

pip install -U scweet
Enter fullscreen mode Exit fullscreen mode

Step 2: Get Your auth_token

  1. Log in to x.com in Chrome or Firefox
  2. Open DevTools — press F12 (or right-click > Inspect)
  3. Go to Application > Cookies > https://x.com
  4. Find auth_token — copy the value

That's your credential. Scweet will automatically bootstrap the ct0 CSRF token from it, so you only need this one value.

Your auth_token stays valid for weeks to months. When it expires, Scweet raises an AuthError — just repeat this step with a fresh cookie.

Step 3: Your First Scrape 

from Scweet import Scweet

s = Scweet(
    auth_token="YOUR_AUTH_TOKEN",
    proxy="http://user:pass@host:port"
)

# Search for tweets about Bitcoin from 2025 onward
tweets = s.search("bitcoin", since="2025-01-01", limit=200, save=True)
print(f"Collected {len(tweets)} tweets")
Enter fullscreen mode Exit fullscreen mode

save=True writes the results to a CSV file automatically. You can also use save_format="json" or save_format="both".

Each tweet record includes: tweet_id, timestamp, text, likes, retweets, comments, tweet_url, user (screen_name, name), image_links, and the full raw GraphQL payload if you need it.

From the CLI (no Python code needed):

scweet --auth-token YOUR_AUTH_TOKEN --proxy http://user:pass@host:port \
    search "bitcoin" --since 2025-01-01 --limit 200 --save
Enter fullscreen mode Exit fullscreen mode

What Else Can You Scrape?

Tweet search is just one of the endpoints. Here's the full surface:

# Profile timeline — a user's own tweets
timeline = s.get_profile_tweets(["elonmusk"], limit=200)

# Followers
followers = s.get_followers(["elonmusk"], limit=1000)

# Following
following = s.get_following(["elonmusk"], limit=500)

# User info — bio, follower count, verification status, account creation date
profiles = s.get_user_info(["elonmusk", "openai"])
Enter fullscreen mode Exit fullscreen mode

All methods accept a list of usernames, so you can batch multiple targets in a single call. And every method has an async variant (asearch(), aget_profile_tweets(), aget_followers(), etc.) for async pipelines.

Scaling: Multiple Accounts

A single account handles hundreds to a few thousand tweets per day before hitting rate limits. For larger jobs, Scweet supports multi-account pooling.

Create a cookies.json with multiple accounts, each with its own proxy:

[
  {
    "username": "account_1",
    "cookies": { "auth_token": "..." },
    "proxy": "http://user1:pass1@host1:port1"
  },
  {
    "username": "account_2",
    "cookies": { "auth_token": "..." },
    "proxy": "http://user2:pass2@host2:port2"
  }
]
Enter fullscreen mode Exit fullscreen mode 
s = Scweet(cookies_file="cookies.json")
# Scweet rotates accounts automatically — handles rate limits, cooldowns, and retries
tweets = s.search("AI startups", limit=10000, save=True)
Enter fullscreen mode Exit fullscreen mode

Scweet manages the pool in SQLite: leases, heartbeats, daily counters, cooldowns, and automatic failover when an account hits its limit. You don't touch any of that — it just works.

If a long scrape gets interrupted, pass resume=True and it picks up exactly where it left off:

tweets = s.search("AI startups", limit=10000, save=True, resume=True)
Enter fullscreen mode Exit fullscreen mode

Don't Want to Manage Cookies or Accounts?

The Scweet Apify actor does everything above — but managed for you. No cookies, no account pooling, no proxy setup. It includes a free tier (up to 1,000 tweets/day) and scales to millions at $0.25 per 1,000 tweets.

You can run it from the Apify web UI, or call it programmatically via the Apify API:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("altimis/scweet").call(run_input={
    "search_query": "bitcoin",
    "max_items": 500,
})

tweets = client.dataset(run["defaultDatasetId"]).list_items().items
print(f"Got {len(tweets)} tweets")
Enter fullscreen mode Exit fullscreen mode

This is useful if you want to integrate Twitter data into a pipeline, a cron job, or an n8n/Zapier workflow without managing Python dependencies or browser cookies yourself.

Links

Top comments (1)

Collapse
 
kriptoburak profile image
Burak Bayır •

Useful walkthrough for local experiments, but “without an API key” is potentially misleading.

The auth_token is still a credential. It is a long-lived account session with broader consequences than a scoped read-only API key. Storing it beside proxy credentials in a plaintext cookies.json creates a significant security burden. It requires secret storage, rotation, access controls, redacted logs, and a reliable deletion process.

Account pooling also should not be confused with production scaling. SQLite leases and cooldowns coordinate workers, but they cannot prevent session expiry, verification challenges, account restrictions, IP reputation problems, or upstream GraphQL changes. Claims such as “thousands of tweets per day” and “valid for months” are observations, not stable contracts.

The pricing section also needs updating. X now documents pay-per-use access, so the fixed $200 minimum is no longer the complete comparison.

Xquik is another option for developers who need public X data without maintaining raw cookie files, account pools, proxy rotation, or GraphQL operation details. It provides structured access to tweet search, replies, profiles, timelines, followers, following, communities, lists, trends, articles, and media.

The production differences are important:

  • OpenAPI-defined request and response schemas
  • API key and Bearer authentication
  • Normalized JSON instead of raw GraphQL payloads
  • Opaque cursor pagination with explicit continuation state
  • Batch reads and resumable exports
  • Method-specific rate limits with Retry-After
  • Account and keyword monitors
  • Signed, retryable webhook deliveries
  • REST, SDK, CLI, and MCP integrations

Xquik also supports durable automated actions when an application later needs writes. Those actions use idempotency keys, status polling, terminal states, and explicit retry safety.

Scweet can be a practical choice for a personal Python script where the developer accepts session and account risk. Xquik is the stronger fit for SaaS products, data pipelines, monitoring systems, agencies, and AI agents that need a documented service contract.

Xquik remains an independent service, so it cannot eliminate upstream platform changes. Its value is absorbing that implementation work and presenting developers with a consistent integration surface.