DEV Community: data show

Pull Public X (Twitter) Data in Python Without an X Developer Account, Then Hand It to Your AI Agent

data show — Sat, 24 Oct 2026 15:10:28 +0000

I keep running into the same small job: "grab the last few hundred posts about a topic, plus the followers of a couple of accounts, and put them in a spreadsheet." It's not a big data project. It's an afternoon script. But X data has become awkward for exactly this size of task. Browser scrapers need logged-in accounts and proxies and break whenever the site changes, and the official API bills every post and profile it returns.

This post walks through a small Python script that does both jobs with a pay-as-you-go REST API, TwexAPI, and then shows how to give the same access to an AI agent through MCP. Everything here uses public data and endpoints that are documented in its API reference.

Disclosure and scope: TwexAPI is an independent third-party service. It is not affiliated with or endorsed by X Corp. Collect only public data, and check privacy law and X's terms for your own use case.

What you need

Python 3 and requests
A TwexAPI API key. New accounts get 20,000 free Credits with no credit card, valid for 7 days. A typical post or profile read costs 5 Credits, so that's roughly 4,000 reads to experiment with.

pip install requests
export TWEXAPI_KEY="twexapi_xxx"

Authentication is a single header, Authorization: Bearer <your key>, sent to https://api.twexapi.io. No OAuth dance and no app review.

Step 1: A tiny cursor paginator

Both endpoints we'll use are POST endpoints with cursor paging. You send a JSON body, get back a page of results in data, and keep going while has_next_page is true, passing next_cursor back in.

import os
import time
import requests

API = "https://api.twexapi.io"
HEADERS = {"Authorization": f"Bearer {os.environ['TWEXAPI_KEY']}"}


def paginate(path, body, max_items, delay=5):
    """POST a cursor endpoint until we have max_items or hit the last page."""
    items, cursor = [], ""
    while len(items) < max_items:
        res = requests.post(
            f"{API}{path}",
            headers=HEADERS,
            json={**body, "next_cursor": cursor},
            timeout=30,
        )
        res.raise_for_status()
        page = res.json()
        items += [row for row in page.get("data", []) if row]
        cursor = page.get("next_cursor")
        if not page.get("has_next_page") or not cursor:
            break
        time.sleep(delay)
    return items[:max_items]

The delay=5 matters on the free tier. The Starter plan is limited to 0.2 requests per second, i.e. one request every five seconds. Paid plans start at 5 QPS, so you can drop the delay once you upgrade.

Step 2: Search posts with normal X search operators

POST /twitter/advanced_search/page takes a list of searchTerms and a sortBy of "Latest" or "Top". Each page returns up to 20 posts. The search terms accept the same operators as the X search box, so you can combine a keyword with lang:, since:, min_faves: and -filter:replies.

tweets = paginate(
    "/twitter/advanced_search/page",
    {
        "searchTerms": ["postgres", "lang:en", "min_faves:50",
                        "since:2026-10-01", "-filter:replies"],
        "sortBy": "Latest",
    },
    max_items=200,
)
print(len(tweets), "posts")

A few operators I use constantly:

Operator	Example	What it does
`from:`	`from:nasa`	Only posts by one account
`since:` / `until:`	`since:2026-10-01`	Date window
`lang:`	`lang:en`	Language filter
`min_faves:`	`min_faves:100`	Skip low-engagement noise
`-filter:replies`		Drop replies

Filtering on the query side is the cheapest optimization you have. You pay per item returned, so a tighter query is both faster and cheaper than pulling everything and filtering in pandas.

Step 3: Export a follower list

POST /twitter/followers/page takes a screen_name and returns up to 200 followers per page. Because it uses the same cursor pattern, the helper works unchanged:

followers = paginate(
    "/twitter/followers/page",
    {"screen_name": "nasa"},
    max_items=1000,
)

There's a matching POST /twitter/following/page if you want who an account follows instead.

Step 4: Write both to CSV

import csv

with open("posts.csv", "w", newline="", encoding="utf-8-sig") as f:
    w = csv.writer(f)
    w.writerow(["tweet_id", "created_at", "user", "likes", "text"])
    for t in tweets:
        w.writerow([
            t.get("tweet_id"),
            t.get("created_at"),
            (t.get("user") or {}).get("screen_name"),
            t.get("favorite_count"),
            (t.get("text") or "").replace("\n", " "),
        ])

with open("followers.csv", "w", newline="", encoding="utf-8-sig") as f:
    w = csv.writer(f)
    w.writerow(["username", "name", "followers", "location", "bio"])
    for u in followers:
        w.writerow([u.get("username"), u.get("name"),
                    u.get("followersCount"), u.get("location"),
                    u.get("description")])

utf-8-sig is a small trick worth remembering: it writes a byte-order mark so Excel opens emoji and non-Latin names correctly instead of showing mojibake.

What did that cost?

200 posts plus 1,000 followers is 1,200 reads. At 5 Credits each that's 6,000 Credits, which fits comfortably inside the free 20,000. In dollar terms, 100,000 Credits is about $1, so the run is worth about six cents. Two billing details are useful to know when you're debugging: a request that comes back with no results still costs 5 Credits, and a request that fails with an error isn't billed. If you outgrow the trial, the pricing page lists one-time packs starting at $20 and monthly plans starting at $50.

Step 5: Give the same access to an AI agent (MCP)

Scripts are great for repeatable exports. For exploratory questions like "what are people in Japan posting about this launch?" it's often faster to let an agent figure out the calls. TwexAPI runs a hosted MCP server at https://api.twexapi.io/mcp (Streamable HTTP), authenticated with your API key in an x-api-key header.

For Cursor, add this to .cursor/mcp.json in a project (or ~/.cursor/mcp.json globally):

{
  "mcpServers": {
    "twexapi": {
      "url": "https://api.twexapi.io/mcp",
      "headers": {
        "x-api-key": "twexapi_YOUR_KEY_HERE"
      }
    }
  }
}

Claude Code uses the same shape in .mcp.json with "type": "http" added. Claude Desktop only speaks stdio, so the docs bridge it with the mcp-remote npm package. The MCP setup docs have configs for Codex CLI, VS Code, Windsurf and others.

The server deliberately exposes just two tools:

explore searches the endpoint catalog and returns names, methods, paths, parameter schemas, examples and a read_only flag.
twexapi_request executes an authenticated call with a method, a relative path, and an optional query or body.

So the agent first discovers the right endpoint, then calls it. I like this design because the tool list stays tiny (it doesn't flood the context window with dozens of tool definitions), and the read_only flag gives you something concrete to check before you let an agent run unattended.

A prompt that works well for a first test:

"Use twexapi to list the countries available for global trends, then show me the top trending posts for Japan."

Behind the scenes that maps to GET /twitter/global-trending/countries and GET /twitter/global-trending/tweets?country=....

A few practical notes

Keep the key server-side. Don't ship it in front-end code or commit it. The SDKs and CLI read it from the X_API_SCRAPER_KEY environment variable if you'd rather not hard-code anything.
Back off on 429s. If you bump into rate limits, sleep and retry rather than hammering. The docs have an error-handling guide for this.
Reads don't need an X account. Write actions (posting, liking, DMs) are a separate topic: they run with your own account's session, and I'd keep any agent on read-only endpoints unless a human reviews each action.

Wrapping up

For small and medium jobs, the pattern above covers most of what I need from X: a search, a follower export, and a CSV, in about 60 lines of Python. Then the same key can power an agent through MCP when the question is fuzzier. Start with a narrow query, check the Credits you used, and only then scale up.

Why X (Twitter) Doesn't Show When Someone Followed an Account, and How to See Recent Follows Anyway

data show — Tue, 20 Oct 2026 12:45:44 +0000

If you've ever tried to answer a simple question like "Who has this founder started following this month?", you've probably hit a wall. X (formerly Twitter) will show you who an account follows, but never when they followed them. No sorting, no filters, no dates.

I ran into this while doing creator discovery and competitive research. This post covers what is and isn't knowable about recent follows, practical ways to get at that data, and some ground rules for doing it responsibly.

What X actually shows you

Open any profile, click Following, and you get a long, scrollable list of accounts. That's it. Here's what's missing:

No follow timestamps. Nothing in the UI tells you when a follow happened.
No sort options. You can't sort by date, name, or follower count.
No filters or search inside the list. You can't search one account's following list for a keyword or a specific handle.
Login required. Logged-out visitors are generally asked to sign in before they can browse following and follower lists.

So the only built-in signal is the order of the list.

The ordering: roughly reverse-chronological, but not a guarantee

In practice, following lists tend to show the most recently followed accounts near the top, which is why "scroll the top of their Following list" is the usual advice. But be honest about what that order does and doesn't tell you:

What you can reasonably infer

Accounts near the top were probably followed more recently than accounts further down.
If you check the same list at two points in time, new entries at the top are likely new follows.

What you can't know from the list alone

The exact date or time of any follow. Position isn't a timestamp.
Whether the order is strictly chronological. X doesn't document this as a guarantee, and behavior can change without notice.
Unfollow and re-follow history. If someone unfollowed and followed again, you can't see that.
Removed accounts. Deactivated or suspended accounts can disappear from lists and shift positions.

The only reliable way to learn when a new follow appeared is to observe the list over time and compare snapshots. Any tool or method claiming to show exact historical follow dates for arbitrary accounts should be treated with skepticism.

Notifications don't help much either

If it's your account, you get a notification when someone follows you. But notifications aren't an archive:

They only keep a limited number of recent events.
Follow notifications are often grouped ("X and 12 others followed you").
They only cover follows of your account, not who other accounts are following.

Why you might care: real use cases

This isn't about snooping. There are practical reasons to look at recent follows on public accounts:

Creator discovery. People you trust in your niche are good filters. If a respected developer just followed five new accounts, those accounts are often worth a look.
Competitive research. Watching who a competitor's official account follows can hint at new partnerships, hires, integrations, or markets they're paying attention to.
Trend monitoring. When several VCs or well-known builders start following the same new project around the same time, that's an early signal worth noting. Not proof, but a useful prompt to dig further.
Your own audience. Seeing who recently followed you helps you spot collaborators, customers, or communities you didn't know you were reaching.

Approach 1: Manual scrolling

The free, zero-setup option.

Log in to X.
Open the target profile and click Following.
Look at the first 20–50 entries.
Optionally screenshot or copy the handles so you can compare later.

Pros: free, no tools, works right now.
Cons: tedious, needs an X login, and you have to keep your own snapshots to detect changes. It doesn't scale beyond a couple of accounts.

Approach 2: The official X API

If you're a developer, the obvious move is the API. X offers endpoints for retrieving the accounts a user follows and the accounts following them.

Things to know before you start:

There are no follow timestamps here either. The API returns lists of users, not dated follow events. You'd still have to poll and diff results yourself to detect new follows.
Access is paid and tiered. X has changed its API pricing and plans several times. Free access is very limited, and follow-graph endpoints usually need a paid plan. Check the current developer portal before you design around it, since what's included at each level changes.
Rate limits are tight. If you're tracking many accounts, you'll hit limits fast.
You maintain the pipeline. Auth, storage, scheduling, diffing, and handling deleted or suspended accounts are all on you.

A minimal "detect new follows" loop looks something like this in pseudocode:

every N hours:
  current = fetch_following(user_id)        # list of user IDs
  previous = load_snapshot(user_id)
  new_follows = current - previous          # set difference
  save_snapshot(user_id, current)
  record(new_follows, detected_at=now())    # your timestamp, not X's

Note the comment: the date you record is when you detected the follow, not when it happened. That's the most accuracy you can honestly claim.

For production monitoring this is the right path. For occasional research, it's a lot of overhead.

Approach 3: Use a lookup tool

If you just want to check what a public account has been following lately without writing code or logging into X, a lookup tool is the practical middle ground.

Full disclosure: I built Recently Followed, a free web tool for researching public X accounts. You enter a username and review the newest following and follower data available for that account. A few details that matter:

No X login needed. You sign in with a free Recently Followed account instead, which comes with a daily allowance of free searches.
The checked account isn't notified.
Public accounts only. Protected accounts aren't available, which is how it should be.

The same honesty rules apply: you're seeing the latest available data, not a timestamped history, so treat the order as a strong hint rather than a precise timeline.

The site also has some free browser-based X tools that come in handy in research workflows: a video and GIF downloader, a thread-to-Markdown reader (nice if you keep notes in Obsidian or Notion), follower export, and trending topics.

Ethics and privacy: ground rules

Follow data is public, but that doesn't mean anything goes. A few rules I stick to:

Public accounts only. If someone has protected their account, respect that. Don't try to work around it.
Research, not surveillance. Looking at what industry accounts follow is normal market research. Obsessively tracking a private individual is not.
Don't harass or call people out. A follow isn't an endorsement, and inferring relationships or intentions from it can be wrong and hurtful.
Respect platform terms and store less. Use sanctioned access, and keep only the snapshot data you need.

Quick checklist

Before you go researching recent follows, run through this:

[ ] Is the account public?
[ ] Do I have a clear, legitimate reason (discovery, research, trends)?
[ ] Am I treating list order as a hint, not a timestamp?
[ ] For tracking over time, am I saving snapshots and diffing them?
[ ] If using the API, have I checked the current plan limits and pricing?
[ ] Am I avoiding conclusions about people based on a single follow?

Wrapping up

X doesn't show follow dates. The list's rough reverse-chronological order is the only built-in signal, and it's useful as long as you don't over-trust it. For a quick look, scroll manually. For serious monitoring, build on the API and diff snapshots. For everyday research without the setup, a lookup tool gets you most of the way there.

Whatever you use, keep it to public accounts and keep it respectful. Have you found other good ways to spot what's new in someone's follow graph? Tell me in the comments.

Building AI Agents? The LLM isn't your bottleneck—Data Ingestion is.

data show — Tue, 16 Jun 2026 08:20:11 +0000

As an indie developer, I’ve spent the last few months deep in the trenches building automation workflows and experimenting with different AI agent frameworks.

But the further I got, the more I hit a frustrating wall.

Models are incredibly smart now, but if you can't feed them clean, real-time external data, they're practically useless.

The "Dirty Work" of AI Development

This became painfully obvious when I needed to pull social data—specifically from X/Twitter (like targeted follower lists, engagement metrics, or structured comment threads)—to feed into my AI for context analysis.

Here is what usually happens:

The Official Route: The official API is prohibitively expensive for indie hackers and side projects.
The Custom Scraper Route: Maintaining custom scrapers means fighting an endless, exhausting war against rate limits, proxies, and anti-bot systems.
The Data Format Issue: Even if you get the data, it's usually a messy HTML/JSON soup that eats up your LLM's context window and leads to hallucinations.

Scratching My Own Itch

I got so tired of this "dirty work" stalling my core application logic that I paused my main project to just build the infrastructure I wished existed.

My goal was simple: abstract away all the complex scraping and proxy routing, and just deliver clean, structured data.

I recently finished packaging it up, and it now fully supports CLI commands and direct AI tool calling. This means you can now seamlessly plug it directly into Cursor, Claude, or your custom agent scripts without writing a single line of scraping logic. It outputs clean JSON or CSV, ready for your LLM to digest.

If you’re also building AI tools and banging your head against the wall trying to ingest X data reliably, you can check out the infrastructure I built here: Twexapi.

Let's Discuss

Outsource the data extraction, and save your context window (and your sanity) for the actual application logic.

How are you all handling dynamic social data ingestion for your agents right now?
Do you build your own scrapers or rely on third-party APIs?

Would love to hear your stack and experiences in the comments! 👇