DEV Community

Ethan Teague
Ethan Teague

Posted on

How to Scrape Bluesky Data in 2026 (Posts, Profiles, and Search)

#api #python #socialmedia #webscraping

Note: this walkthrough uses a tool I built, but the API details apply to any Bluesky scraping project.

Bluesky is built on the AT Protocol, which means a lot of its data is available through a clean, documented, public API — no login required for most of it. Here's how to get posts, profiles, followers, and threads in 2026, plus the one thing that does need authentication.

What's open without a login

Bluesky's public AppView (https://public.api.bsky.app) serves these unauthenticated:

  • app.bsky.actor.getProfile — bio, follower/following/post counts
  • app.bsky.feed.getAuthorFeed — a user's posts (with reply/repost filters)
  • app.bsky.graph.getFollowers / getFollows — the social graph
  • app.bsky.feed.getPostThread — a post and its full reply tree

For example, a profile is one GET away:

curl "https://public.api.bsky.app/xrpc/app.bsky.actor.getProfile?actor=bsky.app"
Enter fullscreen mode Exit fullscreen mode

The one catch: keyword search needs auth

app.bsky.feed.searchPosts returns 403 unauthenticated. Any tool claiming anonymous Bluesky keyword search is quietly routing your queries through someone else's account. The correct way is to create a free app password (Bluesky → Settings → Privacy and security → App passwords — never your main password), call com.atproto.server.createSession on your PDS to get a token, and pass it as a Bearer header. App passwords can't manage your account and are revocable, so this is safe.

Handling the edge cases

  • Some profiles show handle.invalid (broken handle verification) — fall back to the DID for building URLs.
  • Post URLs use the record key: at://did/app.bsky.feed.post/<rkey> maps to bsky.app/profile/<handle>/post/<rkey>.
  • Paginate with the cursor field; back off on 429s.

The no-code way

I published a Bluesky Scraper on Apify that wraps all of the above into flat JSON/CSV — profiles, feeds, followers, threads, and (with your app password) keyword search.

curl -X POST "https://api.apify.com/v2/acts/ethanteague~bluesky-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"userHandles": ["bsky.app"], "maxPostsPerUser": 100}'
Enter fullscreen mode Exit fullscreen mode

It's $0.50 per 1,000 posts, and it's callable from Python/JS, on a schedule, or as an AI-agent tool via MCP.

What to build

  • Track a topic or brand across Bluesky
  • Export a follower graph for community analysis
  • Feed fresh social data to an AI agent
  • Archive threads before they're deleted

Questions or edge cases? Comments are open.

Top comments (0)