How to Scrape Mastodon (Hashtags, Accounts & Trends) — Python + No-Code

#webscraping #mastodon #fediverse #python

Mastodon is the largest open, federated social network — millions of active users
across thousands of independent servers. And like the rest of the fediverse, its data
is refreshingly easy to get: every instance exposes a public REST API, no login
required for public content. If you do social listening or trend research, Mastodon
is a clean, ad-free, bot-light signal that almost no one is mining.

Why Mastodon?

Public API on every instance — mastodon.social, mas.to, fosstodon.org and thousands more, all with the same endpoints, no key.
Federated reach — query one instance or the broader federated timeline it sees.
Clean signal — real communities, hashtags and trends, with engagement counts.

Hashtag posts (Python, no auth)

import httpx
from bs4 import BeautifulSoup

base = "https://mastodon.social"
r = httpx.get(f"{base}/api/v1/timelines/tag/bitcoin", params={"limit": 40}, timeout=30)
for s in r.json():
    text = BeautifulSoup(s["content"], "lxml").get_text(" ", strip=True)
    print(text[:80], "—", s["favourites_count"], "favs")

Post bodies are HTML, so strip the tags for clean text. A user's posts come from
/api/v1/accounts/lookup?acct=Gargron → /api/v1/accounts/{id}/statuses; the public
or federated firehose from /api/v1/timelines/public; and what's hot right now from
/api/v1/trends/statuses.

The catch: boosts, paging & HTML

Pagination uses max_id (older items); boosts (reblogs) carry their real content in a
nested reblog object you need to unwrap; and every post body is HTML to clean. Doing
that across hashtags or accounts is the part worth automating.

The no-code option

The Mastodon Scraper on Apify
handles it — pick an instance and a mode (hashtag, account, public, trends, profile),
click Run.

{
  "mode": "hashtag",
  "instance": "mastodon.social",
  "hashtags": ["ai", "bitcoin"],
  "maxItems": 500
}

Output is one clean row per post — text (HTML stripped), date, language, boosts,
favourites, replies, hashtags, media URLs and author — ready for a spreadsheet, a
database, or an LLM.

Common use cases

Social listening — track a topic or brand across the whole fediverse.
Trend & sentiment analysis — feed hashtag/trend streams into an LLM.
Open-social research — study communities and how content federates.
Influencer & audience research — profile stats and posting activity.

FAQ

Do I need an account or API key? No for hashtags, accounts, public timelines and
trends. Only keyword search needs an access token.

Which instance? Any — mastodon.social is the largest and sees most of the
federated timeline; niche servers are great for niche communities.

Can I scrape remote accounts? Yes — use the full user@instance handle.

Is it legal? You're reading publicly available data via Mastodon's own public API.
Use it responsibly and within each instance's terms.

Building something with social data? The Mastodon Scraper handles the API for you. See also the Bluesky Scraper and Lemmy Scraper.

DEV Community