Mastodon's acct Field Hides the Domain You Already Know

#webscraping #python #api #apify

Quick answer

Mastodon's acct field — the one field everyone reaches for to get "which instance is this account on" — leaves the domain off entirely when the account lives on the instance you're already querying. We pulled the #python hashtag timeline from mastodon.social live: 8 of 40 accounts came back as a bare hugovalters or pycon_nl, no @domain anywhere, while the other 32 of 40 came back fully qualified, like pythonrennes@social.breizhcamp.org. Split acct on @ to get a domain and you'll get None for exactly the accounts that live on the server you already know the address of — which is the one case a naive parser is least likely to guard against, because it feels like it shouldn't be the edge case.

Verifying it live 🔬

curl -s "https://mastodon.social/api/v1/timelines/tag/python?limit=40" | \
  python3 -c "
import json, sys
data = json.load(sys.stdin)
accts = [s['account']['acct'] for s in data]
local  = [a for a in accts if '@' not in a]
remote = [a for a in accts if '@' in a]
print('local (no domain):', len(local), local[:3])
print('remote (has domain):', len(remote), remote[:3])
"

local (no domain): 8  ['hugovalters', 'sgofferj', 'pycon_nl']
remote (has domain): 32 ['pythonrennes@social.breizhcamp.org', 'buherator@infosec.place', 'hackaday@www.urbanmind.net']

This isn't a bug — it's documented Mastodon API behaviour, and it makes sense once you think about why: acct is meant to be what you'd type to follow someone from another instance, and "follow @hugovalters" is unambiguous when you're already talking to hugovalters's home server. But "unambiguous to the server" is not the same as "safe to split on @" in a client that assumes every account handle carries its own domain.

Where this actually bites

The dataset row this Actor ships carries account_acct exactly as the API returns it — we don't rewrite it, because doing so would hide the very distinction that matters: an unqualified acct is the signal that the account is local to instance. What we do instead is ship instance as its own top-level field on every row, set from the host you configured, precisely so you never have to reconstruct "which server is this account actually on" by parsing acct yourself. If you're building anything downstream that groups posts by home instance, that's the field to key on — not a regex against acct.

The same instance-scoping shows up in a second place worth knowing about before you build around it: Mastodon's /api/v1/timelines/public endpoint — the one that would give you a single global feed instead of walking hashtags/accounts one at a time — returned an HTTP 422 when we hit it against mastodon.social directly, unauthenticated, live. Plenty of smaller instances leave it open; the flagship one doesn't. That's why this Actor targets hashtag and account timelines specifically: they're the two query shapes that stay reliably open, unauthenticated, across the fediverse rather than working on some instances and not others.

What we handle for you 🛡️

Hashtag and account timelines share one rate-limit bucket per instance — 300 requests per 5 minutes on mastodon.social — so we pace requests against it and read both Retry-After and x-ratelimit-reset before backing off, rather than guessing at a fixed delay. Every request goes out with a real Chrome TLS handshake via curl-cffi impersonation, and pagination follows the Link: rel="next" header's max_id cursor rather than assuming offset-based paging exists. A hashtag that doesn't exist, or a handle you mistyped, gets logged and skipped — the run only fails outright if every configured query fails, so five typos in a batch of fifty hashtags cost you nothing but those five.

Output

One row per post, tagged with the query that produced it:

{
  "id": "117243600697061639",
  "instance": "mastodon.social",
  "source_type": "hashtag",
  "source_query": "python",
  "created_at": "2026-09-10T09:14:02.000Z",
  "content_text": "...",
  "account_acct": "hugovalters",
  "account_display_name": "Hugo Valters",
  "tags": [{"name": "python", "url": "https://mastodon.social/tags/python"}],
  "mentions": [],
  "reblogs_count": 1,
  "favourites_count": 4
}

from apify_client import ApifyClient

client = ApifyClient("<YOUR_API_TOKEN>")
run = client.actor("DevilScrapes/mastodon-posts-scraper").call(
    run_input={"instance": "mastodon.social", "hashtags": ["python"], "results_per_query": 40}
)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["account_acct"], "—", item["content_text"][:80])

Pricing: $0.20 per run plus $0.002 per post — about $2.20 per 1,000 posts. Zero matches for a hashtag/account is a normal, successful outcome; you pay only the start fee.

→ Mastodon Posts Scraper on Apify

Built by Devil Scrapes. We rotate real browser TLS fingerprints, pace against the shared rate-limit bucket, and skip a dead hashtag instead of failing your run — because a federated network with thousands of independently-run servers guarantees you'll eventually query one that's gone, misspelled, or just closed.

DEV Community