DEV Community

Ben
Ben

Posted on

How to Scrape Lemmy — the Federated Reddit Alternative (Python + No-Code)

#webscraping #lemmy #reddit #python

When Reddit locked down its API, a chunk of its communities moved to Lemmy — an
open, federated Reddit alternative. The best part for anyone who needs discussion
data: Lemmy's API is completely public, no key, no login, and it returns clean
JSON with Reddit-style metrics (score, upvotes, downvotes, comments). If you build
RAG datasets, do social listening, or track communities, Lemmy is an easy, untapped
source.

Why Lemmy?

  • Open API — every instance (lemmy.world, lemmy.ml, beehaw.org, …) exposes /api/v3/ with no authentication for public data.
  • Federated — query one instance or the whole network; reach cross-instance communities like technology@lemmy.world.
  • Reddit-shaped data — posts, comments, communities, scores and vote counts, so it slots straight into anything you built for Reddit.

Front-page or community posts (Python) 

import httpx

base = "https://lemmy.world"
# front page (whole federated network), sorted Hot:
r = httpx.get(f"{base}/api/v3/post/list",
              params={"sort": "Hot", "type_": "All", "limit": 50}, timeout=30)
for it in r.json()["posts"]:
    p, c = it["post"], it["counts"]
    print(p["name"], "—", c["score"], "pts,", c["comments"], "comments")
Enter fullscreen mode Exit fullscreen mode

Want a specific community? Add community_name=technology. Want comments? Hit
/api/v3/comment/list. Want to find communities? /api/v3/community/list or
/api/v3/search?type_=Communities.

The catch: pagination & normalization

Lemmy paginates with page=N (up to 50 per page), and each item nests post,
creator, community and counts objects you'll want flattened into one clean row.
Across multiple communities and sort windows, that's the part worth automating.

The no-code option

The Lemmy Scraper on Apify does
it for you — pick a mode (posts, community, search, comments, communities), an
instance and a sort order, click Run.

{
  "mode": "community",
  "instance": "lemmy.world",
  "communities": ["technology", "asklemmy"],
  "sort": "TopWeek",
  "maxItems": 500
}
Enter fullscreen mode Exit fullscreen mode

Output is one clean row per post (or comment/community) — title, body, link, score,
upvotes, downvotes, comment count, community, author and URLs — ready for a
spreadsheet, a database, or an LLM.

Common use cases

  • Reddit-migration research — follow communities and audiences that left Reddit.
  • Community & topic monitoring — track discussions across the fediverse.
  • RAG / LLM datasets — clean, open, license-friendly discussion data.
  • Trend & sentiment analysis — feed posts and comments into an LLM.

FAQ

Do I need an account or API key? No — public posts, comments and communities work
with no login.

Which instance should I use? Any. lemmy.world is the largest; with
listingType: All it sees most of the federated network.

Can I scrape a community on another instance? Yes — use community@instance.

Is it legal? You're reading publicly available data via Lemmy's own public API.
Use it responsibly and within each instance's terms.

Building something with discussion data? The Lemmy Scraper handles the API so you can focus on the product. See also the Reddit Scraper and Bluesky Scraper.

Top comments (0)