DEV Community

Coco
Coco

Posted on

How to scrape Facebook in 2026?

#webscraping #python #tutorial #api

Three Python functions that scrape a public Facebook page, the payload behind a public post URL, and public group metadata, using the Chocodata API. Every snippet below was executed on 27 July 2026 and the output blocks come from that run.

Here is what the finished script prints:

NASA - National Aeronautics and Space Administration  id=100044561550831  likes=28,660,955
  about fields: Name, About, Category, Likes, Talking about, Image, Profile URL
NASA - National Aeronautics and Space Administration
  NASA - National Aeronautics and Space Administration. 28,660,954
Python  891,000 members  (public)
wrote facebook_groups.csv
Enter fullscreen mode Exit fullscreen mode

TL;DR

  • Page lookups return 8 rows: one with type == "page" carrying the record, seven type == "about" field pairs.
  • Fields are at the top level. There is no data wrapper, so r.json()["name"] is correct.
  • Group privacy is the gate. Public groups return member counts, private ones return 3,800 members and a private flag and nothing else.
  • members_count and privacy populate intermittently. Retry instead of writing a null row.

Why is it hard to scrape Facebook?

Facebook serves almost everything behind a login, so a logged-out request to most URLs lands on a sign-in redirect rather than content, and the public HTML that does render uses generated class names that rotate often enough to rot a CSS parser within days. What survives reliably without a session is the OpenGraph metadata block, which is why the records below are shaped around page IDs, captions and images. The part that wastes the most time is discovering per-URL which fields are actually populated, because the same endpoint returns a full record for one target and a sparse one for the next.

Prerequisites

To scrape Facebook with this code you need three things.

1. A free Chocodata API key. Sign up and copy it from the dashboard. Free tier, no card.

The Chocodata dashboard showing where to copy a free API key

2. Python 3.9+ and requests. Tested on Python 3.13.7 with requests 2.34.2 in July 2026.

pip install requests
Enter fullscreen mode Exit fullscreen mode

3. A public target. A page name, or the URL of a public post or public group.

Look up a public page

To scrape a Facebook page, send the page name to the page endpoint and pull the record out of the results list.

1. Send the lookup and check the status 

import requests

BASE = "https://api.chocodata.com/api/v1"
API_KEY = "YOUR_API_KEY"

r = requests.get(f"{BASE}/facebook/page",
                 params={"page": "NASA", "api_key": API_KEY}, timeout=30)
r.raise_for_status()
d = r.json()
print(r.status_code, "|", d["results_count"], "of", d["total_results"], "results")
Enter fullscreen mode Exit fullscreen mode

Output:

200 | 8 of 8 results
Enter fullscreen mode Exit fullscreen mode

Terminal showing the page lookup returning eight results

2. Filter the results by type

The rows are not homogeneous. Exactly one carries type == "page" with the full record, and the rest are about pairs. Filtering is mandatory, not tidiness.

pages = [x for x in d["results"] if x["type"] == "page"]
about = [x for x in d["results"] if x["type"] == "about"]
print(len(pages), "page rows |", len(about), "about rows")

p = pages[0]
print(p["page_id"], "|", p["name"])
print(f"{p['likes']:,} likes | {p['talking_about_count']:,} talking about")
Enter fullscreen mode Exit fullscreen mode

Output:

1 page rows | 7 about rows
100044561550831 | NASA - National Aeronautics and Space Administration
28,660,953 likes | 126,699 talking about
Enter fullscreen mode Exit fullscreen mode

Terminal showing the page row separated from the about rows

3. Collect the about field-value pairs

The about rows are a flat key-value view of the page, so a dict comprehension turns them into something storable.

fields = {x["field"]: x["value"] for x in about}
for k, v in fields.items():
    print(f"{k:<14} {str(v)[:48]}")
Enter fullscreen mode Exit fullscreen mode

Output:

Name           NASA - National Aeronautics and Space Administrat
About          Explore the universe and discover our home planet
Category       video.other
Likes          28660953
Talking about  126699
Image          https://scontent-lga3-3.xx.fbcdn.net/v/t39.30808-
Profile URL    https://www.facebook.com/NASA/
Enter fullscreen mode Exit fullscreen mode

Note that Likes arrives as a string here while the page row gives an integer, so prefer the page row for anything numeric.

Terminal printing the about field-value pairs

Pull a public post payload

To scrape a Facebook post, send the public URL to the post endpoint and read the author, caption and image off the top level.

1. Fetch the payload for a URL 

r = requests.get(f"{BASE}/facebook/post",
                 params={"url": "https://www.facebook.com/NASA",
                         "api_key": API_KEY}, timeout=60)
r.raise_for_status()
post = r.json()
print(len(post), "fields")
print(post["author"], "|", post["page_id"])
Enter fullscreen mode Exit fullscreen mode

Output:

16 fields
NASA - National Aeronautics and Space Administration | 100044561550831
Enter fullscreen mode Exit fullscreen mode

Terminal showing the post payload with its author and page ID

2. Keep only the fields that are populated

Of the 16 fields, ten came back with values on this URL. Filtering nulls once, up front, is cheaper than guarding every read downstream.

filled = {k: v for k, v in post.items() if v is not None}
print(sorted(filled))
Enter fullscreen mode Exit fullscreen mode

Output:

['author', 'caption', 'data_source', 'id', 'image', 'page_id', 'source',
 'thumbnail', 'title', 'url']
Enter fullscreen mode Exit fullscreen mode

data_source reports opengraph, which tells you the record came from the public metadata block rather than a rendered page.

Terminal listing the populated fields on the post record

3. Guard the counts that come back null

reactions_count, comments_count and shares_count are absent on many public URLs, so coalesce them before any arithmetic.

reactions = post.get("reactions_count") or 0
comments = post.get("comments_count") or 0
print("engagement:", reactions + comments)
print("caption:", post["caption"][:64])
Enter fullscreen mode Exit fullscreen mode

Output:

engagement: 0
caption: NASA - National Aeronautics and Space Administration. 28,660,953
Enter fullscreen mode Exit fullscreen mode

Terminal showing null engagement counts coalesced to zero

Read public group metadata

To scrape a Facebook group, send the group URL and read privacy before you read anything else.

1. Request the group metadata 

GROUP = "https://www.facebook.com/groups/python"

r = requests.get(f"{BASE}/facebook/group",
                 params={"url": GROUP, "api_key": API_KEY}, timeout=60)
r.raise_for_status()
g = r.json()
print(g["name"], "|", g["privacy"], "|", g["members_count"])
Enter fullscreen mode Exit fullscreen mode

Output:

Python | public | 891000
Enter fullscreen mode Exit fullscreen mode

Terminal showing the group request returning name, privacy and member count

2. Reject anything not marked public

The privacy flag is the gate, but it arrives empty on some calls for a group that is public on the next one. Empty is a retry condition, and only a populated value that is not public is a stop condition. Collapsing the two is how public groups end up silently skipped.

import time

for _ in range(3):
    if g.get("privacy") and g.get("members_count"):
        break
    time.sleep(2)
    r = requests.get(f"{BASE}/facebook/group",
                     params={"url": GROUP, "api_key": API_KEY}, timeout=60)
    r.raise_for_status()
    g = r.json()

if g["privacy"] != "public":
    raise SystemExit(f"{g['name']} is {g['privacy']}, stopping")

row = {"id": g["id"], "name": g["name"], "url": g["url"],
       "members": g["members_count"], "privacy": g["privacy"],
       "description": (g.get("description") or "")[:80]}
print(row)
Enter fullscreen mode Exit fullscreen mode

Output:

{'id': 'python', 'name': 'Python', 'url': 'https://www.facebook.com/groups/python/',
 'members': 891000, 'privacy': 'public',
 'description': 'This is a group for Python developers.'}
Enter fullscreen mode Exit fullscreen mode

Terminal showing the public check passing and the assembled row

3. Write the rows to CSV 

import csv

with open("facebook_groups.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=list(row.keys()))
    w.writeheader()
    w.writerow(row)
Enter fullscreen mode Exit fullscreen mode

Output:

id,name,url,members,privacy,description
python,Python,https://www.facebook.com/groups/python/,891000,public,This is a group for Python developers.
Enter fullscreen mode Exit fullscreen mode

Pass encoding="utf-8" and newline="" here or non-ASCII group names arrive mangled and every row gains a blank line on Windows.

Terminal showing the CSV written with the group row

The part that breaks

Four failures, all hit while writing this, none of them in the network layer.

Indexing the results list without filtering. The about rows have no page_id key at all, so a comprehension over every row dies on row two:

Traceback (most recent call last):
  File "scrape_facebook.py", line 18, in <module>
    ids = [x["page_id"] for x in d["results"]]
           ~^^^^^^^^^^^
KeyError: 'page_id'
Enter fullscreen mode Exit fullscreen mode

Casting a null count. reactions_count is None on most public URLs, and int() refuses it:

TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
Enter fullscreen mode Exit fullscreen mode

Formatting a null member count. Some groups return members_count: None, and an f-string comma format raises rather than printing empty:

TypeError: unsupported format string passed to NoneType.__format__
Enter fullscreen mode Exit fullscreen mode

The intermittent one, which is the real trap. The same group URL does not return the same field coverage every time. Three consecutive calls to the same three groups gave this:

gamedev          None/None      | None/None      | 108000/public
python           891000/public  | 891000/public  | None/None
webdevelopers    28/public      | 28/public      | None/None
Enter fullscreen mode Exit fullscreen mode

Writing whichever response arrives first gives you a table half full of nulls. The fix is a small retry that only accepts a populated record, which is what get_group does in the script below.

Full script 

"""Facebook public-data scraper: pages, posts, groups.
Tested: Python 3.13.7, requests 2.34.2, 27 July 2026."""
import csv
import sys
import time

import requests

sys.stdout.reconfigure(encoding="utf-8")

BASE = "https://api.chocodata.com/api/v1"
API_KEY = "YOUR_API_KEY"


def call(job, timeout=60, tries=3, **params):
    """One GET against the API. Fields come back at the top level, no wrapper.

    Retried, so a batch run survives a dropped response.
    """
    params["api_key"] = API_KEY
    for attempt in range(tries):
        r = requests.get(f"{BASE}/{job}", params=params, timeout=timeout)
        if r.ok:
            return r.json()
        time.sleep(2)
    r.raise_for_status()


def find_page(name):
    """Public page lookup. Returns the page row plus the about pairs."""
    data = call("facebook/page", page=name)
    pages = [x for x in data["results"] if x["type"] == "page"]
    if not pages:
        raise LookupError(f"no page row for {name!r}")
    p = pages[0]
    return {
        "page_id": p["page_id"],
        "name": p["name"],
        "url": p["url"],
        "likes": p.get("likes"),
        "followers": p.get("followers"),
        "talking_about": p.get("talking_about_count"),
        "about": {x["field"]: x["value"] for x in data["results"]
                  if x["type"] == "about"},
    }


def get_post(url):
    """Public post payload. Counts are often absent, so never assume an int."""
    p = call("facebook/post", url=url)
    return {
        "url": p["url"],
        "page_id": p["page_id"],
        "author": p["author"],
        "caption": p["caption"],
        "image": p["image"],
        "reactions": p.get("reactions_count") or 0,
        "comments": p.get("comments_count") or 0,
    }


def get_group(url, tries=4, pause=1.5):
    """Group metadata. members_count and privacy populate intermittently,
    so retry until they arrive rather than writing a null row."""
    for attempt in range(tries):
        g = call("facebook/group", url=url)
        if g.get("privacy") and g.get("members_count"):
            return {"id": g["id"], "name": g["name"], "url": g["url"],
                    "members": g["members_count"], "privacy": g["privacy"],
                    "description": (g.get("description") or "")[:80]}
        if attempt < tries - 1:
            time.sleep(pause)
    raise RuntimeError(f"group metadata never populated for {url}")


def to_csv(rows, path):
    with open(path, "w", newline="", encoding="utf-8") as f:
        w = csv.DictWriter(f, fieldnames=list(rows[0].keys()))
        w.writeheader()
        w.writerows(rows)
    return path


if __name__ == "__main__":
    page = find_page("NASA")
    print(f"{page['name']}  id={page['page_id']}  likes={page['likes']:,}")
    print(f"  about fields: {', '.join(page['about'])}")

    post = get_post("https://www.facebook.com/NASA")
    print(f"{post['author']}\n  {post['caption'][:64]}")

    group = get_group("https://www.facebook.com/groups/python")
    if group["privacy"] != "public":
        raise SystemExit("not a public group, stopping")
    print(f"{group['name']}  {group['members']:,} members  ({group['privacy']})")

    to_csv([group], "facebook_groups.csv")
    print("wrote facebook_groups.csv")
Enter fullscreen mode Exit fullscreen mode

The retry inside get_group is the only non-obvious part, and it is the difference between a clean table and one where a third of the rows are empty.

Summary

Three endpoints cover the public Facebook surface: a page lookup that returns one record row plus seven about pairs, a post request that turns any public URL into an author, caption, image and numeric page ID, and a group request whose privacy flag decides whether you continue at all. The thing to carry away is that every failure here is a data-shape failure rather than a blocking failure, so filter the results list by type, coalesce the counts, and retry group metadata until it is populated. Python is used throughout, though any language that can send a GET works identically.

FAQ

Is it legal to scrape Facebook?

Public pages and public groups sit in a different position from private profiles and logged-in content, so stay on public URLs and check Facebook's terms for your specific use.

Can you scrape a private Facebook group?

No. A private group returns its name and a private privacy flag with no member list, and that flag is the signal to stop rather than a problem to route around.

Why does the same group URL return different fields on different calls?

Public group metadata is not always exposed on every request, so retry until privacy and members_count are populated instead of storing the first response.

Top comments (0)