DEV Community

Coco
Coco

Posted on • Edited on

How to scrape Instagram in 2026?

#python #webscraping #tutorial #api

Scraping Instagram public profile analytics in Python: follower count, following count, post count, bio, display name and verification status for business and creator accounts, using the Chocodata profile endpoint. Every request below was executed on 27 July 2026 and the screenshots are that output.

Here is what the finished script prints:

canva     2,578,854 followers   2,819 posts  verified=True
figma       938,129 followers   1,040 posts  verified=True

wrote instagram_profiles.csv (2 rows, captured=2026-07-27)
Enter fullscreen mode Exit fullscreen mode

TL;DR

  • One GET returns 18 top-level fields for a public profile. No wrapper object, so r.json()["follower_count"] is the whole access path.
  • Four of the 18 are None on public profiles: external_url, category, is_business, posts. Use .get() or you will ship a NoneType traceback.
  • og_description disagrees with the numeric fields. For canva it says 1,450 Following while following_count is 1405. Trust the integers.
  • follower_count moves between calls. Canva returned 2,578,851 and 2,578,854 minutes apart, so assert on a range, never on equality.
  • Tested on Python 3.13.7 and requests 2.34.2, July 2026.

Why is it hard to scrape Instagram?

Scraping Instagram is hard because the public profile page hydrates its counts from an embedded JSON payload whose key names are hashed and rotate between deploys, so selector-based parsers return zero rows instead of raising. On top of that a logged-out client gets a small number of views before the page is replaced by a sign-in interstitial that still looks like a successful response. The layer that wastes the most time is IP reputation, because the same code that works from a home connection hits a checkpoint challenge from any cloud range.

Prerequisites

To scrape Instagram profile data with the code below you need three things, and the first is free.

  1. A free Chocodata API key. Sign up, copy the key from the dashboard, and pass it as api_key on every request. Free to start, no card required.

Copying the free API key from the dashboard

  1. Python 3.9 or newer and requests. No browser driver, no proxy, no Instagram account. 
python -m venv .venv
pip install requests
Enter fullscreen mode Exit fullscreen mode
  1. The handles you want to pull, for example canva and figma.

Tested on Python 3.13.7 with requests 2.34.2 in July 2026.

How to scrape Instagram profiles?

One GET to the profile endpoint returns the full public profile as JSON, and the three steps below take it from a URL to a CSV row.

1. Build the request URL

To scrape an Instagram profile, the endpoint is instagram/profile and it takes two query parameters: the handle as username and your key as api_key.

import requests

BASE = "https://api.chocodata.com/api/v1"
API_KEY = "YOUR_API_KEY"

url = f"{BASE}/instagram/profile"
params = {"username": "canva", "api_key": API_KEY}

r = requests.get(url, params=params, timeout=60)
print(r.status_code, r.headers.get("content-type"))
Enter fullscreen mode Exit fullscreen mode 
200 application/json
Enter fullscreen mode Exit fullscreen mode

The profile request returning 200 and a JSON content type

2. Fetch the profile JSON and check a field, not the status

The Instagram profile payload puts all 18 fields at the top level, so guard on a field you expect rather than on the status code, because a logged-out interstitial upstream can still produce a well-formed response.

r.raise_for_status()
profile = r.json()

if "username" not in profile:
    raise RuntimeError("no profile in response - check the handle")

print(len(profile), "fields")
for k in ["username", "full_name", "follower_count", "following_count",
          "posts_count", "is_verified", "is_private", "biography"]:
    print(f"{k:16} {profile[k]}")
Enter fullscreen mode Exit fullscreen mode 
18 fields
username         canva
full_name        Canva
follower_count   2578854
following_count  1405
posts_count      2819
is_verified      True
is_private       False
biography        What will you design today?
Enter fullscreen mode Exit fullscreen mode

All 18 fields printed from the profile response

The remaining populated fields are id, url, profile_pic_url, og_description, source and data_source.

3. Flatten the profile into a CSV row

Flattening the profile into one dated row is what turns a lookup into a dataset, so the capture date goes in before anything else.

import csv
from datetime import date

FIELDS = ["captured", "username", "full_name", "follower_count",
          "following_count", "posts_count", "is_verified", "url"]

def to_row(p):
    row = {k: p.get(k) for k in FIELDS[1:]}
    row["captured"] = date.today().isoformat()
    return row

with open("instagram_profiles.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=FIELDS)
    w.writeheader()
    w.writerow(to_row(profile))
Enter fullscreen mode Exit fullscreen mode 
captured,username,full_name,follower_count,following_count,posts_count,is_verified,url
2026-07-27,canva,Canva,2578854,1405,2819,True,https://www.instagram.com/canva/
Enter fullscreen mode Exit fullscreen mode

The written CSV opened showing the header and the dated row

Note the .get() in to_row. That is not defensive style for its own sake, and the reason is in the failures section below. The same request works from any language with an HTTP client, so Node, Go and PHP need nothing beyond their standard library.

Track a competitor set over time

Tracking an Instagram competitor set is the same profile call in a loop, with the capture date doing the real work. Absolute follower counts tell you who is bigger. The diff between two captures tells you who is growing.

import time

HANDLES = ["canva", "figma"]

def fetch(handle, tries=3):
    """One handle, retried, so a batch run survives a dropped response."""
    for attempt in range(tries):
        r = requests.get(f"{BASE}/instagram/profile",
                         params={"username": handle, "api_key": API_KEY},
                         timeout=60)
        if r.ok:
            return r.json()
        time.sleep(2)
    r.raise_for_status()

rows = []
for h in HANDLES:
    p = fetch(h)
    rows.append(to_row(p))
    print(f"{p['username']:8} {p['follower_count']:>10,} followers  "
          f"{p['posts_count']:>6,} posts  verified={p['is_verified']}")
    time.sleep(2)
Enter fullscreen mode Exit fullscreen mode 
canva     2,578,854 followers   2,819 posts  verified=True
figma       938,129 followers   1,040 posts  verified=True
Enter fullscreen mode Exit fullscreen mode

Append rather than overwrite, run it on a schedule, and compute the delta at read time:

import collections

history = collections.defaultdict(list)
with open("instagram_profiles.csv", encoding="utf-8") as f:
    for row in csv.DictReader(f):
        history[row["username"]].append((row["captured"], int(row["follower_count"])))

for handle, points in history.items():
    points.sort()
    if len(points) > 1:
        delta = points[-1][1] - points[0][1]
        print(f"{handle:8} {points[0][0]} -> {points[-1][0]}  {delta:+,}")
Enter fullscreen mode Exit fullscreen mode

One capture gives you nothing from that loop, which is the point. Take the first snapshot before you have a reason to.

The part that breaks

Four failures, all hit while writing this.

Four fields are None on public profiles. external_url, category, is_business and posts come back empty, and bracket access stores the None silently until something calls a method on it.

print({k: profile[k] for k in ["external_url", "category", "is_business", "posts"]})

profile["category"].lower()
Enter fullscreen mode Exit fullscreen mode 
{'external_url': None, 'category': None, 'is_business': None, 'posts': None}
Traceback (most recent call last):
  File "instagram_profiles.py", line 24, in <module>
    profile["category"].lower()
AttributeError: 'NoneType' object has no attribute 'lower'
Enter fullscreen mode Exit fullscreen mode

Build the schema around the fourteen populated fields and read the rest with .get().

There is no wrapper object. The fields are at the top level, so reaching for one costs you a lookup that never resolves.

profile["data"]
Enter fullscreen mode Exit fullscreen mode 
Traceback (most recent call last):
  File "instagram_profiles.py", line 31, in <module>
    profile["data"]
KeyError: 'data'
Enter fullscreen mode Exit fullscreen mode

og_description disagrees with the numeric fields. It is a display string, rounded and formatted, and it does not match the integers beside it.

print(profile["following_count"])   # 1405
print(profile["og_description"])
# 3M Followers, 1,450 Following, 2,819 Posts - See Instagram photos and videos from Canva (@canva)
Enter fullscreen mode Exit fullscreen mode

Figma shows the same split: following_count is 87 while its og_description reads 93 Following. Parse the integers, never the sentence.

Counts move between calls. Canva returned 2578851 and then 2578854 within the same session, and Figma moved by one in the other direction.

assert profile["follower_count"] == 2578851        # fails on the next run
Enter fullscreen mode Exit fullscreen mode 
Traceback (most recent call last):
  File "instagram_profiles.py", line 44, in <module>
    assert profile["follower_count"] == 2578851
AssertionError
Enter fullscreen mode Exit fullscreen mode 
assert profile["follower_count"] > 1_000_000       # what you actually want
Enter fullscreen mode Exit fullscreen mode

Any regression test that pins an exact follower count goes red on its second run.

Full script 

"""Instagram public profile analytics via the Chocodata profile endpoint.
Tested: Python 3.13.7, requests 2.34.2, 27 July 2026.
Public business and creator profiles only.
"""
import collections
import csv
import time
from datetime import date

import requests

BASE = "https://api.chocodata.com/api/v1"
API_KEY = "YOUR_API_KEY"
CSV_PATH = "instagram_profiles.csv"

FIELDS = ["captured", "username", "full_name", "follower_count",
          "following_count", "posts_count", "is_verified", "url"]


def fetch_profile(handle, timeout=60):
    """Return the profile dict for one public handle."""
    r = requests.get(f"{BASE}/instagram/profile",
                     params={"username": handle, "api_key": API_KEY},
                     timeout=timeout)
    r.raise_for_status()
    profile = r.json()
    if "username" not in profile:
        raise RuntimeError(f"no profile returned for {handle!r}")
    if profile.get("is_private"):
        raise RuntimeError(f"{handle!r} is private - counts are withheld")
    return profile


def to_row(profile):
    """Flatten a profile into one dated CSV row. .get() because four
    fields come back empty on public profiles."""
    row = {k: profile.get(k) for k in FIELDS[1:]}
    row["captured"] = date.today().isoformat()
    return row


def capture(handles, path=CSV_PATH, delay=2.0):
    """Append one dated row per handle. Safe to run on a schedule."""
    try:
        with open(path, encoding="utf-8") as f:
            existing = bool(f.readline())
    except FileNotFoundError:
        existing = False

    rows = []
    for i, handle in enumerate(handles):
        profile = fetch_profile(handle)
        rows.append(to_row(profile))
        print(f"{profile['username']:8} {profile['follower_count']:>10,} followers  "
              f"{profile['posts_count']:>6,} posts  verified={profile['is_verified']}")
        if i < len(handles) - 1:
            time.sleep(delay)

    with open(path, "a", newline="", encoding="utf-8") as f:
        w = csv.DictWriter(f, fieldnames=FIELDS)
        if not existing:
            w.writeheader()
        w.writerows(rows)

    print(f"\nwrote {path} ({len(rows)} rows, captured={rows[0]['captured']})")
    return rows


def deltas(path=CSV_PATH):
    """Follower change per handle between the first and latest capture."""
    history = collections.defaultdict(list)
    with open(path, encoding="utf-8") as f:
        for row in csv.DictReader(f):
            history[row["username"]].append(
                (row["captured"], int(row["follower_count"])))

    out = {}
    for handle, points in history.items():
        points.sort()
        if len(points) > 1:
            out[handle] = points[-1][1] - points[0][1]
            print(f"{handle:8} {points[0][0]} -> {points[-1][0]}  {out[handle]:+,}")
    return out


if __name__ == "__main__":
    capture(["canva", "figma"])
    deltas()
Enter fullscreen mode Exit fullscreen mode

First run writes the header plus two rows and prints nothing from deltas(), because one capture has nothing to compare against. That is expected and it is the whole reason to start capturing early.

Summary

One GET to the profile endpoint returns 18 top-level fields for a public Instagram profile, and the fourteen populated ones cover the analytics people actually chart: display name, bio, follower count, following count, post count and verification status. What does not work is treating any single reading as exact, since follower counts drift between calls, og_description rounds and disagrees with the integers next to it, and four fields are always empty on public profiles. The one thing to carry away is that this is a time-series job, so write the capture date into the row on the first run rather than the tenth.

FAQ

Is scraping Instagram profiles legal?

Public profile pages are not automatically off limits, but Instagram's terms prohibit automated collection, which makes this a contract question rather than a criminal one, and none of this is legal advice.

Can I scrape a private Instagram account?

No, and the script above raises on is_private rather than trying, because a private account's counts are withheld from anyone it has not approved.

Why do my follower counts change between two runs minutes apart?

Because the count is live and moves on large accounts, which is why you compare dated captures instead of asserting on an exact number.

Top comments (0)