DEV Community: Intropix-team

How to scrape Instagram posts and engagement data without login (Python)

Intropix-team — Fri, 24 Jul 2026 13:56:09 +0000

If you have ever tried to pull post data out of Instagram in code, you already know the official route is a dead end unless you own the account. The Graph API hands you your own business profile and basically nothing beyond it. So anyone doing competitor analysis, influencer verification, or plain content research ends up scraping instead, and then runs straight into the next wall: anonymous scraping of Instagram stopped working years ago.

A client wanted posts and reels from a list of brand accounts, with engagement numbers attached, running on a daily schedule, and able to backfill a historical date range on demand. What I landed on runs without an Instagram login, cookies, or a headless browser, and the core of it is maybe 15 lines of Python.

Why not instaloader and a burner account?

You can, until the account dies. The open-source libraries (instaloader, instagrapi) are genuinely good, but a burner running against them starts collecting checkpoint challenges and shadow-bans within days, and at that point you are running an account-warming operation, not an analytics one. Feeds are a bit friendlier to scrape than stories, but the moment you want reliable engagement counts and deep pagination, Instagram wants to see a logged-in session again.

So I let someone else carry the session problem. I use a hosted scraper on Apify (the Instagram Posts & Reels Scraper actor): usernames in, JSON out. Full disclosure, it's mine, so factor that in. Any hosted scraper with an API gets you the same shape of result; the exact keys below are what this one returns.

What you need

A free Apify account (free tier is enough to test: the actor caps free runs at 10 posts per run, around 20 per day)
Your Apify API token, from Console > Settings > API & Integrations
Python 3.9+

Step 1: install the client

pip install apify-client

Step 2: run the scraper

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("intropix/instagram-posts-reels-scraper").call(
    run_input={
        "usernames": ["natgeo"],
        "maxPosts": 2,
    }
)

items = list(client.dataset(run["defaultDatasetId"]).iterate_items())
print(f"{run['status']}: {len(items)} posts")
for post in items:
    print(post["post_type"], post["like_count"], post["permalink"])

call() blocks until the run finishes. My last test run took 10 seconds wall clock including container startup.

What the output looks like

One dataset item per post, always the same 24 fields, with every media slide nested inside it. This is a real reel from the run above (URLs truncated, caption shortened):

{
  "username": "natgeo",
  "user_pk": "787132",
  "full_name": "National Geographic",
  "is_verified": true,
  "is_private": false,
  "follower_count": 280000000,
  "following_count": 148,
  "profile_pic_url": "https://instagram.fxxx.fna.fbcdn.net/v/t51.2885-19/...",
  "biography": "Experience the world through the eyes of National Geographic photographers.",
  "post_pk": "3512345678901234567",
  "shortcode": "DbrNlfeg60B",
  "permalink": "https://www.instagram.com/p/DbrNlfeg60B/",
  "post_type": "reel",
  "taken_at": "2026-08-05T22:29:25Z",
  "caption": "Black rhinos once again roam Matusadona National Park, decades after poachers turned the Zimbabwean refuge into what one conservationist called...",
  "like_count": 16545,
  "comment_count": 77,
  "view_count": 1253265,
  "is_paid_partnership": false,
  "is_off_grid": false,
  "coauthors": [],
  "hashtags": [],
  "mentions": ["marcuswestbergphotography", "warren_smart"],
  "media": [
    {
      "slide_index": 0,
      "media_type": "video",
      "media_url": "https://scontent-lax3-1.cdninstagram.com/o1/v/t2/f2/m86/...",
      "cover_url": "https://scontent-lax3-2.cdninstagram.com/v/t51.82787-15/...",
      "width": 720,
      "height": 1280,
      "video_duration": 13.696000099182129
    }
  ]
}

The fields that make this useful for analysis:

like_count and comment_count on everything; view_count on reels and videos. Instagram publishes no view count for photos or carousels, so those are null rather than a misleading zero. This is the data the official API will not give you for accounts you don't own.
follower_count and following_count: the account's public totals at scrape time, stamped on every item, so a dataset row carries its own denominator for engagement-rate maths.
coauthors and is_paid_partnership: collab posts and Instagram's own sponsorship label, the two signals you want for influencer verification.
media is one entry per slide. A 10-slide carousel is ONE item with 10 nested slides, not 10 items. Your item counts match reality and so does your bill.
post_pk is a stable numeric ID, use it for dedup. permalink gives you the clickable evidence URL.

The schema is a strict allowlist: every item has exactly these 24 keys, every slide the same 7. Upstream changes drop out instead of leaking into your pipeline.

The date filters are the actually good part

sinceDate and untilDate bound the fetch server-side:

run = client.actor("intropix/instagram-posts-reels-scraper").call(
    run_input={
        "usernames": ["natgeo", "bbcearth"],
        "maxPosts": 200,
        "sinceDate": "2026-07-22",
    }
)

That pattern turns a scheduled daily run into an incremental pipeline: yesterday's date in sinceDate, and each run collects exactly the new posts instead of re-downloading (and re-paying for) the whole feed. For one-off historical work, set both bounds and backfill a campaign window post by post.

Two input details worth knowing. maxPosts gives you the N newest posts, which is less obvious than it sounds: Instagram's feed is not date-ordered, pinned posts sit at the top carrying their original dates, and even the tail arrives out of order, so taking the first N the feed hands over quietly returns the wrong posts. The actor reads past the limit, sorts, then truncates. And maxPosts is a total across all usernames, filled account by account in input order, not a per-account allowance. For a fixed number per account, run one account per run (tasks and schedules make that painless) or use date bounds instead.

The reels that aren't on the profile grid

Some reels never show up on the profile grid the feed endpoint reads. Two kinds: reels published only to the reels tab, and trial reels, which Instagram shows to non-followers for a window before the account owner decides whether to keep them. Both are invisible to anything that only walks the grid.

includeOffGridReels (off by default) returns them alongside everything else, and every item carries is_off_grid so you can tell which is which. The flag is on every post, not just when you opt in, so your columns never move.

One thing to know if this is why you're here: trial reels are listed only while the trial is live. When the window closes the reel leaves every public listing without ever joining the grid, so nothing can fetch it afterwards. I watched three of them drop out over about 40 minutes. That makes this a case where a scheduled daily run genuinely beats a backfill, because a backfill cannot recover what is no longer listed.

Can it read age-restricted (18+) accounts?

Yes, same as its stories sibling. Alcohol, vape, OF and nightlife brands often sit behind Instagram's age gate and return nothing to anonymous scrapers. This actor reads their feeds at the standard price and I verified delivery against an age-gated global beer brand's account (carousels and all). Scope boundary: public and age-gated brand content. Private accounts are out.

What does it cost?

Pay-per-event, no subscription: $0.005 per run start, $0.002 per profile scanned, and from $1.90 per 1,000 posts delivered. A carousel counts as one post no matter how many slides. View counts cost nothing extra, and off-grid reels bill as ordinary delivered posts, so turning that flag on spends your maxPosts budget rather than adding a surcharge. Failed lookups (typo'd or deleted usernames) are never charged. A daily incremental pull on two competitor accounts costs about two cents a day; a 1,000-post backfill across 10 accounts lands around $1.93 all-in.

Getting it into a spreadsheet

Flatten to CSV and open it in Sheets or Excel; the engagement columns are what you usually want:

import csv

FIELDS = ["username", "post_pk", "post_type", "taken_at", "permalink",
          "like_count", "comment_count", "view_count",
          "follower_count", "is_off_grid",
          "is_paid_partnership", "hashtags", "mentions"]

with open("posts.csv", "w", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=FIELDS, extrasaction="ignore")
    writer.writeheader()
    for post in items:
        row = {**post,
               "hashtags": " ".join(post["hashtags"]),
               "mentions": " ".join(post["mentions"])}
        writer.writerow(row)

Sort by like_count and you have a poor man's top-content report; group by ISO week of taken_at and you have a posting-cadence benchmark.

No-dependency Node version

The plain REST API does the whole thing in one call:

const res = await fetch(
  "https://api.apify.com/v2/acts/intropix~instagram-posts-reels-scraper" +
    "/run-sync-get-dataset-items",
  {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      // Token in a header, never in the query string: URLs land in
      // shell history, proxy logs and server access logs.
      Authorization: `Bearer ${process.env.APIFY_TOKEN}`,
    },
    body: JSON.stringify({ usernames: ["natgeo"], maxPosts: 2 }),
  }
);
const posts = await res.json();
console.log(posts.length, "posts");

Node 18+ has fetch built in, so that is the entire program.

Running it on a schedule

Apify has a scheduler built in: actor page > Schedule, pick an interval. Pair it with sinceDate for the incremental pattern above, and add a webhook (Integrations tab) if you want each finished run POSTed to your endpoint.

Wrap-up

Complete runnable versions of the examples are in the examples repo: github.com/Intropix-team/instagram-scraper-examples. The actor, with the full 24-field reference and pricing, is at apify.com/intropix/instagram-posts-reels-scraper. If you need the ephemeral side of Instagram instead (stories, which disappear after 24 hours), that is a separate actor with its own write-up: How to scrape Instagram stories without login. The free tier is enough to test whether it fits before any money is involved.

Got a posts or engagement use case that doesn't quite fit the above? Drop it in the comments. The date-range filtering in particular came out of exactly these kinds of questions.

How to scrape Instagram stories without login (Python, JSON in 30 seconds)

Intropix-team — Fri, 24 Jul 2026 13:50:40 +0000

Instagram stories disappear after 24 hours, which makes them the most annoying content type to work with programmatically. Posts sit there forever and give you time to figure things out. Stories punish you for slow pipelines, meaning if your scraper has a bad day, the data is simply gone.

For a client project I needed active stories from a list of brand accounts, as structured data, on a schedule. This post is the setup I ended up with. It uses no Instagram login, no cookies, no browser automation, and the whole thing is about 15 lines of Python.

Why not just log in with a burner account?

Because Instagram is very good at killing burner accounts. The usual open-source route (instaloader, instagrapi and friends) works right up until it doesn't. First a checkpoint challenge, then forced 2FA, then the session quietly gets shadow-banned, and suddenly you are babysitting a stable of warm accounts and residential proxies instead of doing your actual job. Stories make this worse than posts do: the web UI has no anonymous path to them at all, so every story fetch normally rides on a logged-in session.

The alternative is to let someone else own that problem. I use a hosted scraper on Apify (the Instagram Stories Scraper actor). You send usernames, you get JSON back, and session management stops being your concern. Full disclosure: this is my own actor, so treat me as biased. The technique below is the same with any hosted scraper that exposes an API; the specific field names you'll see are from mine.

What you need

A free Apify account (the free tier is enough to test: the actor caps free runs at 10 stories per run, around 20 per day)
Your Apify API token, from Console > Settings > API & Integrations
Python 3.9+

Step 1: install the client

pip install apify-client

Step 2: run the scraper

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("intropix/instagram-stories-scraper").call(
    run_input={
        "usernames": ["natgeo"],
        "maxResults": 2,
    }
)

items = list(client.dataset(run["defaultDatasetId"]).iterate_items())
print(f"{run['status']}: {len(items)} stories")
for story in items:
    print(story["username"], story["media_type"], story["media_url"][:60])

call() starts the actor and blocks until it finishes. On my last test run this took 27 seconds wall clock including container startup; the scrape itself is 2-3 seconds per account.

What the output looks like

One dataset item per story, always the same 22 fields. This is a real item from the run above (media URL truncated):

{
  "username": "natgeo",
  "user_pk": "787132",
  "full_name": "National Geographic",
  "is_verified": true,
  "is_private": false,
  "story_pk": "3942549043641043661",
  "taken_at": "2026-07-16T13:28:11Z",
  "expiring_at": "2026-07-17T13:28:11Z",
  "media_type": "image",
  "media_url": "https://instagram.frix7-1.fna.fbcdn.net/v/t51.82787-15/747613074_...",
  "video_duration": null,
  "is_paid_partnership": false,
  "attribution_url": null,
  "width": 750,
  "height": 1334,
  "caption": null,
  "link_urls": [],
  "hashtags": [],
  "mentions": ["disneyparks", "disneyland"],
  "music_title": null,
  "music_artist": null,
  "coauthors": []
}

A few fields worth calling out:

expiring_at tells you exactly when the story dies, so you know your download deadline.
media_url is a direct CDN link to the full-resolution image or video. These links expire after a while, so if you want the file, download it in the same job.
is_paid_partnership, mentions and link_urls are the interesting ones for influencer-verification and competitor-monitoring use cases: they tell you who a story tags and where it links.
story_pk is a stable numeric ID, use it for dedup.

The schema is fixed. If Instagram adds junk fields upstream, they get dropped rather than leaking into your dataset, so downstream code does not break when Instagram changes something.

Can it read age-restricted (18+) accounts?

Yes, and this was the reason I built it. Alcohol, vape and nightlife brands often mark their accounts age-restricted, and those accounts return nothing to anonymous story viewers and to most scrapers. If you do competitor monitoring in those verticals, or you verify influencer placements for a drinks brand, that gap is exactly where your data needs to be. This scraper reads them like any other public account: I verified delivery against age-gated brand accounts (Budweiser was my test case, 17 story items from an account that an anonymous viewer cannot see at all).

To be clear about scope, this is for public and age-gated brand content. Private accounts stay private and the actor does not touch them.

What does it cost?

Pay-per-event, no subscription. Three events: $0.005 per run start, $0.002 per profile scanned, $0.0025 per story delivered. That lands at "from $2.50 per 1,000 stories", or about $2.70 per 1,000 all-in on typical runs. A daily check on two competitor accounts costs about $0.014 per run. Failed lookups (typo'd or deleted usernames) are never charged.

If you run at serious volume across 100+ accounts, do the math against per-username-priced scrapers, which get cheaper at scale; this one wins on small and mid-size runs.

Running it on a schedule

You could cron the Python script, but Apify has a scheduler built in, so I let the platform do it: actor page > Schedule, pick an interval, done. Add a webhook (Integrations tab) if you want each finished run POSTed to your endpoint.

For no-code pipelines there is an official Apify node for n8n; I published ready-made n8n templates for story archiving to Google Drive, competitor tracking to Sheets with an AI daily brief, and influencer placement verification to Notion. Search "Instagram stories" in the n8n template gallery, or check the examples repo below.

No-dependency Node version

If you would rather not install a client library, the plain REST API does the same job. This endpoint starts a run, waits, and returns the dataset items in one call:

const res = await fetch(
  "https://api.apify.com/v2/acts/intropix~instagram-stories-scraper" +
    "/run-sync-get-dataset-items",
  {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      // Send the token as a header, not in the query string. URLs land in
      // shell history, proxy logs and server access logs.
      Authorization: `Bearer ${process.env.APIFY_TOKEN}`,
    },
    body: JSON.stringify({ usernames: ["natgeo"], maxResults: 2 }),
  }
);
const stories = await res.json();
console.log(stories.length, "stories");

Node 18+ has fetch built in, so that is the entire program.

Getting it into a spreadsheet

The lazy path that works everywhere: flatten to CSV, open in Google Sheets or Excel.

import csv

FIELDS = ["username", "story_pk", "taken_at", "expiring_at", "media_type",
          "caption", "link_urls", "mentions", "is_paid_partnership", "media_url"]

with open("stories.csv", "w", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=FIELDS, extrasaction="ignore")
    writer.writeheader()
    for story in items:
        row = {**story,
               "link_urls": " ".join(story["link_urls"]),
               "mentions": " ".join(story["mentions"])}
        writer.writerow(row)

For a fully automated Sheets pipeline (append rows on every scheduled run), use the n8n template instead of reinventing it.

Wrap-up

Complete runnable versions of all three examples (Python, Node, CSV export) are in the examples repo: github.com/Intropix-team/instagram-scraper-examples. The actor itself, including full field reference and pricing, is at apify.com/intropix/instagram-stories-scraper. The free tier is enough to test whether it fits your use case before any money is involved.

If you need the permanent side of Instagram instead (posts and reels, with like, comment and play counts, follower counts, date-range filtering, and reels that never appear on the profile grid), that is a separate actor, the Instagram Posts & Reels Scraper, and I wrote up that one the same way: How to scrape Instagram posts and engagement data without login.

If you have a stories use case that is close but not quite covered here, leave a comment. I read them, and edge cases are how the actor got most of its fields.