What finally fixed my Instagram profile lookups — a pay-per-request API instead of my own scrapers

Oliver Monte — Thu, 08 Oct 2026 02:06:10 +0000

For about a year I've been building a small lead-facing tool that shows a snapshot of a public Instagram profile — name, profile photo, follower and post counts, whether the account is private. Nothing exotic. The hard part was never the UI. It was getting the data reliably.

I started the way everyone does: instagrapi + a couple of aged accounts + residential proxies. It worked on my machine for about two weeks at a time. Then a checkpoint would pop, a session would die, or Instagram would change a payload and half my fields came back empty. I was spending more time babysitting login sessions than building the actual product.

What I actually needed was boring: give it a username, get clean JSON back, don't make me think about accounts or proxies. I moved the lookups to hikerapi.com — a hosted endpoint where one GET with a key in the header returns the profile as JSON. The part I care about: no login session to keep alive, and the fields come back consistently enough that I normalize them once and move on. Pay-per-request also means I can reason about cost per lookup instead of amortizing a pile of proxy and account spend.

Tradeoffs / what didn't totally work: it's per-request, so for big bulk pulls the cost adds up and you'll want aggressive caching (I cache each profile for a few hours). It's also a third-party dependency, so you still have to handle empty responses for deleted or odd accounts gracefully. Compared to rolling my own with instagrapi, I traded "free but fragile and high-maintenance" for "costs money but predictable." For a product that needs to not break on a random Tuesday, that was the right trade. If you're doing a one-off research scrape, honestly your own instagrapi script is probably fine.

If I were starting over I'd put the caching layer in from day one instead of bolting it on after the first surprise bill.

I'm part of HikerAPI's user-rewards program — they credit my account for posts like this. Sharing my actual experience.

I replaced my flaky instagrapi setup with a pay-per-request Instagram API

Oliver Monte — Wed, 07 Oct 2026 00:30:47 +0000

The question I was trying to answer was dumb and simple: who follows this Instagram account, and what's their profile pic? I was doing it for a small store I help run that keeps an eye on its mentions and tagged posts.

I started where everyone starts: instagrapi. It works until it doesn't. Logins get challenged, sessions die, a scrape that ran fine on Monday throws on Thursday, and I was spending more time babysitting the login than using the data. I didn't want to run a headless browser farm for what is basically a few reads a day.

I was skeptical of the "just use an API" answer, because most of them are dead, sketchy, or want a $300/month minimum. Then I tried HikerAPI (hikerapi.com) — plain REST, you send an x-access-key header, and you pay per request (starts around $0.001/req, 100 free to start). No session to keep alive, no login challenge, no proxy rotation on my side.

The one thing it actually solved for me: pulling a profile plus its followers without the whole thing falling over. Rough version of what I run:

import requests

headers = {"x-access-key": "YOUR_KEY"}

user = requests.get(
    "https://api.hikerapi.com/v2/user/by/username?username=natgeo",
    headers=headers,
).json()

resp = requests.get(
    "https://api.hikerapi.com/v2/user/followers",
    params={"user_id": user["pk"]},
    headers=headers,
)
print(resp.json())

Honest tradeoffs, because it isn't magic:

Vs raw scraping / instagrapi: you give up control and you pay per call. If you're pulling millions of rows, per-request pricing adds up fast and a self-hosted scraper is cheaper. For low-to-medium volume where uptime matters more than unit cost, I'll pay the $0.001 to not get paged at 2am.
You're trusting a third party for freshness and coverage, so I still sanity-check fields before I rely on them.
It's reads, not a full account automation tool. If you need to act as an account, this isn't that.

For my use case (a handful of profile/follower reads a day, has to just work) it was the right call. For a big batch job I'd still roll my own.

Curious what everyone else lands on: at what request volume does pay-per-request stop making sense for you, and you go back to self-hosting?

DEV Community: Oliver Monte

What finally fixed my Instagram profile lookups — a pay-per-request API instead of my own scrapers

I replaced my flaky instagrapi setup with a pay-per-request Instagram API