A research script for Reels topics hit rate limits, then a challenge. What failed, what a hosted API changed, and what it still will not do.
I used a real login to pull public Reels from accounts in the niches I was considering, so I could see which topics were getting views before I planned a batch of videos. A throwaway felt unnecessary for "just reading public posts." Three evenings later the session was stuck on a challenge, and the spreadsheet I was building the content calendar from was half empty.
The job was narrow. I did not need follower graphs or DMs. I needed a repeatable read of recent Reels: caption, taken-at, play count, likes, comments. Enough to compare topics across a short list of public accounts before committing a week of filming.
What the unofficial client actually bought me
instagrapi against a logged-in session worked for a while. The first account returned clips. The second did too. I added a sleep because the docs and every forum thread say to slow down.
That sleep was the wrong fix. Instagram was not only counting requests per minute. The session had a device fingerprint, a request shape, and almost no normal app behavior around it. Lookup, clips, lookup, clips, from a datacenter IP, on a young session. After a stretch of HTTP 429s I got feedback_required, then challenge_required. Cooling the session off for a day did not bring it back in a usable state.
Rotating a proxy helped for an afternoon. It did not help the next morning. I was now maintaining account warm-up, proxy reputation, and a retry policy so I could decide whether "packing orders" was outperforming "desk setup" in a niche I do not even post in yet. That is a bad trade for a research script.
Why I did not want a hosted API
I assumed a REST wrapper would be the same scraper with an invoice. Opaque, laggy, and one Instagram change away from returning empty lists. The pricing I kept seeing, prepaid and per request, also looked easy to blow through if every pagination page counts.
What changed my mind was the scope. I was not building a product on top of Instagram. I was sampling public Reels to rank topics. A hosted client that already rotates sessions is a reasonable rental for that, provided the responses are synchronous JSON and I can stop when the sample is big enough.
I landed on HikerAPI. It is a hosted REST API for public Instagram data, pay per request from $0.001, with 100 requests free on signup and no card required for that trial. Default throughput is 15 requests per second. They bill successful responses, including some 4xx results, and do not bill 5xx. That last part matters when you are debugging.
The request I actually ship
Auth is a header. No session file, no device id.
import requests
headers = {"x-access-key": "YOUR_KEY"}
user = requests.get(
"https://api.hikerapi.com/v2/user/by/username?username=apple",
headers=headers,
).json()
r = requests.get(
f"https://api.hikerapi.com/v2/user/followers?user_id={user['pk']}",
headers=headers,
)
print(r.json())
v2 returns the raw payload, so field names track Instagram's rather than a cleaned schema. v1 is the validated shape and is what I use for the research job. Followers are a separate call, and I almost never need them for topic picking. A full follower walk is where this gets expensive: pages are short, each page is a request, and verified accounts often will not paginate followers at all.
For the calendar, I only pull clips.
import requests
headers = {"x-access-key": "YOUR_KEY"}
base = "https://api.hikerapi.com"
def get(path, **params):
resp = requests.get(f"{base}{path}", params=params, headers=headers, timeout=30)
resp.raise_for_status()
return resp.json()
def recent_clips(username, pages=2):
user = get("/v1/user/by/username", username=username)
clips, max_id = [], None
for _ in range(pages):
params = {"user_id": user["pk"]}
if max_id:
params["max_id"] = max_id
page = get("/v1/user/clips", **params)
clips.extend(page.get("items") or [])
max_id = page.get("next_max_id")
if not max_id:
break
return clips
def topic_row(clip):
caption = ((clip.get("caption") or {}).get("text") or "").split("\n")[0][:80]
return {
"caption": caption,
"plays": clip.get("play_count") or 0,
"likes": clip.get("like_count") or 0,
"comments": clip.get("comment_count") or 0,
}
for clip in recent_clips("apple"):
print(topic_row(clip))
Two pages per account is usually enough to see which hooks are ahead of that account's own baseline. I group captions by a hand-maintained topic list, then compare median plays inside the account, not across accounts. A large account's quiet Reel still beats a small account's hit, and cross-account totals were the other mistake I made early on.
What I gave up
This is public data only. Private accounts, and anything behind a login wall, are out. Play counts are whatever Instagram exposes on that response. They are not an analytics export, and they move.
You are not on the official Instagram Graph API. Meta's terms still forbid automated collection. A vendor sits between you and that, which is operationally simpler and not the same as a blessed integration. If the use case is an app that needs owner-authorized insights on your own professional account, the Graph API is the correct tool and this is the wrong one.
Cost scales with calls, not with how interesting the result was. Profile lookup plus two clip pages, across a dozen accounts, fits in the free hundred while you test the shape. Pulling comments or followers for the same set does not. I cache username to pk, and I do not re-pull an account I sampled this week.
Endpoints move. v1 is the stable surface I code against. Some older v2 paths have already been retired with a 410, which at least is not billed. Pin the paths you depend on and treat a sudden empty list as a schema change, not as "this topic died."
The script is boring now. That was the point. The account lockout did not teach me which Reels to make. Stopping the session maintenance did.
Top comments (0)