Kacper

Posted on May 24

Building an Instagram Analytics Tool with HikerAPI (Without Fighting Instagram Blocks)

#webscraping #python #api

I recently built a small Instagram monitoring tool for osint. The goal was simple:

Look up Instagram profiles
Pull recent posts
Track follower growth over time
Avoid maintaining fragile scraping infrastructure

I started with browser scraping and later experimented with instagrapi. Both worked… until they didn’t.

Instagram is aggressive about rate limits, session invalidation, checkpoints, and IP reputation. After enough maintenance headaches, I switched to HikerAPI — a REST API for Instagram data that abstracts away most of the anti-bot pain.

This post shows how I used it, where it helped, and the tradeoffs compared to rolling your own scraper.

Why I Stopped Using Traditional Scraping

My first version used:

Selenium
rotating proxies
cookie sessions
retry logic
random delays

It technically worked, but reliability became the real cost.

Typical failures looked like:

login checkpoints
“Please wait a few minutes”
accounts getting flagged
inconsistent HTML
GraphQL endpoints changing

I then tried instagrapi, which is honestly a solid library for many use cases. But it still depends on maintaining authenticated Instagram sessions, and eventually I was spending more time stabilizing automation than building features.

That’s when I started looking for a hosted API layer.

What HikerAPI Actually Gives You

HikerAPI exposes Instagram data through a REST API.

You authenticate with an x-access-key header and make normal HTTP requests. According to their docs, pricing starts around $0.001/request with 100 free requests included for testing. (HikerAPI)

The nice part is that it feels like a normal backend API instead of a scraping pipeline.

Quick Start

Install requests:

pip install requests

Then run this:

import requests

headers = {"x-access-key": "YOUR_KEY"}

r = requests.get(
    "https://api.hikerapi.com/v2/user/by/username?username=instagram",
    headers=headers
)

print(r.json())

That returns structured JSON for the Instagram account.

No browser.
No cookies.
No webdriver.

Real Use Case: Monitoring Competitor Accounts

One practical thing I built was a daily snapshot job for competitor research.

The workflow:

Pull account metadata
Fetch latest posts
Store engagement metrics
Compare growth over time

Here’s a simplified version.

Fetching Profile Data

import requests

API_KEY = "YOUR_KEY"

headers = {
    "x-access-key": API_KEY
}

username = "instagram"

url = f"https://api.hikerapi.com/v2/user/by/username?username={username}"

response = requests.get(url, headers=headers)

data = response.json()

print({
    "username": data.get("username"),
    "followers": data.get("follower_count"),
    "following": data.get("following_count"),
    "posts": data.get("media_count")
})

This is much cleaner than parsing HTML or reverse-engineering internal requests.

Pulling Recent Posts

Once I had the user ID, I could fetch media:

import requests

headers = {
    "x-access-key": "YOUR_KEY"
}

user_id = "25025320"

url = f"https://api.hikerapi.com/v1/user/medias?user_id={user_id}"

response = requests.get(url, headers=headers)

posts = response.json()

for post in posts.get("items", []):
    print({
        "caption": post.get("caption_text"),
        "likes": post.get("like_count"),
        "comments": post.get("comment_count")
    })

I stored these snapshots in PostgreSQL and generated weekly engagement reports.

Why I Preferred This Over Browser Automation

The biggest difference was operational simplicity.

With scraping, the stack usually becomes:

proxies
browsers
CAPTCHA handling
fingerprinting
retries
account warmup
session recovery

With HikerAPI, I mostly just handled:

retries
caching
rate management

That’s a huge reduction in complexity.

Honest Tradeoffs

This is where things get important.

HikerAPI Pros

Very fast to integrate
Structured JSON responses
No browser infrastructure
No session juggling
Easier backend deployment
Predictable pricing
Good for analytics pipelines

Their docs mention 100+ Instagram endpoints and pay-per-request pricing. (HikerAPI)

HikerAPI Cons

You are still depending on a third-party provider.

That means:

external uptime dependency
request-based cost
less control than self-hosted scraping
platform policy risk always exists

Also, if you need full user simulation (posting content, DM workflows, mobile emulation), traditional automation tools may still fit better.

HikerAPI vs instagrapi

This is the comparison I wish I had earlier.

Feature	HikerAPI	instagrapi
Setup speed	Very fast	Medium
Browser needed	No	No
Instagram accounts needed	Usually no	Yes
Session management	Minimal	Required
Proxy maintenance	Minimal	Often needed
Cost	Pay per request	Mostly infrastructure cost
Reliability	Higher for me	Good until sessions fail
Flexibility	API-focused	More customizable

For small hobby projects, instagrapi can absolutely be enough.

For production systems where uptime matters, I found the hosted API approach easier to maintain.

Cost Reality

One thing I liked was the pricing model.

A lot of social scraping providers push expensive monthly subscriptions immediately.

HikerAPI instead uses pay-per-request pricing and includes 100 free requests to test with. (HikerAPI)

That made experimentation cheap for <my project>.

What I’d Still Use Scraping For

I wouldn’t say APIs completely replace scraping.

I’d still consider direct scraping when:

I need unsupported endpoints
I need complete rendering behavior
I need custom interaction flows
I want total infrastructure ownership
I’m building experimental research tooling

But for standard Instagram data extraction, using a managed API dramatically reduced maintenance.

Final Thoughts

The biggest lesson from this project:

The hard part of Instagram automation usually isn’t writing requests. It’s surviving Instagram’s anti-abuse systems over time.

If your goal is shipping a product instead of maintaining scraping infrastructure, using a REST API abstraction can save a lot of engineering effort.

For my use case, HikerAPI ended up being the pragmatic middle ground between:

fragile scraping
and the limitations of the official Instagram APIs

And honestly, that tradeoff was worth it.

DEV Community