I have a PNG I actually look at every morning: one line per account I follow, follower count up the side, one dot per day. When something I posted lands, I can see the step the next day instead of guessing. It is about forty lines of Python and a cron entry, and it took a weekend.
I am writing it down because I was wrong about the hard part.
The thing I was wrong about
I assumed follower counts were public enough that any HTTP client could read them. They are public on the page. They are not public to a program.
A logged-out request to Instagram's own web profile endpoint comes back 401 with require_login, and the profile URL redirects to the login screen before it renders anything. I checked that from a plain server, then again through a residential address in case it was an IP reputation problem. Same answer both times. Whatever the page shows a human, it does not show a script.
That left three options:
- Keep a pool of Instagram accounts alive and feed their cookies to the script.
- Ask anyone who uses my tool to paste their own session cookie.
- Pay someone who already solves this.
The first option is a job, not a feature — accounts get flagged and you find out when your chart goes flat. The second one makes the tool unusable for anyone but me. I took the third.
The fetch
This is the whole data layer. Two calls: resolve the handle to an internal id, then ask for whatever you want about that id.
import requests
headers = {"x-access-key": "YOUR_KEY"}
user = requests.get("https://api.hikerapi.com/v2/user/by/username?username=spotify", headers=headers).json()
resp = requests.get(f"https://api.hikerapi.com/v2/user/stories?user_id={user['pk']}", headers=headers)
print(resp.json())
Two things I learned by running it rather than reading about it.
First, the payload is the app's own response, not a tidied-up version of it. On the call above the profile fields live one level down, under user, so in my own code the id is user["user"]["pk"] and the number I care about is user["user"]["follower_count"]. Print the keys before you index into anything.
Second, Instagram ids are large enough to lose precision once JSON hits a float. There is a safe_int=true query parameter that returns them as strings. I turned it on after an id came back ending in a zero that was not there before, which is a fun bug to find at midnight.
One row a day is the whole trick
There is no "follower history" to request. Nobody has it. The history is the thing you create by writing a row every day and never deleting the file.
import csv, datetime, os, requests
HANDLES = ["spotify", "nasa"]
HEADERS = {"x-access-key": os.environ["HIKER_KEY"]}
CSV_PATH = "followers.csv"
def snapshot(handle):
r = requests.get(
"https://api.hikerapi.com/v2/user/by/username",
params={"username": handle, "safe_int": "true"},
headers=HEADERS,
timeout=60,
)
r.raise_for_status()
u = r.json().get("user") or {}
return {
"date": datetime.date.today().isoformat(),
"handle": handle,
"followers": u.get("follower_count"),
"following": u.get("following_count"),
"posts": u.get("media_count"),
}
def main():
new = os.path.exists(CSV_PATH) is False
with open(CSV_PATH, "a", newline="") as f:
w = csv.DictWriter(f, fieldnames=["date", "handle", "followers", "following", "posts"])
if new:
w.writeheader()
for handle in HANDLES:
try:
row = snapshot(handle)
except Exception as err: # one bad handle must not kill the day
print(f"{handle}: {err}")
continue
if row["followers"] is None: # private or renamed account
print(f"{handle}: no follower count in the response, skipping")
continue
w.writerow(row)
print(row)
if __name__ == "__main__":
main()
One cron line and it never needs attention again:
17 6 * * * cd /srv/followers && /usr/bin/python3 snapshot.py >> snapshot.log 2>&1
I put it at 6:17 rather than on the hour out of superstition about everyone else's cron jobs. Append mode matters more than it looks: the file is the only copy of the history, so the script is only ever allowed to add to it.
Drawing it
import csv, collections
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
series = collections.defaultdict(list)
with open("followers.csv") as f:
for row in csv.DictReader(f):
if row["followers"]:
series[row["handle"]].append((row["date"], int(row["followers"])))
fig, ax = plt.subplots(figsize=(10, 5))
for handle, points in series.items():
points.sort()
ax.plot([d for d, _ in points], [n for _, n in points], marker="o", label=handle)
ax.set_ylabel("followers")
ax.legend()
fig.autofmt_xdate()
fig.tight_layout()
fig.savefig("followers.png", dpi=140)
That is the whole chart. No dashboard, no database, no frontend. A CSV and a PNG are enough for something only I read.
What was harder than expected
Not the API. The waiting.
There is no backfill. I wanted to open the chart and see the last year, and instead I got a single dot. The day you start the cron is day zero, and the chart is useless for about a week and boring for about a month. I nearly deleted the project twice in that first week because it looked like it was not doing anything.
It was doing exactly what it was supposed to. I just had not understood that building a time series means the tool is worth nothing on the day you finish it and a bit more every day after. If I had known that, I would have started the cron first and written the chart later, instead of the other way round.
The second small surprise: a day will be missing eventually. The machine reboots, the network drops, something. Do not interpolate it — a gap that stays a gap is honest, and a line drawn through a hole is a quiet lie about a day you did not measure.
What it costs
Per request, and only for requests that answer. My whole chart is two handles times one call a day, which is sixty-something calls a month — far below where the pricing starts to matter. Published rates begin at a tenth of a cent per call, and the signup gives you a hundred free ones, which is enough to find out whether the shape of the data fits your idea before you spend anything: HikerAPI.
That last part is the bit I would tell myself a weekend ago. Before writing the script, spend ten of those free calls just printing raw responses for the accounts you actually care about. Half the design decisions in the code above came from looking at what the payload really contains rather than what I assumed it would.
Would I build it again
Yes, and smaller. The version I am still running is the first one, minus the parts I thought I needed: a database, a web view, alerting on a threshold. What survived is a cron line, a CSV, and a chart I open with my coffee. That turned out to be the entire product.
Top comments (0)