Who unfollowed you? My cron finds out

#webdev #nextjs #api #typescript

I have a small side project that shows people who unfollowed them on Instagram. Most of it runs in the user's browser. But I wanted one paid feature: if your follower count goes down while your computer is off, you get an email with the names of who left.

Sounds easy. It wasn't. The server has to get follower lists by itself, from Vercel functions, and I really did not want to keep anyone's Instagram password. I'm not an expert on this, so this is just what I learned building it.

Why not do it myself

The official Graph API only covers Business/Creator accounts, and only your own followers. Doing it myself would mean running my own logged-in Instagram accounts, rotating them, and dealing with checkpoints and bans. I didn't want a second project just to keep the first one alive.

Only crawl when something happened

For follower lists I used HikerAPI: plain REST, one x-access-key header, and you pay per request.

Paying per request changed how I built everything. The normal cron never fetches names. It only checks if the count changed. When the count goes down, a second job pulls the list and compares it with the last snapshot.

Only 25 per page

I assumed about 100 users per page. /v1/user/followers/chunk gives you 25, and each request takes about 2 seconds. A 2,000-follower account is 80 requests, around 160 seconds. Vercel functions time out way before that.

So I crawl in slices. Each cron run fetches a few pages, saves the max_id cursor, and the next run continues from there

export async function fetchFollowerSlice(userId: string, cursor: string | null, maxRequests: number) {
  const users: Follower[] = [];
  let at = cursor;
  try {
    for (let i = 0; i < maxRequests; i++) {
      const qs = new URLSearchParams({ user_id: userId });
      if (at) qs.set("max_id", at);
      const page = readPage(await hiker(`/v1/user/followers/chunk?${qs}`));
      users.push(...page.users);
      if (!page.cursor || page.cursor === at) return { users, next: null }; // done
      at = page.cursor;
    }
    return { users, next: at }; // resume here next tick
  } catch {
    return null; // keep the old cursor, retry later
  }
}

Other things I ran into:

The response shape is not always the same. v1 chunk returns a [users, max_id] tuple, other versions return an object, and v2 /user/followers now answers 410. My readPage accepts both. If parsing still fails, the user gets a plain "your count dropped" email instead of nothing.
Private accounts answer 403. Nothing to crawl there, so those users only get the count alert.
Don't lose the cursor. If a request times out in the middle, the function returns null and the saved cursor stays where it was. The next run tries the same page again instead of skipping 25 people.

The cost math

At $0.001 per request, a full 2,000-follower crawl is about $0.08. And it only runs when someone actually lost followers, so most days it costs nothing.

One thing to know: that price needs a minimum prepaid balance. Plan for it before you launch the feature.

If I started again

I would add two boring things on the first day.

First, an alert for when the API says no. I found this one by accident. I was testing the crawl and every request came back 402 InsufficientFunds, but my account still had $1.02 in it. My free credit was gone and the cheap rate needs a minimum balance, so $1 was not enough to do anything. My code caught the error and kept the cursor like it should, so nothing looked broken. It was just silently not working. Now the cron sends me one email a day when it sees a 402.

Second, an off switch. The names feature is behind a flag now, so if the API stops working I turn it off and people still get the normal count email. Nothing else in the app breaks.

The alert is about 30 lines and the flag is one line. I wish I wrote them before, not after.

I'm part of HikerAPI's user-rewards program,they credit my account for posts like this.

DEV Community