# Backing Up My Instagram Posts, Stories, and Highlights: Lessons From Rate Limits and Blocks

NisaNil — Thu, 12 Nov 2026 19:44:33 +0000

Tags: #python #webscraping #api

If you maintain an Instagram account and want independent backups of your posts, stories, and highlights, this is the kind of problem I ran into.

I wanted a small, repeatable backup process for my own Instagram content. Nothing particularly sophisticated: retrieve available media, save the original files, preserve useful metadata, and avoid downloading the same content repeatedly.

The difficult part wasn't saving images or videos. It was keeping the data collection process reliable.

After running into request limits, access restrictions, and the maintenance requirements of unofficial extraction methods, I started evaluating hosted APIs.

Here are the technical lessons I took from that process.

1. Instagram Backups Are Not Just File Downloads

My initial approach was straightforward:

Identify the Instagram account.
Retrieve its published posts.
Download the available media.
Collect accessible stories and highlights.
Store metadata alongside each file.
Repeat periodically to capture new content.

That sounds simple until different content types enter the picture.

Regular posts are relatively straightforward because they remain associated with an account unless deleted or archived.

Stories are more complicated because they are temporary. Highlights introduce another layer because their contents and organization can change.

A useful backup system therefore needs more than a directory of JPEG and MP4 files.

I wanted to preserve:

Media identifiers
Publication timestamps
Captions and available metadata
Original media URLs when available
Local filenames and checksums
Download timestamps
Information about failed or incomplete downloads

The goal was to make backups verifiable and repeatable.

2. Rate Limits Changed the Design

My early approach treated data collection as a sequence of HTTP requests.

Request a profile, fetch media, download files, and repeat.

The problem is that Instagram doesn't behave like an unrestricted public data repository.

Automated access can encounter rate limits, authentication challenges, session expiration, and other restrictions.

Browser automation also adds maintenance overhead when page structures change.

Unofficial libraries can simplify some operations, but they don't eliminate platform restrictions.

I eventually stopped treating failed requests as something to overcome through increasingly aggressive retries.

Instead, I designed the backup process around bounded requests, caching, explicit failures, and resumable jobs.

The most important lesson was simple:

A backup system that repeatedly triggers restrictions is not a reliable backup system.

3. Why I Started Using a Hosted REST API

I evaluated HikerAPI as an alternative to maintaining my own Instagram extraction infrastructure.

It's a hosted REST API with authentication through an x-access-key header.

Advertised pricing starts at $0.001 per request, and 100 free requests are available for testing.

For my use case, the main advantage was reducing the operational work associated with data extraction.

That doesn't mean a hosted API guarantees unlimited access.

Endpoint availability, account permissions, rate limits, media coverage, pricing, and response consistency still need to be checked.

For backing up my own account, I also consider Instagram's official data export an important option, especially when completeness matters.

A third-party API is most useful when I need programmable retrieval and ongoing automation that its supported endpoints can provide.

4. Testing the API With Python

I began with a basic request to inspect account data.

The following example retrieves a public profile and then requests a chunk of its followers.

The follower request is included to demonstrate the authentication and request structure. It is not a media-backup endpoint.

import requests

headers = {"x-access-key": "YOUR_KEY"}

user = requests.get(
    "https://api.hikerapi.com/v2/user/by/username?username=nike",
    headers=headers
).json()

resp = requests.get(
    "https://api.hikerapi.com/gql/user/followers/chunk",
    params={"user_id": user["pk"]},
    headers=headers
)

print(resp.json())

This helped establish the basic request pattern.

For production use, I wouldn't hardcode the API key or assume every response contains the expected fields.

I would also handle HTTP errors before parsing JSON.

A more defensive request helper looks like this:

import os
import requests

API_KEY = os.environ["HIKERAPI_ACCESS_KEY"]

session = requests.Session()
session.headers.update({
    "x-access-key": API_KEY
})

def fetch_json(url, params=None):
    response = session.get(
        url,
        params=params,
        timeout=30
    )

    response.raise_for_status()

    return response.json()

This keeps authentication centralized and makes request handling easier to maintain.

The actual backup implementation still requires identifying and validating the supported endpoints for posts, stories, and highlights.

I wouldn't assume that retrieving a profile or its followers proves those media endpoints work.

5. Making Downloads Resumable

Once media records are available, the next challenge is downloading files safely.

I wanted downloads to be idempotent: running the same job twice shouldn't create duplicate files.

For each media record, I would store its identifier and download status in a local database.

A minimal downloader can use a temporary file before committing a completed download:

from pathlib import Path
import os
import requests

def save_media(url, destination):
    destination = Path(destination)
    destination.parent.mkdir(
        parents=True,
        exist_ok=True
    )

    temporary = destination.with_suffix(
        destination.suffix + ".part"
    )

    try:
        with requests.get(
            url,
            stream=True,
            timeout=60
        ) as response:

            response.raise_for_status()

            with open(temporary, "wb") as file:
                for chunk in response.iter_content(
                    chunk_size=65536
                ):
                    if chunk:
                        file.write(chunk)

        os.replace(temporary, destination)

    except Exception:
        temporary.unlink(missing_ok=True)
        raise

This function downloads a supplied media URL. It doesn't discover Instagram media URLs by itself.

In a complete implementation, I would also validate content types, calculate checksums, and record completed downloads.

6. Handling Failures Without Aggressive Retries

I separated failures into categories:

Failure	Handling strategy
HTTP 429	Respect `Retry-After` and pause collection
HTTP 401 or 403	Stop and investigate authentication or authorization
HTTP 5xx	Limited retries with exponential backoff
Network timeout	Retry a bounded number of times
Missing media	Record the missing item without inventing a replacement
Invalid response	Log the error and preserve the original metadata

I also cache responses when appropriate.

For a backup job, requesting the same unchanged records repeatedly wastes time and API credits.

A scheduled process should prioritize newly published content and unresolved failures.

Stories deserve particular attention because waiting too long can mean the content is no longer retrievable through the available interface.

7. Hosted APIs Have Their Own Costs

Moving away from self-managed scraping doesn't remove operational considerations. It changes them.

With DIY scraping, I would be responsible for maintaining browser automation, handling extraction failures, and managing infrastructure.

With a hosted API, I depend on an external provider.

The relevant costs include more than the advertised minimum request price:

Endpoint-specific pricing
Pagination requirements
Failed and repeated requests
Media availability
Storage and bandwidth
API changes
Provider outages

I would measure the real cost per successfully backed-up media item rather than relying exclusively on the minimum per-request rate.

I would also keep my local backup format independent of the API provider so that switching data sources doesn't require reorganizing every saved file.

8. What I Would Do Differently

The biggest change in my approach was separating three responsibilities:

Data discovery: Determine which posts, stories, and highlights are available.

Media retrieval: Download available files and verify their integrity.

Backup management: Track completed work, preserve metadata, and identify missing content.

This separation makes failures easier to diagnose.

If an API endpoint changes, the backup database and existing media files remain usable.

If a download fails, the discovery process doesn't need to restart.

If a provider becomes unavailable, I can evaluate another authorized data source without discarding the entire backup structure.

Final Lessons

My main takeaways are:

Don't confuse successful API requests with verified backups.
Treat rate limits and access restrictions as operational boundaries.
Store media metadata separately from downloaded files.
Make backup jobs resumable and idempotent.
Verify support for posts, stories, and highlights individually.
Consider official Instagram exports for account-owned content.
Calculate the real cost per completed backup, not just per API call.

For my own Instagram content, I now prefer a design that can combine official exports with programmable retrieval where appropriate.

A hosted API reduces some extraction-maintenance work, but it doesn't replace validation, error handling, or a proper backup strategy.

The practical lesson: the most important part of an Instagram backup tool isn't how quickly it collects data. It's whether I can trust what it saved when I need to restore something.

DEV Community: NisaNil