Caching provider API responses in flat files to survive outages on shared hosting

My video downloader site runs on plain shared hosting — no Redis, no Memcached, no background workers. When the third-party API it depends on has a busy spell, every visitor pasting a link gets an error and a shrug. So I added a cache: provider API responses are saved as flat files, and when the API is down or throwing errors, the site serves the last good answer instead of failing. Here is how it works and why flat files were enough.

What I cache, and what I don't

When a visitor pastes a video link, the server calls the provider API and gets back JSON: the resolved download URLs, formats, titles, thumbnails. That JSON is the perfect cache payload — small, text, and keyed naturally by the video URL.

What I deliberately do not cache: the video files themselves. Downloading and storing other people's videos on my server would be a legal grey area and a disk-space disaster. The cache only holds metadata — the map to the files, not the files.

The layout

A cache/ directory holds one file per resolved URL. The filename is an MD5 of the normalised video URL with a .json extension. Each file stores two things: the payload and the time it was saved.

$key  = md5($videoUrl);
$file = __DIR__ . '/cache/' . $key . '.json';

if (file_exists($file)) {
    $cached = json_decode(file_get_contents($file), true);
    if ($cached && (time() - $cached['saved_at']) < $ttl) {
        return $cached['data'];
    }
}

The TTL depends on the platform. For TikTok and Instagram links, I use a few hours — the URLs the providers return are often signed and expire anyway, so caching longer would hand out dead links. For Facebook and YouTube, where links also expire, I keep it short. The cache is not there to be permanent; it is there to bridge the gap during a provider outage.

The outage path

The important bit is when the cache gets used. The normal flow is: check the cache, and if it misses, call the provider API. The outage flow is different — if the API call fails or returns an error after my retry loop, I do a second, more generous cache read: anything that was saved successfully before, even if it is older than the TTL, is returned with a small note that the link is from an earlier lookup.

// API failed — try a stale fallback before giving up
$stale = json_decode(@file_get_contents($file), true);
if ($stale && isset($stale['data'])) {
    return $stale['data']; // honest label: 'from an earlier lookup'
}

This matters because the provider's 'Service is busy' spells happen during peak hours — exactly when the most visitors are trying to download. Serving a slightly stale answer with an honest label beats an error page every time.

Why flat files instead of something fancier

Shared hosting ruled out Redis and Memcached. SQLite would have worked, but a cache of JSON responses doesn't need queries — it needs a key lookup, which is exactly what a filename is. Flat files have three advantages here:

1. Atomic writes. I write to a temp file and rename() it into place, so a concurrent request never reads a half-written cache entry.
2. Zero setup. No service to install, no daemon to keep alive, no config for the host to forget after a migration.
3. Easy eviction. A small cron-style cleanup script deletes files older than a day, so the cache never grows unbounded. Disk is cheap, but unbounded growth on shared hosting eventually gets you a support ticket.

What the cache cannot do

It cannot fix expired signed URLs — if the provider rotates signing keys, a cached response hands out dead links, and the honest thing is to show an error. It also cannot cache API responses that were errors themselves, which is why I only write to the cache after validating the payload looks like real data (has a download URL, correct structure). Caching an error would just make the error permanent.

The result

During the provider's next busy spell, visitors kept downloading instead of hitting error pages, and I only found out about the outage from my logs. On shared hosting, boring solutions win: a directory of JSON files, a TTL, and a fallback read. If you run any kind of tool site that leans on a third-party API, a stale-cache fallback is the cheapest resilience you can buy — and on hosting where you can't install anything, flat files are all you need.

I run this on ramrom.io, a small multi-platform video downloader I built on plain PHP.