DEV Community

Mobeen Ahmed
Mobeen Ahmed

Posted on

Why some CDNs serve empty responses to datacenter IPs — and the direct-link fallback that saved my downloader

I run a small video downloader, and one day YouTube downloads broke in a way that made no sense. The provider API answered fine. It gave me a real download link. But when my server tried to fetch that link, it got nothing back. No error, no status code I could argue with — just an empty response.

The same link worked perfectly from my laptop. I pasted it into a browser and the file came down without a fuss. So the link was good, the provider was good, and my code was doing what it had always done. The only difference was where the request came from.

That is the thing about datacenter IPs. The server my site lives on sits in a hosting provider's range of addresses, and plenty of CDNs and media servers treat those ranges differently from a home broadband connection. Some return a 403. Some throttle the connection to a crawl. The one I ran into just closed the door silently and returned nothing at all.

Why do they do it? It is not personal. A datacenter IP is a strong signal that the request is automated: scrapers, bots, mass-download scripts. Media hosts filter those ranges outright, and the filter is crude — it catches honest tools like mine in the same net — but from the CDN's side it works, so it stays.

I spent a while assuming the fault was mine. I checked my timeouts, my headers, my redirect handling. None of it helped, because the request was never going to succeed from that IP range. The honest test was simple: run the same request from a different network and watch it work.

So what do you do when your server cannot fetch a file your visitor's browser can? You stop trying to fetch it yourself.

My downloader has two kinds of links. Most go through my server as a proxy: the visitor clicks download, my server fetches the file from the provider and hands it on, so the visitor gets a clean filename and a download that starts instead of playing in the browser. That works for TikTok, Instagram and Facebook, where the hosts accept my server's requests.

For YouTube, the formats now come back marked as "direct". When a visitor picks one, their browser goes straight to the provider's tunnel link. My server never touches the file. From the visitor's point of view almost nothing changes — they still click a button and the video still downloads. The only real difference is that the file comes from the provider's server, not mine, so I cannot set the filename or guarantee the download behaviour in the same way.

There is a trade-off. A proxied download gives me control: I can set the headers, the filename, and a sensible error message if something fails. A direct link gives me none of that — if the provider's link is dead, the visitor sees whatever the provider serves, often a blank page. But a direct link that works is better than a proxied download that silently fails.

The fix was not a clever workaround. It was accepting that one leg of the journey would never work from my server's address, and designing around it: flag those formats as direct, label them clearly, and let the visitor's browser — which the CDN is happy to serve — do the fetching.

If you are building something similar, here is the lesson I took from it. When a request fails only from your server and works everywhere else, check the IP before you rewrite the code. Try the same request from a home connection. Try it from a phone on mobile data. If it works there and not on your server, you have found your answer, and no amount of header tweaking will fix it. Build the fallback into the design instead — and tell your users what is happening, plainly, rather than letting a spinner spin forever.

That is what I did on ramrom.io. The YouTube formats are direct links, the rest are proxied, and each one says what it is. It is not the elegant solution I wanted. It is the one that actually works.

Top comments (0)