A couple of weeks ago I wrote about LinkUnzip, a Chrome extension that pulls files out of a zip on the internet without downloading the zip. It reads the zip's table of contents with an HTTP Range request, then downloads only the bytes of the files you want and writes them straight into a folder.
The idea worked. The install didn't.
To write files into a real folder, version 1.0 needed a small helper program on your PC. I wrote it in Rust and the extension talked to it through Chrome's native messaging. It was fast. It was also a Windows .exe from someone nobody has heard of, unsigned, so Windows SmartScreen warned about it. A few people told me straight up that the helper was a dealbreaker. And of the first five people who uninstalled the extension, four were on Linux or ChromeOS, where the helper didn't exist at all. They installed it, found out it couldn't work, and left.
So for 1.1.0 I moved the whole engine into the browser. Here's how that went, including the parts where the browser is slower.
What the browser already has
It turns out almost everything the Rust helper did has a browser API now:
-
fetchwith aRangeheader gets any slice of the zip. -
DecompressionStream("deflate-raw")inflates zip data natively. Zip uses raw deflate, without the zlib header, and Chrome supports exactly that. - CRC-32 is about 40 lines of JavaScript with a lookup table.
- The File System Access API (
showDirectoryPicker,createWritable) writes into a real folder the user picked. - An offscreen document gives the engine a page that keeps running after the popup closes.
The core is pretty small. Simplified, one file looks like this:
// fetch one file's compressed bytes and inflate them as they arrive
const res = await fetch(zipUrl, { headers: { Range: `bytes=${start}-${end}` } });
if (res.status !== 206) throw new Error("the server ignored the Range header");
const plain = res.body.pipeThrough(new DecompressionStream("deflate-raw"));
await plain.pipeTo(await fileHandle.createWritable());
The real code does more: it groups neighbouring files into one request, skips each file's local header, counts CRC-32 and size on the way through, and only gives the file its real name once both match. But that's the heart of it, and it's all stuff the browser ships.
Where the browser is slower
I measured both versions on the same machine.
The Blender 4.2 zip is 365.8 MB with 5,518 files. The Rust helper extracted all of it in 15 to 17 seconds. The browser version takes 52 seconds.
The download isn't the problem. The folder is. Chrome runs file system operations one at a time, and each file needs about four of them: get a handle, create a writable, write, close. At roughly 2 ms each, that's about 7 ms per file no matter how many connections are downloading in parallel. For a zip of big files you don't notice. For a zip of thousands of tiny files you do. In a benchmark with 70,000 tiny files, Chrome took 500 seconds where plain Node took 101.
I decided that's fine as long as the extension says so. The helper still exists as an optional extra for people who want the speed.
Chrome has opinions about where you save
My first test build asked you to pick a folder. Then I tried to pick Downloads and Chrome said no. The folder picker refuses Downloads, Desktop and Documents themselves. You can pick a folder inside them, but not the folder itself. For most people "put it in Downloads" is the whole point, so that was a problem.
The fix was to use Chrome's own downloads for the default case. Each checked file becomes a Blob and goes through chrome.downloads.download into Downloads/<zip name>/. No picker, no permission prompt. Three things I learned the hard way:
- If the Blob has no type, Chrome renames every file to .txt.
application/octet-streamfixes it. - Chrome holds back some file types (.dll, .ini, .cfg, .manifest and a few more) until you click Keep on each one. The extension now counts them before it starts and warns you.
- It's slow: about 150 ms per file, so 6 or 7 files a second. For a zip with thousands of files the popup now tells you how long it will take and suggests picking a folder instead, which is the fast path (that 7 ms per file).
The bug that only showed up for real users
The first build worked in all my tests. Then I tried it as a normal user and got "the engine didn't start".
Chrome stops an extension's background service worker when it's idle. My offscreen page, the one doing the actual work, outlived the worker and was left holding a dead connection. When the worker woke up again, it saw that the page already existed and waited for it to say hello. It never did.
Now a busy page reconnects by itself, an idle leftover page gets a second and a half and is then replaced, and running extractions survive a worker restart. The test suite stops the worker on purpose, both while idle and in the middle of an extraction, so this can't quietly come back.
A bonus: preview
Once the engine could pull one file out of a zip, showing that file was a small step. Every file in the list now has an eye icon that opens it in a tab: text and code, CSV as a table, images, PDF, audio and video. Only that one file is downloaded.
One photo out of the COCO train2017 dataset, an 18.01 GB zip, downloads 219 KB for the preview. For CSV it stops even earlier: the first 1,000 rows of movies.csv from the MovieLens 32M dataset cost 23.3 KB out of a 227.9 MB zip. Checking whether a dataset is the one you want before downloading 18 GB of it is honestly the feature I use most now.
What still doesn't work
- Servers that ignore Range requests and send the whole file every time. GitHub's green "Download ZIP" button is one of them. For those you still need the helper, which streams the file once from start to end.
- RAR, 7z and password-protected zips.
- I've only tested it on Windows so far. It should work in Chrome on Mac, Linux and ChromeOS, since nothing in it is Windows-specific any more, but I haven't seen it with my own eyes yet.
Try it
It's free and open source, no account, nothing to install besides the extension:
- Chrome Web Store: https://chromewebstore.google.com/detail/dapeiljhjlaelolngnoagobeojcpkica?utm_source=devto&utm_medium=article
- Code: https://github.com/CincaAlex/LinkUnzip
If you're on Mac or Linux I'd really like to know if it works for you. And if you've hit the same Chrome limits with the File System Access API, how did you get around the one-operation-at-a-time thing? I couldn't find a way.
Top comments (0)