DEV Community

Dev Bhamaria
Dev Bhamaria

Posted on AI-assisted

I ran my PDF tools on a 5 GB file and a 10,000-page file. Here is what worked and what did not.

My last article tested IHatePDF on small PCs. This one goes the other way: very big files. I wanted to know where the tools stop working. I had huge files I wanted to compress, and the online tools I found had a size limit. I wanted to be sure my offline tool did not.

The two files

  • A 5.0 GB PDF of 1,436 scanned pages (300 dpi JPEG, every page different).
  • A 10,000-page text PDF, only 14.6 MB: a heading, 40 lines and a small table on every page.

Both ran in the Windows app, using the app's own memory budget. The 10,000-page run was on Windows 11 with 32 GB of RAM.

The 5 GB file

Tool Result Time Most memory used by the app
Watermark finished 86 s 924 MB
Page numbers finished 56 s 1.0 GB
Merge (+ 20 pages) finished 65 s 1.0 GB
Protect (AES-256) finished 62 s 990 MB
PDF/A finished 71 s 983 MB
Compress finished, every image recompressed 743 s 2.1 GB

Five of these tools used about 1 GB of memory or less on a 5 GB file. Compress is the heavy one: about 12 minutes and 2.1 GB.

What did not work: previews over 2 GB

Tools that show page thumbnails could not open the 5 GB file at all. Rotate sat on "Loading pages…" with no thumbnail for 10 minutes.

The cause is a limit in the page engine: it cannot allocate a block of memory of 2 GB or more (new Uint8Array(2 GB) fails, in the page and in workers), and the PDF viewer library keeps one the size of the file. So page previews cannot open files over 2 GB. The app now says so: "This PDF is 5.0 GB. Page previews work up to 2 GB." It also lists the tools that still work, and Split, which needs previews, shows the same message.

The 10,000-page file

All 14 tools finished.

Tool Time Most memory used by the app
Watermark 6 s 1.2 GB
Rotate 4 s 1.2 GB
Page numbers 5 s 1.2 GB
Merge (+ 20 pages) 3 s 0.9 GB
Organize (reverse order) 4 s 1.2 GB
Split (every page to ZIP) 5 s 1.3 GB
Protect (AES-256) 4 s 0.9 GB
Compress 17 s 1.7 GB
PDF/A 4 s 0.9 GB
PDF to Markdown 32 s 2.1 GB
PDF to Word 9 s 4.1 GB
PDF to Excel 67 s 2.0 GB
Redact (a word on every page) 63 min 1.7 GB
Edit PDF (open and retype a paragraph) 3 s 1.5 GB

Twelve of the fourteen finished in 32 seconds or less. Two things to know:

  • PDF to Word on 10,000 pages needs about 4 GB of RAM. On a 4 GB lab PC the memory limit would slow it down or stop it, so split such a file first.
  • Redact on every one of 10,000 pages takes about an hour. It redraws each page. That is fine for a few pages and slow for all of them.

What I would change

What surprised me most was how unoptimised IHatePDF was at the start. In the early version, a file of about 100 MB used roughly 5 to 7 times its own size in RAM. I had to optimise it. (The 5 to 7 times figure is from my own early testing, not from the test report.)

Code and the full test report: https://github.com/Burthcer/IHatePDF

Top comments (0)