A ten-page text-only PDF of 137,782 bytes looks like an easy target for a compressor. It barely moves. On these samples the biggest single lossless change to it came from qpdf 12.4.0's --object-streams=generate, which took it to 132,065 bytes (−4.15%). The same switch moved the image-heavy files by only −0.02% to −0.74%. Both make sense once the bytes are sorted by object type. Scope: five sample files, one Apple-silicon Mac, every setting run once.
Most of the bytes are fonts and images
The accounting: take each top-level object's offset from qpdf --show-xref, size it as the distance to the next offset, and classify it from its dictionary with PyMuPDF 1.26.5. On the text-only file, called X1 below, fonts came to 121,442 bytes (88.14%), content streams 9.16%, page and structure objects 1.95%, and the xref table plus trailer 0.71%. The image-heavy samples were 95.93% image (T1, two text pages with a photo), 98.27% (X2, an eight-page NASA fact sheet) and 99.84% (T2, a scan of T1). Structure in the narrow sense, meaning page objects, xref and file header, added up to somewhere between 0.04% and 2.70% across these four files. None of the four had object streams to begin with.
It is tempting to read "cross-reference stream" as the headline feature and expect the xref to be the win. On X1 the xref table and trailer are 977 bytes. Deleting them outright would save 0.71%, so most of the 4.15% is something else.
Object streams reach the text that Flate never touched
In a classic PDF, every object that is not a stream is written as plain text between obj and endobj. That covers page dictionaries, font descriptors, and the /W width array of a CID font, which can be a long run of numbers. Flate only ever applied to stream data, so this layer sat in the file uncompressed. The schematic below is written from the PDF spec to show the shape. It is not copied from the sample files.
12 0 obj
<< /Type /Font /Subtype /CIDFontType2 /W [ 1 [ 277 ] 3 [ 556 556 ] ... ] >>
endobj
An object stream (PDF 1.5) packs a batch of those objects into one stream, so Flate gets to compress them for the first time. The xref turns into a cross-reference stream at the same time and is compressed as well. Stream objects themselves cannot go into an object stream. Image data and embedded font programs stay exactly where they were, already compressed. A control run agrees: writing X1 with --compress-streams=n --decode-level=generalized, which decodes every stream, made it 2.25 times its original size. Its streams were already compressed; the dictionary layer was what sat raw.
A constructed file shows which bytes move
A constructed sample, D1, makes it visible. It stores the same self-made 800×600 JPEG (91,844 bytes) and the same font subset five times over five pages, 756,741 bytes in total. With object streams alone it went to 633,713 bytes (−16.26%). The bytes classified as font dropped from 266,381 to 134,332, and the new object stream itself was 12,727 bytes. Those five copies of the font carry five copies of the non-stream font objects, /W arrays included, and that is the part Flate could now reach. The font bytes alone fell by 132,049, more than the whole file's drop of 123,028, so the images contributed nothing, which fits the rule that stream data stays out of object streams. That −16.26% comes from a file built to have duplicates, not an average.
The image-heavy samples follow from the same rule: T1 −0.74%, X2 −0.25%, T2 −0.02%. With 95.93% to 99.84% of the bytes in image streams, the slice that object streams can touch is tiny.
The other switches did less
Recompressing Flate streams at level 9 (--recompress-flate --compression-level=9) gave X1 −1.11% on its own, and stacked on object streams it reached 130,933 bytes (−4.97%). A default qpdf in out rewrite gave X1 −0.29% but made D1 1.03% larger. --remove-unreferenced-resources=yes without --pages removed nothing on these files, and --decode-level=specialized changed nothing. PyMuPDF's save(use_objstms=1) on its own gave X1 −0.99%, well short of qpdf's −4.15%, while garbage=4 plus deflate plus object streams reached 132,169 (−4.07%). Why the PyMuPDF object-stream switch alone did so much less was not looked into here, and it is the number I'd chase next.
In the browser I use ImgIng https://imging.ai/. On the same Mac, its lossless structure mode took X1 from 137,782 to 131,156 bytes (−4.81%), the same range as qpdf with everything on, and the scan T2 moved −0.03%. Page count and full-text readback matched the originals. How it gets there internally isn't something visible from the outputs.
On the text-only sample a structural rewrite topped out around 5%, and object streams were most of it; they only reach the non-stream part of that 88% in fonts. Before expecting any gain, qpdf --show-xref on the input is a quick check: entries listed as compressed; stream = … mean the file already uses object streams, and the biggest structural switch has already been spent.
Image-and-text sample: NASA/JPL "Mars Exploration Rover" NASA Facts (2004), Wikimedia Commons, public domain. The other samples are self-made with fictional content. The photo in T1 comes from Wikimedia Commons (CC0).
Top comments (0)