DEV Community

Cover image for I generated 25 file formats at exact byte sizes. Here's every way it broke.
Rahul Anand
Rahul Anand

Posted on

I generated 25 file formats at exact byte sizes. Here's every way it broke.

The problem sounds trivial

You need a 25 MB PDF to test an upload limit. Or a 105-page DOCX for pagination. Or a 1 GB binary to benchmark throughput.

The obvious answer is dd if=/dev/urandom of=test.pdf bs=1M count=25. You get exactly 25 MB. You also get a file that any format-sniffing validator rejects instantly, because it isn't a PDF — it's noise wearing a .pdf extension.

So you need real headers and an exact byte count. Those two requirements fight each other, and every format loses the fight differently.

I built a generator for 25 formats. Here's what actually went wrong.

PDF: the hardcoded /Length

A minimal PDF is a graph of objects and a cross-reference table of byte offsets. My first version emitted a content stream like this:

4 0 obj
<< /Length 44 >>
stream
BT /F1 24 Tf 100 700 Td (Sample) Tj ET
endstream
Enter fullscreen mode Exit fullscreen mode

That /Length 44 was hardcoded. The actual stream is 38 bytes.

Browser viewers rendered it fine. Acrobat rejected it, and so did pypdf. A strict reader trusts /Length, reads 44 bytes, sails past endstream, and hits a parse error.

The second bug in the same object: /Resources << >> was empty while the content stream referenced /F1. A page that draws text with an undeclared font is invalid, even though lenient viewers substitute one silently.

Both only surfaced because I ran the output through a strict parser:

from pypdf import PdfReader
r = PdfReader("out.pdf")           # raises on malformed structure
assert r.pages[0]["/Resources"]["/Font"]["/F1"]["/BaseFont"] == "/Helvetica"
Enter fullscreen mode Exit fullscreen mode

The fix is to compute everything from the assembled bytes — stream lengths and xref offsets — rather than predicting them. Accumulate offsets as you append each object:

const offsets = [];
for (const obj of objects) {
  offsets[obj.id] = body.length;   // measure, never estimate
  body += obj.render();
}
Enter fullscreen mode Exit fullscreen mode

Padding then goes after %%EOF. That's the one region a conforming reader is required to ignore, which is what makes an exact byte count possible without corrupting the document.

DOCX and XLSX: padding that breaks XML

An Office file is a ZIP containing word/document.xml. To hit a target size I padded the XML — by appending filler after the closing tag.

Word opened it. python-docx did not:

lxml.etree.XMLSyntaxError: Extra content at the end of the document
Enter fullscreen mode Exit fullscreen mode

Which is the worst possible failure mode, because the entire audience for generated Office files is automated pipelines that parse them. A fixture that only opens in Word is useless to CI.

The fix is to put the padding somewhere the XML spec permits — inside a comment, before the closing tag:

<w:body>...</w:body><!--AAAAAAAA...--></w:document>
Enter fullscreen mode Exit fullscreen mode

Now python-docx and openpyxl both accept it, and the byte count is still exact.

PNG and JPEG: two different legal hiding places

PNG is a chunk stream. Padding goes in a tEXt ancillary chunk inserted before IEND, with a correct CRC-32. Decoders skip unknown ancillary chunks, so the image still decodes.

JPEG uses FFFE COM markers, each holding up to 65,533 bytes, chained for larger targets. Placement matters: put them immediately after SOI, not near EOI. Padding near the end decodes unreliably across libraries — I never fully root-caused why, and moving it made the problem disappear.

Verify with something strict:

from PIL import Image
Image.open("out.png").verify()   # catches CRC and chunk errors
Enter fullscreen mode Exit fullscreen mode

Structured text has a minimum viable size

Ask for a 20-byte JSON file and there is no correct answer. The smallest valid JSON document with any structure is larger than that.

My generator quietly fell back to emitting whitespace — which produced a file of exactly the right size that failed JSON.parse(). Silent, and much worse than an error.

Empirically measured floors for my output shape:

JSON   69 bytes
XML   114 bytes
SVG   170 bytes
PDF   600 bytes
Enter fullscreen mode Exit fullscreen mode

Below those, round up and tell the user. Don't emit something that satisfies the size check and violates the format.

PPTX: don't hand-roll the theme

PPTX needs presentation.xml, presProps.xml, a slide master, a slide layout, and a theme — and the master and layout reference each other. Get it slightly wrong and PowerPoint shows a repair prompt, which fails the whole "produces valid files" promise.

I stopped trying. Instead I ship a known-good minimal deck as a static asset, fetch it at runtime, clone its blank slide N times, and patch three files: [Content_Types].xml, presentation.xml.rels, and presentation.xml.

Exact sizing comes from a binary search on an orphan partppt/pad.xml, covered by the default XML content type and referenced by nothing. PowerPoint ignores unreferenced parts entirely.

Borrowing a validated artifact beat generating one from spec.

The actual lesson

Every bug above passed the test I was running and failed the test my users would run.

dd and fsutil hit the size exactly and produce nothing parseable. My early versions produced something parseable by lenient readers and broken for everyone else. Both are the same category of mistake: validating against a tolerant consumer.

So validate with the strictest parser your users will realistically reach for. For this project that meant a matrix across pypdf, python-docx, openpyxl, Pillow, wave and a hand-written MP4 box-walker — every format, at several sizes, including the degenerate small ones where most of the bugs lived.

"It opens on my machine" is not validation. It's a single data point from the most forgiving reader you own.


I maintain a browser-based sample file generator that implements all of this — it runs entirely client-side, so nothing is uploaded. Happy to answer format-specific questions in the comments.

Top comments (0)