Understanding PDF Incremental Updates (%%EOF Chains): How Appended Byte Streams Leak Deleted Data

When a user edits a PDF document—whether removing a sensitive paragraph, applying a black redaction box, replacing an embedded image, or adding a digital signature—most people assume the modified file overwrites the old bytes on disk. In reality, the ISO 32000-1 (PDF 1.7) and ISO 32000-2 (PDF 2.0) specifications include an architectural feature designed in the early 1990s for slow magnetic disks and network streams: Incremental Updates.

Instead of rewriting the entire binary file from byte 0, a PDF writer can simply append the modified objects, a new Cross-Reference (xref) table or stream, a new trailer dictionary containing a /Prev byte-offset pointer, and a fresh %%EOF marker to the very end of the existing file.

While incremental updates are essential for multi-signer cryptographic workflows (PAdES), they are also one of the most common sources of accidental data leakage in legal, financial, and medical documents. Anyone with a hex editor—or a 30-line script—can truncate the file at an earlier %%EOF boundary and recover the exact document state prior to the edit.

In this technical deep dive, we will examine:

1. The binary layout of a multi-revision PDF file and how /Prev pointer chains work.
2. How to detect and parse multiple %%EOF revisions and startxref offsets using browser-native ArrayBuffer and Uint8Array APIs.
3. How to extract and compare historical object generations (obj ... endobj) across revisions.
4. How to reconstruct and flatten a multi-revision PDF into a single-generation binary stream so historical layers are permanently purged.

---

1. Binary Architecture of a Multi-Revision PDF

A standard, single-generation PDF file consists of four sequential regions:

1. Header: %PDF-1.7 followed by a binary comment marker (four bytes > 127, e.g., %âãÏÓ) to signal binary-safe transport.
2. Body: Indirect objects (<objNum> <genNum> obj ... endobj), which hold page trees, font descriptors, content streams, and metadata dictionaries.
3. Cross-Reference Table (xref): A byte-offset lookup table mapping every <objNum> <genNum> pair to its exact byte offset from the beginning of the file (n for in-use objects, f for freed objects).
4. Trailer & EOF: A trailer << ... >> dictionary pointing to the /Root catalog and /Info dictionary, followed by startxref, the byte offset of the xref section, and %%EOF.

When an application saves an Incremental Update, the original bytes remain 100% untouched. A new section is appended after the original %%EOF:

+-------------------------------------------------------------------+
| Byte 0: %PDF-1.7                                                  |
|                                                                   |
| 1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj                  |
| ...                                                               |
| 12 0 obj << /Length 84 >> stream                                  |
|   BT /F1 12 Tf (CONFIDENTIAL: Salary = $185,000) Tj ET            |
| endstream endobj                                                  |
|                                                                   |
| xref                                                              |
| 0 13                                                              |
| 0000000000 65535 f                                                |
| ...                                                               |
| 0000000412 00000 n   <-- Object 12 0 at offset 412                |
| trailer << /Size 13 /Root 1 0 R >>                                |
| startxref                                                         |
| 1024                                                              |
| %%EOF  <================== [REVISION 1 BOUNDARY: Byte 1420]       |
+-------------------------------------------------------------------+
| 12 0 obj << /Length 68 >> stream                                  |
|   BT /F1 12 Tf (CONFIDENTIAL: Salary = [REDACTED]) Tj ET          |
| endstream endobj                                                  |
|                                                                   |
| xref                                                              |
| 0 1                                                               |
| 0000000000 65535 f                                                |
| 12 1                                                              |
| 0000001422 00000 n   <-- Object 12 0 now points to offset 1422    |
| trailer << /Size 13 /Root 1 0 R /Prev 1024 >>                     |
| startxref                                                         |
| 1548                                                              |
| %%EOF  <================== [REVISION 2 BOUNDARY: Byte 1790]       |
+-------------------------------------------------------------------+

How a PDF Viewer Resolves Objects

When a compliant PDF viewer opens this 1,790-byte file:

1. It reads the last 1,024 bytes of the file to locate the final startxref (1548).
2. It jumps to byte 1548 and reads the incremental xref subsection, which maps Object 12 0 to byte offset 1422 (the redacted text).
3. It reads the /Prev 1024 entry in the latest trailer, jumps back to byte 1024 to load the remaining unmodified objects (1 0 through 11 0), and ignores the older entry for Object 12 0 (offset 412) because a newer offset was already registered.

Crucially, byte offset 412 still exists inside the file. The string (CONFIDENTIAL: Salary = $185,000) is sitting in plain sight inside the binary buffer, unreferenced by the active cross-reference chain.

---

2. Detecting Incremental Revisions in JavaScript (Uint8Array)

Because PDF files mix ASCII structural tokens (obj, endobj, xref, startxref, %%EOF) with arbitrary compressed binary streams (FlateDecode, DCTDecode), converting a whole PDF into a UTF-8 string via TextDecoder corrupts byte offsets whenever multi-byte sequences appear inside binary streams.

Instead, all offset scanning must operate directly on raw bytes (Uint8Array). Below is a zero-dependency scanner that locates every %%EOF marker in a PDF buffer, verifies whether it belongs to a valid startxref block, and extracts the exact byte boundaries of each historical revision:

/**
 * Scans a raw PDF Uint8Array for all incremental %%EOF revision boundaries.
 * @param {Uint8Array} bytes - Raw PDF file bytes
 * @returns {Array<{ revision: number, eofOffset: number, byteLength: number, startxref: number | null }>}
 */
function scanPdfRevisions(bytes) {
  // ASCII byte sequence for "%%EOF" -> [0x25, 0x25, 0x45, 0x4F, 0x46]
  const EOF_SIG = [0x25, 0x25, 0x45, 0x4f, 0x46];
  const decoder = new TextDecoder("latin1");
  const revisions = [];

  for (let i = 0; i <= bytes.length - EOF_SIG.length; i++) {
    if (
      bytes[i] === EOF_SIG[0] &&
      bytes[i + 1] === EOF_SIG[1] &&
      bytes[i + 2] === EOF_SIG[2] &&
      bytes[i + 3] === EOF_SIG[3] &&
      bytes[i + 4] === EOF_SIG[4]
    ) {
      // Advance past optional trailing CR/LF (\r\n) after %%EOF
      let endOffset = i + EOF_SIG.length;
      while (
        endOffset < bytes.length &&
        (bytes[endOffset] === 0x0d || bytes[endOffset] === 0x0a || bytes[endOffset] === 0x20)
      ) {
        endOffset++;
      }

      // Inspect the preceding 128 bytes to extract the associated "startxref <offset>"
      const lookbackStart = Math.max(0, i - 128);
      const tailWindow = decoder.decode(bytes.subarray(lookbackStart, i));
      const match = tailWindow.match(/startxref\s+(\d+)\s*$/);

      if (match) {
        revisions.push({
          revision: revisions.length + 1,
          eofOffset: i,
          byteLength: endOffset,
          startxref: parseInt(match[1], 10),
        });
      }
    }
  }

  return revisions;
}

Example Output on an Edited Document

If you run scanPdfRevisions() on a PDF that has been saved three times across different editors, you will see output like this:

[
  { "revision": 1, "eofOffset": 48210, "byteLength": 48216, "startxref": 47102 },
  { "revision": 2, "eofOffset": 51940, "byteLength": 51946, "startxref": 51488 },
  { "revision": 3, "eofOffset": 54312, "byteLength": 54319, "startxref": 53990 }
]

Any revision where revision < revisions.length can be extracted as a standalone, valid PDF document simply by slicing the buffer from byte 0 to byteLength:

/**
 * Extracts a specific historical snapshot of a PDF prior to subsequent edits.
 * @param {Uint8Array} pdfBytes
 * @param {number} revisionIndex - 0-based index from scanPdfRevisions()
 * @returns {Blob}
 */
function extractHistoricalRevision(pdfBytes, revisionIndex) {
  const revisions = scanPdfRevisions(pdfBytes);
  if (revisionIndex < 0 || revisionIndex >= revisions.length) {
    throw new RangeError("Invalid revision index");
  }
  const targetEnd = revisions[revisionIndex].byteLength;
  const slice = pdfBytes.slice(0, targetEnd);
  return new Blob([slice], { type: "application/pdf" });
}

Because each previous %%EOF terminated a complete PDF state at the moment it was originally written, pdfBytes.slice(0, targetEnd) produces a syntactically valid PDF containing the exact text, images, annotations, and metadata from that earlier save.

---

3. Following /Prev Chains in Cross-Reference Streams (PDF 1.5+)

Modern PDF producers (such as Adobe Acrobat, Quartz PDFContext, and Chromium's SkPDF) frequently use Cross-Reference Streams (/Type /XRef, introduced in PDF 1.5) or Hybrid-Reference Files instead of plain-text xref tables.

In an /XRef stream PDF, startxref points not to the word xref, but to an indirect stream object whose dictionary carries the /Prev pointer and /W field-width array:

45 0 obj
<<
  /Type /XRef
  /Size 46
  /W [1 3 1]
  /Root 1 0 R
  /Info 38 0 R
  /Prev 47102
  /Filter /FlateDecode
  /Length 142
>>
stream
  ... zlib-compressed binary entries ...
endstream
endobj

To audit what changed between revisions, we can parse the trailer dictionaries or /XRef stream headers at each startxref offset and walk the /Prev linked list backward from the end of the file:

/**
 * Walks the /Prev pointer chain starting from the final startxref offset.
 * @param {Uint8Array} bytes
 * @param {number} initialXrefOffset
 * @returns {Array<{ xrefOffset: number, prevOffset: number | null, isStreamXref: boolean }>}
 */
function traceXrefChain(bytes, initialXrefOffset) {
  const decoder = new TextDecoder("latin1");
  const chain = [];
  const visited = new Set();
  let currentOffset = initialXrefOffset;

  while (currentOffset !== null && currentOffset >= 0 && currentOffset < bytes.length) {
    if (visited.has(currentOffset)) {
      break; // Guard against malformed circular /Prev pointers
    }
    visited.add(currentOffset);

    // Read a 2 KB window starting at currentOffset to inspect trailer or /XRef dict
    const windowEnd = Math.min(bytes.length, currentOffset + 2048);
    const snippet = decoder.decode(bytes.subarray(currentOffset, windowEnd));

    const isStreamXref = /^\s*\d+\s+\d+\s+obj\b/.test(snippet) && snippet.includes("/XRef");
    const prevMatch = snippet.match(/\/Prev\s+(\d+)/);
    const prevOffset = prevMatch ? parseInt(prevMatch[1], 10) : null;

    chain.push({
      xrefOffset: currentOffset,
      prevOffset,
      isStreamXref,
    });

    currentOffset = prevOffset;
  }

  return chain;
}

---

4. Identifying Shadowed Objects (Dead Bytes)

When an incremental update modifies an existing object—say, Object 7 0 obj (a page content stream or an /Info metadata dictionary)—the original 7 0 obj at byte offset $O_1$ becomes shadowed by the new 7 0 obj at byte offset $O_2$ ($O_2 > O_1$).

Even if a user edits the document title, author name, or page contents and saves the PDF, a forensic scan for repeated <objNum> <genNum> obj headers reveals every shadowed object still resident in the file:

/**
 * Finds indirect objects defined multiple times across incremental updates.
 * @param {Uint8Array} bytes
 * @returns {Map<string, number[]>} Map of "objNum genNum" -> array of byte offsets
 */
function findShadowedObjects(bytes) {
  const decoder = new TextDecoder("latin1");
  const rawText = decoder.decode(bytes);
  const objRegex = /(?:^|[\r\n\s])(\d+)\s+(\d+)\s+obj\b/g;

  const occurrences = new Map();
  let match;

  while ((match = objRegex.exec(rawText)) !== null) {
    const key = `${match[1]} ${match[2]}`;
    const offset = match.index + (match[0].length - match[0].trimStart().length);
    if (!occurrences.has(key)) {
      occurrences.set(key, []);
    }
    occurrences.get(key).push(offset);
  }

  // Filter down to objects that have more than 1 historical definition in the file
  const shadowed = new Map();
  for (const [key, offsets] of occurrences.entries()) {
    if (offsets.length > 1) {
      shadowed.set(key, offsets);
    }
  }
  return shadowed;
}

If findShadowedObjects(bytes) returns Map { "12 0" => [412, 1422], "38 0" => [910, 1680] }, it proves that Object 12 0 and Object 38 0 were overwritten via incremental updates and their original definitions at bytes 412 and 910 are still embedded inside the binary payload.

---

5. How to Properly Sanitize and Flatten Incremental Updates

Simply deleting bytes after the first %%EOF reverts the document to its oldest version, whereas deleting bytes before the last update breaks the file because the incremental section only contains the delta (changed objects), not the unchanged pages, fonts, or resources from Revision 1.

To keep only the latest visible state of the document while permanently destroying all historical revisions and shadowed objects, a sanitizer must perform a full object-graph garbage collection and serialization:

1. Resolve the Latest Object Table: Parse the active cross-reference chain starting from the final startxref so that each <objNum, genNum> points exclusively to its newest byte offset.
2. Reachability Traversal (Mark-and-Sweep from /Root): Traverse all indirect references (R) reachable from the latest /Root catalog dictionary (and optionally sanitized /Info metadata). Unreferenced objects from deleted pages, old annotations, or superseded content streams are excluded from the reachable set.
3. Renumber and Serialize from Scratch: Write a fresh %PDF-1.7 header, serialize only the live reachable objects sequentially with generation number 0, compute a brand-new single xref table, and emit a single trailer (with no /Prev key) and a single %%EOF.

Here is how this garbage-collected rewrite works using pdf-lib in a browser Web Worker or Node.js environment by cloning the active document tree into a clean container:

import { PDFDocument } from "pdf-lib";

/**
 * Rebuilds a multi-revision PDF into a clean, single-generation PDF buffer
 * with all incremental update layers, /Prev chains, and orphaned objects removed.
 *
 * @param {Uint8Array} inputPdfBytes
 * @returns {Promise<{ cleanBytes: Uint8Array, removedRevisions: number, bytesSaved: number }>}
 */
export async function flattenIncrementalUpdates(inputPdfBytes) {
  const revisionsBefore = scanPdfRevisions(inputPdfBytes);

  // Load the active state of the PDF (resolves the latest xref chain)
  const srcDoc = await PDFDocument.load(inputPdfBytes, {
    ignoreEncryption: false,
    updateMetadata: false,
  });

  // Create a brand-new, empty PDF container with a clean xref table
  const cleanDoc = await PDFDocument.create();

  // Copy only the active pages and their reachable resource graphs (fonts, XObjects, streams)
  const pageIndices = srcDoc.getPageIndices();
  const copiedPages = await cleanDoc.copyPages(srcDoc, pageIndices);
  for (const page of copiedPages) {
    cleanDoc.addPage(page);
  }

  // Copy standard document-level metadata from the latest revision only
  if (srcDoc.getTitle()) cleanDoc.setTitle(srcDoc.getTitle());
  if (srcDoc.getAuthor()) cleanDoc.setAuthor(srcDoc.getAuthor());
  if (srcDoc.getSubject()) cleanDoc.setSubject(srcDoc.getSubject());
  cleanDoc.setProducer("Single-Generation PDF Sanitizer");
  cleanDoc.setModificationDate(new Date());

  // Serialize as a fresh binary stream (1 xref, 0 /Prev pointers, 1 %%EOF)
  const cleanBytes = await cleanDoc.save({
    useObjectStreams: true,
    addDefaultPage: false,
  });

  return {
    cleanBytes,
    removedRevisions: Math.max(0, revisionsBefore.length - 1),
    bytesSaved: inputPdfBytes.length - cleanBytes.length,
  };
}

If you want to inspect how many %%EOF revisions a file contains and strip appended update layers directly in your browser without uploading sensitive files to a remote server, you can test it with the client-side PDF Incremental Update Cleaner, which parses the cross-reference chain locally in memory and rebuilds a single-generation PDF.

---

6. Key Edge Cases: When You Should Not Flatten Incremental Updates

While flattening incremental updates is critical before publishing redacted legal filings, financial disclosures, or sanitized engineering schematics, there is one scenario where incremental updates must be preserved: Cryptographic Digital Signatures (/ByteRange).

In a digitally signed PDF:

• The first signer signs a specific /ByteRange [0 offsetA offsetB lengthB] covering the entire Revision 1 byte stream except the PKCS#7 /Contents hex placeholder.
• If a second person countersigns the document, or if a timestamp authority appends a Document Security Store (/DSS) for Long-Term Validation (LTV), that addition must be written as an incremental update after Revision 1's %%EOF.
• If you flatten or garbage-collect a signed PDF, the byte offsets shift, invalidating the cryptographic hash over /ByteRange.

Therefore, a production sanitization pipeline should always check the /AcroForm dictionary for /SigFlags or /FT /Sig fields before flattening, warning the user if active cryptographic signatures are present, and proceeding with full object-graph flattening whenever the goal is permanent data sanitization.

---

Summary Checklist for Engineers Handling User-Submitted PDFs

1. Never trust visual redaction alone: Drawing a rectangle over text or deleting a page in a standard PDF editor often leaves the original object stream intact prior to an earlier %%EOF.
2. Count %%EOF markers on ingestion: Any PDF with more than one startxref ... %%EOF sequence contains historical state deltas.
3. Rebuild the object graph: Only by copying reachable objects from the latest /Root catalog into a newly allocated cross-reference table can you guarantee that shadowed objects and unreferenced streams are completely eliminated from the binary output.