Why My PST Reader Showed 24 of 160 Items in an Outlook OST (and Raised No Error)

A debugging story about row matrices, an 8 KB assumption, and a loop that stopped without telling anyone.

---

Here is the kind of bug I like least: nothing crashes, nothing logs, and the output looks plausible. A folder simply has fewer items than it should.

I work on a browser-based viewer for Outlook mail files. It reads .pst and .ost files locally with a JavaScript library called pst-extractor and a good deal of my own code around it. While testing with an Outlook 2013+ OST (the offline cache Outlook keeps for an Exchange or Microsoft 365 mailbox), I opened a calendar folder named "United States holidays". The folder tree said 160 items. The list showed 24. No exception, no warning.

Here is what I found and how I fixed it. If you ever parse a PST or OST, one of these details may save you an afternoon.

Two layouts that share a name

Outlook's personal-folder files come in more than one layout. The old ANSI format, the Unicode PST, and the Outlook 2013+ files that use 4 KB pages (version number 36 in the header, which is what my test file reports) are all "PST-family" files, but they don't store data blocks the same way.

Folder contents in these files are stored in a Table Context, which is, roughly, a table with a "row matrix": fixed-size rows laid out one after another. Each row starts with a 4-byte row ID. A separate row index maps row IDs to row numbers.

To read row r, a parser needs the byte offset of that row. The library I use computes it the way the Unicode PST layout dictates: data blocks of 8,176 bytes, with padding at the end of each block, so

offset(r) = floor(r / rowsPerBlock) × 8176 + (r mod rowsPerBlock) × rowSize

That is correct for Unicode PSTs. It is wrong for a 4K OST.

What the 4K OST actually contains

In my test file, the contents table of the 160-item calendar folder is a single deflate-compressed block of 54,240 bytes. That is exactly 160 rows × 339 bytes per row, with no padding. The data blocks in this layout are larger than 8 KB.

Now apply the 8 KB arithmetic. Row 24 should start at byte 8,136 (24 × 339), but the formula treats a block as ending at 8,176 and reads row 24 from byte 8,176 instead. It finds whatever bytes are there and reads a garbage row ID.

The folder cursor (the code that pages through items) treats a row ID that doesn't make sense as the end of the table. It stops. No error. The library reported rowCount as 159, and the list showed 24.

That's the whole bug: one layout assumption, applied to a file format that breaks it, and a loop that fails quietly instead of loudly.

The fix: stop computing offsets, find them

I didn't want to rewrite the whole reader. The fix patches the table class for the 4K layout only, and leaves Unicode and ANSI files on the library's original code path.

The idea: don't compute where row r is, locate it, and verify.

1. Row 0 starts at offset 0.
2. Row r starts where row r-1 ended.
3. Check it: read the 4-byte row ID at that offset, look it up in the table's row index, and confirm that it maps to row number r.
4. If it doesn't match, look further, within the next rowSize - 1 bytes. This covers the case where a row matrix spans several blocks and the writer left padding at the end of a block (the libpff documentation describes that padding).
5. If the row still can't be found, end the table rather than return a garbage row.

In code:

export function locateRows(
  readRowId: (offset: number) => number,
  length: number,
  rowSize: number,
  rowIndexById: ReadonlyMap<number, number>,
): number[] {
  const offsets: number[] = [];
  if (rowSize <= 0) return offsets;
  let at = 0;
  for (let row = 0; row < rowIndexById.size; row++) {
    let found = -1;
    for (let p = at; p < at + rowSize && p + rowSize <= length; p++) {
      if (rowIndexById.get(readRowId(p)) === row) { found = p; break; }
    }
    if (found < 0) break;
    offsets.push(found);
    at = found + rowSize;
  }
  return offsets;
}

The check against the row index is the important part. It turns "I assume this offset is right" into "this offset is right, because the row at it says it is row r." A wrong guess ends the table instead of producing wrong data.

Reusing the library's own row parsing

Locating rows is half of it. Decoding a row is still the library's job, and I wanted to keep it. The library reads row r at r × rowSize from the start of a node. So for each row, I hand it a node shifted by the difference between where the row really is and where the library will look:

const shift = offsets[row] - row * t.TCI_bm;
t.rowNodeInfo = new NodeInfo(node.startOffset + shift, node.endOffset + shift, node.pstNodeInputStream);
rows.push(originalGetItems.call(this, row, 1)[0]);

For the duration of the call the library's block size is set to a huge value, so its own padding step never kicks in, and the original node is restored in a finally. The rowCount getter is overridden the same way for 4K files. It is a monkey patch, but a contained one: it only applies when the file's type is the 4K one.

The "end of table" signal

The library's folder cursor stops when it reads a row carrying a particular sentinel. When my locator has no more rows, the patched getItems returns exactly that sentinel (a fake row with item index −1), so the cursor ends cleanly instead of reading past what was located.

Testing, and what I could not test

The regression test uses the library's own test OST (MIT-licensed test data, gzipped in my fixtures). It asserts:

• the header is the 4K layout (!BDN magic, version 36);
• the "United States holidays" folder lists all 160 items, and row 24, the first one the 8 KB arithmetic misplaced, matches the full listing;
• paging works at the end of the table (asking for 50 items from row 150 returns 10);
• every message folder in that file lists the count the folder tree shows, and the result is the same through a paged reader with a small cache;
• a regular Unicode PST still lists the same 71 messages as before, so the patch doesn't touch it.

One limit I want to be upfront about: I did not have a real 4K OST whose row matrix spans multiple blocks. The "look past the block's tail padding" path is covered only by tests on synthetic data, where I build matrices with padding between blocks and check that the offsets come out right. The real-file case I verified is the single-block one. If you have a multi-block 4K sample, that's the case I'd most like to see.

Takeaways

• A "file format" often means several layouts. Check which one your library's arithmetic was written for.
• Silent truncation is worse than a crash. If a loop ends early, cross-check it against a count you already have (here, the folder's 160 versus the 24 listed).
• Prefer locate and verify over compute and trust when a format has self-describing indexes.
• Keep a patch narrowly scoped (one layout), and leave the working path alone.

If you want to try it on your own file, the tool's OST viewer page is where this lives. Disclosure: I built it. It reads the file in your browser without uploading it.