A DOCX is a ZIP. So when a tool hands one back to you, you do not have to squint at the rendered page and hope. You can unzip both copies and hash every member.
I did that on 31 August with a file where exactly one paragraph was supposed to change. Twenty parts in, twenty parts out, identical part-name sets. Eighteen of the twenty hashes matched. Two did not.
The first one was word/document.xml, which is where the body lives. Expected.
The second was [Content_Types].xml. Same 2,125 bytes going in and coming out. Different SHA-256.
What a same-size, different-hash part turns out to be
Seventeen content-type declarations before, seventeen after, and the multiset of entries identical. The only difference: the existing /word/comments.xml override had been sitting at the end of the list and came back grouped in with the other Word overrides.
Nothing added. Nothing removed. Nothing renamed. Reordered.
Which is completely harmless here, and is also exactly the class of change a naive comparison reports as "no difference". If your check parses the XML and compares the set of entries, you get equality. If it compares part names only — the thing most people reach for first, because zipfile.namelist() is right there — you never even look inside. It took a byte-level hash of every member to see it at all.
That is the transferable bit, and it is why I am writing this up rather than the headline result: a part can keep its filename and its byte count and still not be the same part.
The setup, because the result is worthless without it
Synthetic two-page file, built for this. Page 1 carried the things that are expensive to repair if a tool eats them; page 2 carried one ordinary 133-word paragraph, which was the only text selected for rewriting.
| Object on page 1 | Count in | Count out |
|---|---|---|
| Tracked insertion | 1 | 1 |
| Tracked deletion | 1 | 1 |
| Comment body | 1 | 1 |
| Comment range start / end | 1 / 1 | 1 / 1 |
| Comment reference | 1 | 1 |
| Comment relationship / content-type entry | 1 / 1 | 1 / 1 |
| Tagged content control | 1 | 1 |
| Bookmark start / end | 1 / 1 | 1 / 1 |
| REF field start | 1 | 1 |
| Table | 1 | 1 |
| Package parts | 20 | 20 |
I also seeded seven text sentinels, things like TRACK-ANCHOR-23, COMMENT-ANCHOR-47, 37.5% and 12.40 mg, so that silent duplication or deletion of protected text would show up as a count change rather than as something I had to spot by reading. All seven kept their input counts. BOOKMARK-TARGET-59 appeared twice both times, which is correct: once at the bookmark, once as the displayed REF result.
Inside word/document.xml, the formatted diff came back with exactly one changed text node, at /document/body[0]/p[19]/r[0]/t[0]. That is the paragraph that was supposed to change. No other formatted node in that part moved.
Then I rendered both files and compared pages. Two pages before, two pages after, one changed page: page 2. Page 1, which carried the tracked changes, comment anchor, content control, bookmark, REF field and table, rendered identically. Page 2 reflowed, because the new sentence lengths were different, and reflow is not clipping.
Run facts, since a benchmark with no configuration recorded is a story: 20:00 China Standard Time on 31 August 2026, the Balanced v3 strategy, specified-content mode, scope confirmed as one paragraph and 133 English words before starting, 32 seconds, 14 credits, balance 593 → 579. Input 40,191 bytes, output 40,222. Those are records of one job, not a spec.
The control, which is the part most write-ups skip
An unchanged count only means something if the counter can produce a changed one. So I broke two copies on purpose.
Copy one: stripped the comment structures out of the package. The same auditor then reported zero comment bodies, zero range anchors, zero comment references, zero comment relationships, zero comment content-type entries.
Copy two: accepted the tracked changes. The auditor reported zero tracked insertions and zero tracked deletions.
Without that step, a clean row of matching counts is indistinguishable from a measurement that never saw the object. It is the single cheapest thing you can add to a test like this and it is almost never there.
It also does not certify the auditor against faults I did not create. A broken drawing relationship, a malformed equation, a damaged macro. Each of those needs its own probe and its own known-bad copy.
What this does not establish
A list, because the boundary is the result:
- Every instrumented object sat outside the rewritten paragraph. This tests preservation outside a confirmed scope. It says nothing about a comment anchor or tracked change living inside the text being rewritten.
- No manual Microsoft Word open-save-close-reopen. That was blocked on the day and it is the largest gap here. Matching XML shows what is present in the package; it does not show that Word opens the file without a repair prompt, or that the comment is clickable in the Comments pane, or that the REF field refreshes.
- Not tested: live Zotero/EndNote/Mendeley libraries, footnotes, equations, macros, embedded files, text boxes, floating images, section-specific headers and footers, a different strategy, a whole-document rewrite, another Word version or OS.
- No detector was involved at any point. This was a structural question, not a scoring one.
- One file. One run.
If you want to run this against whatever tool you use
The method is tool-agnostic and takes about an hour:
- Build a disposable file containing one example of each object that would be expensive to repair in your real document. Not a generic paragraph. A generic paragraph has no Word structure in it to lose.
- Put a unique text anchor beside each object and write down the expected count.
- Keep all of it outside one ordinary paragraph. Select only that paragraph. Confirm the scope before the run.
-
shasum -a 256the untouched input and record the byte size. - After download, list the parts, then hash every part, then parse the relevant XML for object counts. Do not grep the compressed bytes and call it an audit. That is a search over DEFLATE output.
- Render both and compare page count, extracted text and images. Reflow is not clipping.
- Open the output in the desktop Word version you will actually deliver from. Inspect All Markup, open the Comments pane, exercise a content control, refresh a field, save-as, close, reopen.
- Make one deliberately broken copy and confirm your check reports the loss.
And do not roll the rows up into a percentage. If the comment fails and nine other rows pass, you have a failed requirement, not a 90%.
Disclosure: the file above went through HumanPen, which I work on, so this is a vendor testing its own product. HumanPen rewrites only the scope you confirm and is designed to preserve the structures around it; nothing in one run makes that a guarantee for your document. The full write-up with every hash and part listed is at humanpen.net/blog/ai-humanizer-track-changes-comments-docx-test.
Your DOCX still needs its own acceptance test. That is the actual conclusion.
Top comments (0)