DEV Community

Praveen Pasco
Praveen Pasco

Posted on

python-docx gave me a sentence that was never in the document

A paragraph came back from python-docx as this:

'The sample size was  participants.'
Enter fullscreen mode Exit fullscreen mode

Two spaces where a number should be. The document itself was fine. Someone had changed "forty" to "forty-two" with Track Changes on, and nobody had accepted the change yet.

So python-docx didn't give me the old sentence, and it didn't give me the new one. It gave me a third version that never existed.

I wanted to know exactly what was going on before trusting any text I pull out of a .docx again. So I built the smallest file that reproduces it, where I know the right answer in advance.

The fixture

A tracked change in WordprocessingML is just markup inside the paragraph. A deletion wraps the old run in <w:del> and stores its text in <w:delText> instead of <w:t>. An insertion wraps the new run in <w:ins>. A moved sentence shows up twice: once inside <w:moveTo> where it landed, once inside <w:moveFrom> where it used to be.

This script writes one plain paragraph, one with a changed number, and one with a sentence moved to the front. The move is the same shape as the example in Microsoft's docs for moveFrom.

from docx import Document
from docx.oxml import parse_xml

NS = 'xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"'

def run(text):
    return f'<w:r {NS}><w:t xml:space="preserve">{text}</w:t></w:r>'

def tracked(tag, rid, inner):
    return (f'<w:{tag} {NS} w:id="{rid}" w:author="Reviewer" '
            f'w:date="2026-10-06T10:00:00Z">{inner}</w:{tag}>')

doc = Document()
doc.add_paragraph("A plain paragraph with no revisions.")

# a number changed with Track Changes on
p = doc.add_paragraph()._p
p.append(parse_xml(run("The sample size was ")))
p.append(parse_xml(tracked("del", 1, "<w:r><w:delText>forty</w:delText></w:r>")))
p.append(parse_xml(tracked("ins", 2, "<w:r><w:t>forty-two</w:t></w:r>")))
p.append(parse_xml(run(" participants.")))

# a sentence moved to the front (same shape as the spec's moveFrom example)
p = doc.add_paragraph()._p
p.append(parse_xml(tracked("moveTo", 3, '<w:r><w:t xml:space="preserve">Some moved text. </w:t></w:r>')))
p.append(parse_xml(run("Some text. ")))
p.append(parse_xml(tracked("moveFrom", 4, "<w:r><w:t>Some moved text.</w:t></w:r>")))

doc.save("tracked.docx")
Enter fullscreen mode Exit fullscreen mode

Fair warning: I made this file by hand. Word writes a lot more attributes (rsids, run properties) and I haven't opened this exact file in Word. The element names are the ones from the spec, and that's all the extractors below care about.

Three ways to read it

python-docx 1.2.0:

from docx import Document

for p in Document("tracked.docx").paragraphs:
    print(repr(p.text))
Enter fullscreen mode Exit fullscreen mode
'A plain paragraph with no revisions.'
'The sample size was  participants.'
'Some text. '
Enter fullscreen mode Exit fullscreen mode

Both halves of the number edit are gone, and so is the moved sentence. Paragraph.text only looks at runs that are direct children of <w:p>. Everything wrapped in <w:ins>, <w:del>, <w:moveTo> or <w:moveFrom> is one level down, so it's skipped. This is a known issue. #1123 ("paragraph.text is missing text") has been open since July 2022, and #340 asked for revision support back in 2016.

The quick-and-dirty fix one commenter suggests in that thread is to skip python-docx and grab every <w:t> from the XML. Deleted text lives in <w:delText>, so it drops out:

unzip -p tracked.docx word/document.xml | grep -o '<w:t[ >][^<]*' | sed 's/<w:t[^>]*>//'
Enter fullscreen mode Exit fullscreen mode
A plain paragraph with no revisions.
The sample size was 
forty-two
 participants.
Some moved text. 
Some text. 
Some moved text.
Enter fullscreen mode Exit fullscreen mode

That fixes the number. But the moved sentence now appears twice, because the copy inside <w:moveFrom> uses an ordinary <w:t> too, not <w:delText>. Whether that's what you want depends on what you're doing with the text. For a word count or a diff, it isn't.

Asking for a specific view

What I actually want is to say which version I mean: the text as if every change were accepted, or as if every change were rejected. That's a short walk over the paragraph with a list of containers to skip:

import sys
import zipfile
import xml.etree.ElementTree as ET

W = "{http://schemas.openxmlformats.org/wordprocessingml/2006/main}"
HIDE = {"accepted": {W + "del", W + "moveFrom"},
        "original": {W + "ins", W + "moveTo"}}

def walk(el, view, out):
    for child in el:
        if child.tag in HIDE[view]:
            continue
        if child.tag in (W + "t", W + "delText"):
            out.append(child.text or "")
        elif child.tag == W + "tab":
            out.append("\t")
        elif child.tag in (W + "br", W + "cr"):
            out.append("\n")
        elif child.tag != W + "pPr":
            walk(child, view, out)

def paragraphs(path, view):
    with zipfile.ZipFile(path) as z:
        root = ET.fromstring(z.read("word/document.xml"))
    for p in root.iter(W + "p"):
        out = []
        walk(p, view, out)
        yield "".join(out)

if __name__ == "__main__":
    view = sys.argv[2] if len(sys.argv) > 2 else "accepted"
    for text in paragraphs(sys.argv[1], view):
        print(repr(text))
Enter fullscreen mode Exit fullscreen mode
$ python3 revtext.py tracked.docx
'A plain paragraph with no revisions.'
'The sample size was forty-two participants.'
'Some moved text. Some text. '

$ python3 revtext.py tracked.docx original
'A plain paragraph with no revisions.'
'The sample size was forty participants.'
'Some text. Some moved text.'
Enter fullscreen mode Exit fullscreen mode

Both match what I put in. Standard library only. It recurses into anything that isn't a hidden container or the paragraph properties, so text inside hyperlinks and content controls comes along too.

The check I run first now

Most of the time I don't need either view. I just need to know that a file has unresolved changes before I treat its text as final:

import sys
import zipfile
from collections import Counter
import xml.etree.ElementTree as ET

W = "{http://schemas.openxmlformats.org/wordprocessingml/2006/main}"
KINDS = ("ins", "del", "moveFrom", "moveTo")

with zipfile.ZipFile(sys.argv[1]) as z:
    root = ET.fromstring(z.read("word/document.xml"))

seen = Counter()
for kind in KINDS:
    for el in root.iter(W + kind):
        seen[(kind, el.get(W + "author"))] += 1

for (kind, author), n in sorted(seen.items()):
    print(f"{kind:9} {n:3}  {author}")
sys.exit(1 if seen else 0)
Enter fullscreen mode Exit fullscreen mode
$ python3 has_revisions.py tracked.docx; echo "exit=$?"
del         1  Reviewer
ins         1  Reviewer
moveFrom    1  Reviewer
moveTo      1  Reviewer
exit=1
Enter fullscreen mode Exit fullscreen mode

On a fresh Document() with one paragraph and no revisions it prints nothing and exits 0, so it can sit in a pipeline and fail the step.

What this doesn't handle

I'd rather list these than have someone find them the hard way:

  1. Deleted paragraph marks. If a whole paragraph break was deleted, Word records it as <w:del> inside the paragraph's own properties (w:pPr/w:rPr). In the accepted view those two paragraphs should become one. My script skips w:pPr entirely, so it keeps them separate.
  2. Text boxes. I haven't tested these. Reading my own code, a paragraph inside a text box would be printed twice: once as part of the paragraph whose run holds the drawing, and once on its own, because root.iter() finds it too.
  3. Only word/document.xml. Footnotes, endnotes, headers, footers and comments live in their own parts and can carry tracked changes of their own.
  4. Formatting changes. w:rPrChange and w:pPrChange are revisions too. They don't change the text, so neither script reports them.
  5. The has_revisions count includes paragraph-mark insertions. Those are w:ins elements as well, so a count of 3 doesn't necessarily mean three edits you can see.
  6. Deleted table rows. A deleted row gets an empty <w:del/> in its row properties (w:trPr), and the spec says that marker says nothing about the cells' contents, which are marked separately. My script never looks at row properties, so if the cell text isn't wrapped in its own w:del, the accepted view still prints it.

The first one is the gap I actually care about, and I haven't written it yet. If anyone has a clean way to merge paragraphs across a deleted mark without rebuilding the tree, I'd like to see it.

Top comments (0)