Borys Haiduchyk

Posted on Nov 9 Originally published at atomara.io

I tested ChatGPT's PDF Content Credentials: what survives a re-save, a print and an edit

#pdf #security #ai #python

Disclosure: I'm the founder of Atomara, a PDF forensics API. The files and the code are public, so you can check every result below yourself.

I asked ChatGPT to change the date on an invoice it had generated. It did, and the new file still carries OpenAI's signed statement that ChatGPT created it: the same manifest, byte for byte, describing the version with the old date.

That invoice was one of ten invented documents I generated to measure what this mark tells you and what is left of it after a re-save, a print or a screenshot. OpenAI publicly documents provenance signals for images and audio (C2PA and SynthID for images, SynthID for audio), but its public verification API does not accept PDFs. All ten files come from one account over two days, so they are specimens, not a survey of ChatGPT's behaviour.

The result

The bank statement from the set, one transformation per copy, read three ways:

#	Step	Manifest attached	`grep ChatGPT`	c2pa-python 0.38	Decoded-stream reader	Text layer
0	Original from ChatGPT	yes	3 hits	Valid, hash matches	ChatGPT, gpt-6-thinking	kept
1	`qpdf in.pdf out.pdf`	yes	0 hits	error: invalid JUMBF header	ChatGPT, gpt-6-thinking	kept
2	`pikepdf.open(...).save(...)`	yes	0 hits	error: invalid JUMBF header	ChatGPT, gpt-6-thinking	kept
3	Quartz `CGPDFContext` (Print → Save as PDF)	no	0	no manifest	nothing	kept
4	PDFKit `PDFDocument.write` (Preview)	no	0	no manifest	nothing	kept
5	Rasterised at 144 dpi, wrapped in a PDF	no	0	no manifest	nothing	none

The other nine files matched after each transformation. Rows 3 and 4 call the macOS system APIs directly, the closest scripted stand-ins for the two apps.

What the mark is, in PDF terms

The C2PA spec stores the manifest as an embedded file stream. Its file specification carries /AFRelationship /C2PA_Manifest and the catalog's /AF entry points at it. The payload is JUMBF boxes holding CBOR: the claim, its assertions and a COSE signature. c2pa-python decodes it to:

"claim_generator_info": [{ "name": "ChatGPT", "specVersion": "2.2.0" }],
"assertions": [{ "label": "c2pa.actions.v2", "data": { "actions": [{
    "action": "c2pa.created",
    "when": "2026-11-08T20:24:44.254246Z",
    "softwareAgent": { "name": "gpt-6-thinking", "version": "gpt-6-thinking" },
    "digitalSourceType": "http://cv.iptc.org/newscodes/digitalsourcetype/trainedAlgorithmicMedia"
}]}}],
"signature_info": { "alg": "Ps256", "issuer": "OpenAI OpCo, LLC", "common_name": "OpenAI Media Service" }

This is not a /Creator string anyone can type; it is a signed statement by the tool. The signature makes an altered claim detectable, but a signed manifest can be copied into another PDF without becoming valid for that document. The signature shows who made the claim; only the hash ties it to a file.

An untouched file validates

validation_state: Valid
  claimSignature.validated        claim signature valid
  timeStamp.validated             timestamp message digest matched: OpenAI TSA Leaf
  assertion.dataHash.match        data hash valid
  signingCredential.untrusted     signing certificate untrusted

assertion.dataHash.match is the line that matters: these are the bytes the manifest describes. The last line is there because I loaded no trust list. A valid signature and a matching hash do not by themselves establish a trusted signer: trust requires a recognized certificate chain and a trust policy.

A file in the public veraPDF test corpus (PDF_UA-1/7.5 Tables/7.5-t01-fail-c.pdf) also carries a genuine OpenAI manifest but comes back Invalid with assertion.dataHash.mismatch. Its Producer is the corpus's own builder and its ModDate is fourteen minutes after the signature: the file was rewritten around the manifest.

Finding the manifest says a tool once made a document. Validating it says whether the tool made this one.

Two PDF revisions, and no visible metadata

In these files ChatGPT built the PDF with ReportLab, then appended the manifest as an incremental update. In the statement that is 24,970 bytes after the first %%EOF, with a new catalog.

A revision counter sees two revisions on a file nobody edited; here the second one is the signing. And the appended trailer has no /Info entry, so pdfinfo and pikepdf report no metadata at all, although the first revision's Info is still in the bytes. Cut the file at the first %%EOF and it comes back.

That Info rarely helps anyway. The statement happened to name ChatGPT there, which is why grep finds 3 hits on it. In most of the other files the Creator is "anonymous" or "ReportLab", so the manifest is the one thing that names the tool.

Ten files, one shape

Two versions of a bank statement, an invoice, a till receipt, a payslip, a resume, a letter, a four-page report and an image on a PDF page: these nine untouched files all have two revisions and an empty newest Info, and all validate with a matching hash. Eight name gpt-6-thinking; the payslip, made with a different model, names gpt-5-6-thinking.

The image took two steps: ChatGPT returned a PNG, then a PDF of it. The PNG's own manifest (gpt-image) did not travel into the PDF; ChatGPT signed the PDF separately as the chat model.

Why the rewrite hides the mark

qpdf and pikepdf keep attachments but apply /FlateDecode to streams that were stored raw. Once decoded, the manifest is the same 24,294 bytes, but the raw file no longer contains "ChatGPT". c2pa-python 0.38 failed on these files with "invalid JUMBF header": the c2pa-rs PDF handler follows /AF to the file spec and reads the stream's stored content without applying the filter.

In my tests one qpdf pass hid the mark from a byte scanner and from c2pa-python without removing it. The fix is to decode the stream through a PDF parser before you look.

This is separate from a problem the PDF Association has described in ChatGPT PDFs, a stray /F entry in the embedded stream that makes the credentials unusable. My ten files do not have that entry.

Rasterising removes the PDF's text layer, fonts, attachments and manifest. The resulting image may carry its own metadata or provenance signals, but those are separate from the PDF's credentials.

When ChatGPT edits its own PDF

I uploaded the invoice back into a chat and asked for the date to be changed to 15 October 2026. The page now says 15 Oct 2026 where the original said 09 Nov 2026.

The manifest in the edited file is byte for byte the original's: same SHA-256, same urn:c2pa:…64de0209fc3c. No new claim, no new signature. The file itself was rewritten (one %%EOF, compressed streams, no ReportLab metadata) and the old attachment came along.

So it still says "created by ChatGPT" under a genuine OpenAI signature, about bytes that no longer exist. A tool that only looks for the mark sees what it saw on the original, and c2pa-python can't open the file at all, for the compressed-stream reason above. Only comparing the manifest's hash with the file's bytes shows that the document changed after signing. One edit, one file: this is what ChatGPT did here, not necessarily what it always does.

What this means for a document check

On an issued document, the mark is a red flag. This is the approach we take at Atomara. Bank statements, receipts, payslips and diplomas come from the issuer's own systems, not from a chat assistant, so we treat one that carries an AI mark as not issued by the institution it names until that institution confirms otherwise. On a letter or a resume, the mark only tells you which tool was used. We regularly see such documents among the files sent to us for checking, and those are only the ones that still carry the mark.
Validate, then trust. Act on validation_state, with a trust list loaded. A failed hash alone does not say whether the document was rewritten or the manifest transplanted.
Absence proves nothing. A print or a screenshot removed the mark in every test, with no intent required. When you receive a picture, ask for the original PDF.
Check the numbers too. In the GPT4o-Receipt study of 1,235 receipts the main tell was arithmetic: line items that don't sum, tax that doesn't match. That check needs no file structure and survives every row above.

Where Atomara stands on this

Finding the mark is a forensic signal, not confirmed attribution. Our check reads the same marks and decodes streams, so on these files it reports ChatGPT in rows 0 to 2 and nothing in rows 3 to 5. It does not verify the signature or the hash: a transplanted manifest and the edited invoice read the same as the original. A file without a mark gets no finding from this check. Details and limits: atomara.io/ai-document-fraud.

Reproduce it

The ten files, their checksums and a fuller reader are in atomara-io/chatgpt-pdf-provenance. You need pip install pikepdf c2pa-python, plus qpdf and Poppler's pdftoppm.

# what a PDF says about where it came from
import sys, pikepdf

path = sys.argv[1]
with pikepdf.open(path) as pdf:
    for name, spec in pdf.attachments.items():
        data = spec.get_file().read_bytes()     # decoded: the step grep skips
        print(f"{name!r}: {len(data)} bytes, ChatGPT in manifest: {b'ChatGPT' in data}")

raw = open(path, "rb").read()
print("raw bytes contain ChatGPT:", raw.count(b"ChatGPT"))

try:
    import c2pa
    with c2pa.Reader(path) as reader:
        print(reader.json()[:800])
except Exception as e:
    print("c2pa-python:", e)

The transformations:

qpdf original.pdf 01-qpdf.pdf
python -c "import pikepdf; pikepdf.open('original.pdf').save('02-pikepdf.pdf')"
pdftoppm -r 144 -png original.pdf 05-raster

# 03: the Print dialog's path (Quartz re-render); 04: Preview's path (PDFKit)
# pip install pyobjc-framework-Quartz
import Quartz
from Foundation import NSURL

src = NSURL.fileURLWithPath_("original.pdf")
doc = Quartz.CGPDFDocumentCreateWithURL(src)
ctx = Quartz.CGPDFContextCreateWithURL(NSURL.fileURLWithPath_("03-print.pdf"), None, None)
for i in range(1, Quartz.CGPDFDocumentGetNumberOfPages(doc) + 1):
    page = Quartz.CGPDFDocumentGetPage(doc, i)
    Quartz.CGContextBeginPage(ctx, Quartz.CGPDFPageGetBoxRect(page, Quartz.kCGPDFMediaBox))
    Quartz.CGContextDrawPDFPage(ctx, page)
    Quartz.CGContextEndPage(ctx)
Quartz.CGPDFContextClose(ctx)
Quartz.PDFDocument.alloc().initWithURL_(src).writeToFile_("04-preview.pdf")

If you have a ChatGPT-made PDF of your own, run the script and tell me whether it validates and has the same two-revision shape. More PDFs edited inside ChatGPT would help most, since mine is a sample of one.

DEV Community