Can your RAG answer point back to the right PDF page?

#pdf #rag #python

A correct-looking answer is difficult to review if its citation opens the wrong PDF page. This can happen when several pages share the same heading, or when a pipeline discards page metadata before chunking.

Here is a small check to run before adding a vector database. It uses a two-page teaching PDF and an actual ComPDF parsing result. It checks source mapping; it does not benchmark retrieval or prove that an LLM answers correctly.

Make the two pages deliberately similar

Use a heading of “Return policy” on both pages. Page 1 says “Domestic returns” and “Request a return within 14 days of delivery.” Page 2 says “International returns” and “Request a return within 30 days of delivery.” Export this as a text PDF.

The shared heading matters: it cannot identify the right source on its own. You need the document version and page attached to the relevant text.

Inspect the parser output before flattening it

The ComPDF document parsing API returns structured document content. For this sample, the result archive contained a JSON file with two entries in result.pages. Each page had page_id and a structured list of blocks.

The actual result preserved the 14-day sentence on page_id 1 and the 30-day sentence on page_id 2. It also preserved the repeated heading. Those observations apply to this sample; they are not an accuracy guarantee for other files.

To repeat the request, keep your API key on the server:

curl --fail-with-body \
  'https://api-server.compdf.com/server/v2/process/idp/documentParsing' \
  --header "x-api-key: $COMPDF_API_KEY" \
  --form 'file=@returns-policy.pdf' \
  --output response.json

The response is JSON, not the parsed file. Check the business code and task status, then download the returned result URL and extract result_json.json. Do not treat HTTP 200 as proof of successful parsing. Use the current API documentation for the response wrapper and processing status your account receives.

Figure: values reproduced from the actual JSON output, not a product UI screenshot.

Attach source identity while you still have it

This Python example uses only the standard library. Place the original PDF and extracted JSON beside the script. ComPDF provides the page and block text here; the hash and chunk IDs below are application metadata.

import hashlib
import json
from pathlib import Path

source = Path("returns-policy.pdf")
source_hash = hashlib.sha256(source.read_bytes()).hexdigest()
document = json.loads(Path("result_json.json").read_text(encoding="utf-8"))
chunks = []

for page in document["result"]["pages"]:
    for index, block in enumerate(page["structured"]):
        if block["type"] != "text":
            continue
        chunks.append({
            "source_sha256": source_hash,
            "page_id": page["page_id"],
            "chunk_id": f'{source_hash[:12]}:p{page["page_id"]}:b{index}',
            "text": block["text"],
        })

Path("chunks.json").write_text(json.dumps(chunks, indent=2), encoding="utf-8")
for chunk in chunks:
    if "within" in chunk["text"]:
        print(chunk["page_id"], chunk["text"])

Actual output for the two-page sample:

1 Request a return within 14 days of delivery.
2 Request a return within 30 days of delivery.

The block filter is intentionally narrow. This example checks text sentences, not table contents or all block types. If your task needs tables, inspect and process their structures explicitly rather than assuming they have become ordinary text chunks.

What a useful citation needs to retain

Store the full source hash alongside each chunk, even though the example uses a shortened prefix in its readable ID. Keep an internal mapping from that hash to an immutable original file. The hash identifies file bytes, not access rights, and the chunk ID is only stable for the same extraction and block order. Re-parsing can change that order.

When splitting long blocks, copy their source metadata to every child chunk. If you merge blocks across pages, store a page range or a list of sources; assigning the merged text to just the first page creates a misleading citation.

The page_id in this result matched physical PDF pages 1 and 2. A viewer may expect a zero-based index, and printed page labels can differ from physical pages. Test that conversion in the viewer instead of subtracting one everywhere without checking.

Test failure cases before trusting a citation

First, assert that the domestic sentence belongs to page 1 and the international sentence to page 2. Then deliberately swap the page metadata and make sure the check fails. Also replace the PDF with a changed version and confirm that your source mapping refuses to reuse the old hash.

These are application checks. A parser returning page metadata does not automatically implement citation rendering, authorization, or version management for your system.

If you later add retrieval, inspect the returned chunk metadata before asking an LLM to write an answer. A correct sentence with a missing source should remain a failed citation check. A clickable citation should open the authorized file version and the page containing the quoted sentence.

The document parsing workflow for RAG is a useful next step when evaluating the input stage. Start with your own permitted PDFs: repeated headings, scanned pages, missing fields, and revised versions. Verify what survives parsing, then decide how to carry it through chunking and retrieval. A working citation is a property of the whole pipeline, not just of the text extractor.

DEV Community