Why Disconnected Captions and Footnotes Break Technical RAG (and How to Bind Spatial Annotations)

This article is part of the **PDF Parsing for RAG* series. In Part 1, we resolved multi-column reading order. In Part 2, we reconstructed borderless tables. In Part 3, we serialized mathematical formulas into LaTeX blocks. Today we address floating figures, captions, and orphaned footnotes.*

When ingesting technical whitepapers and research reports into retrieval-augmented generation (RAG) pipelines, queries targeting visual evidence or methodology caveats fail reliably. A user asks: "What dataset was used for the ablation study in Figure 4?" Vector retrieval either returns an unrelated section from the appendix or surfaces a fragmented snippet lacking the figure's context.

The error originates in how PDF renderers place floating objects. Unlike HTML, where figures exist within a structured DOM tree, PDF drawing instructions position figures, captions, and footnotes at arbitrary coordinates on the canvas. Naive extractors read these elements in stream order, slicing captions into the middle of running paragraphs and dropping footnotes into random chunk windows.

The Physical Layout of Floating Annotations

In academic and technical publishing, figures and tables rarely appear directly after their first text mention. Instead, layout engines push visual blocks to the top or bottom of the page or into an adjacent column to maximize whitespace efficiency.

This separation introduces two distinct structural hazards for text extractors:

1. Floating Caption Interleaving: A caption starting with Figure 2: Architecture overview is placed physically below a diagram. Because naive extractors sort blocks purely by vertical coordinates, the caption text is inserted directly between two halves of a main-body sentence that runs alongside the diagram.
2. Footnote Disconnection: Footnotes sit at the bottom margin, separated from the body by a thin vector rule. When text splitters process the page, the footnote marker (such as a superscript 3) lands in Chunk 1, while the corresponding definition sits 400 tokens away in Chunk 2, stripped of its referent.

Three Failure Modes in RAG Ingestion

When spatial annotations are flattened without bounding box resolution, downstream retrieval systems suffer three repeatable failures:

1. Mid-Sentence Injection

When a floating caption is merged into surrounding text, the embedding model processes a garbled sentence:

The transformer backbone processes input tokens Figure 3: Training loss across 50 epochs using AdamW optimizer with a linear warm-up schedule.

The semantic representation of both the main-body sentence and the figure description is degraded, lowering similarity scores for both topics.

2. Orphaned Footnote Disclaimers

Legal disclaimers, dataset licenses, and parameter details frequently live in footnotes. When parsed as independent text lines at the bottom of a page, an embedding model receives isolated fragments like:

*Evaluated on an internal benchmark of 1,200 synthetic prompts; not audited for production SLAs.

Because this chunk lacks the product name or the metric name defined 30 lines above, vector search never retrieves the qualification when answering risk or compliance queries.

3. Visual Hallucinations in Multi-Modal RAG

When combining text chunks with vision-language models (VLMs), passing an image without its bound caption forces the model to guess the axes, legends, or experiment names. Conversely, passing the caption text without coordinates prevents the system from cropping the relevant figure region for visual validation.

Practical Reconstruction: Spatial Caption Binding and Footnote Inlining

To preserve document hierarchy, an ingestion pipeline must separate floating annotations from body paragraphs, identify their target anchors, and inline them cleanly into the Markdown output.

Here is a Python implementation using PyMuPDF (fitz) that detects captions, binds them to adjacent image bounding boxes, and isolates footer regions:

import fitz  # PyMuPDF
import re

def parse_page_with_bound_annotations(page, footer_margin_pt=50):
    """
    Separates floating captions and footnotes from main text blocks,
    binding captions to figure regions and isolating footnotes.

    Args:
        page: fitz.Page object
        footer_margin_pt: Distance from page bottom treated as footnote zone.
    """
    page_rect = page.rect
    page_height = page_rect.height
    page_dict = page.get_text("dict")

    # 1. Identify image bounding boxes
    image_list = page.get_images()
    image_bboxes = []
    for img in image_list:
        xref = img[0]
        for rect in page.get_image_rects(xref):
            image_bboxes.append(rect)

    # 2. Categorize text blocks
    body_blocks = []
    caption_blocks = []
    footnote_blocks = []

    caption_pattern = re.compile(r'^(Figure|Fig\.|Table)\s+\d+[:\.]', re.IGNORECASE)

    for block in page_dict.get("blocks", []):
        if block.get("type") != 0:
            continue

        bbox = fitz.Rect(block["bbox"])
        block_text = ""
        for line in block.get("lines", []):
            line_str = "".join(span["text"] for span in line.get("spans", []))
            block_text += line_str + " "
        block_text = block_text.strip()

        if not block_text:
            continue

        # Check for footnotes near bottom margin
        if bbox.y1 >= (page_height - footer_margin_pt):
            # Check for footnote prefix like a number or symbol
            if re.match(r'^(\d+|[\*\†\‡])\s+', block_text):
                footnote_blocks.append((bbox, block_text))
                continue

        # Check for figure/table caption pattern
        if caption_pattern.match(block_text):
            caption_blocks.append((bbox, block_text))
        else:
            body_blocks.append((bbox, block_text))

    # 3. Bind captions to nearest image box
    bound_figures = []
    used_captions = set()

    for img_box in image_bboxes:
        closest_caption = None
        min_dist = float("inf")

        for idx, (cap_box, cap_text) in enumerate(caption_blocks):
            if idx in used_captions:
                continue
            # Vertical distance between image bottom and caption top
            dist = abs(cap_box.y0 - img_box.y1)
            if dist < min_dist and dist < 60:  # Within 60 points threshold
                min_dist = dist
                closest_caption = (idx, cap_text)

        if closest_caption:
            used_captions.add(closest_caption[0])
            bound_figures.append(f"![{closest_caption[1]}](image_placeholder)\n*{closest_caption[1]}*")

    # 4. Assemble clean Markdown output
    markdown_sections = []

    # Sort body blocks in reading order (top-to-bottom, column-aware)
    body_blocks.sort(key=lambda b: (b[0].y0, b[0].x0))
    for _, text in body_blocks:
        markdown_sections.append(text)

    # Append bound figures as standalone blocks
    for fig_md in bound_figures:
        markdown_sections.append(fig_md)

    # Inline remaining unattached captions
    for idx, (_, cap_text) in enumerate(caption_blocks):
        if idx not in used_captions:
            markdown_sections.append(f"*{cap_text}*")

    # Append footnotes as a structured reference section
    if footnote_blocks:
        footnote_lines = ["\n### Footnotes"]
        for _, fn_text in footnote_blocks:
            footnote_lines.append(f"- {fn_text}")
        markdown_sections.append("\n".join(footnote_lines))

    return "\n\n".join(markdown_sections)

Tradeoffs: Inlining vs. Standalone Chunks vs. Parent-Document Metadata

When formatting annotations for vector indexing, different indexing architectures offer distinct tradeoffs:

Strategy	Token Overhead	Context Density	Retrieval Precision	Best Application
Inlined Callout (> Footnote)	Minimal	High (adjacent to text)	High for local facts	Short technical reports and whitepapers
Standalone Caption Chunk	Moderate	Focused on visual data	High for diagram-specific queries	Multimodal RAG with image generation or VLM routing
Document Metadata Association	Lowest	Medium	High for global filtering	Legal contracts and compliance documentation

For most enterprise pipelines, Inlined Callouts provide the cleanest balance: the explanation remains attached to the paragraph that references it, preventing vector search from returning orphaned disclaimers.

Layout Annotation Audit Checklist

Before finalizing your technical document pipeline, run this four-point audit on your parsed chunks:

1. Verify Caption Isolation: Confirm that strings matching Figure X: or Table Y: never appear inside the text of an unrelated paragraph.
2. Audit Footnote Association: Inspect sample chunks containing footnote superscripts; confirm that the definition is appended to the chunk rather than separated by page breaks.
3. Validate Visual Coordinates: Ensure that every extracted figure retains its bounding box coordinates (x0, y0, x1, y1) so downstream vision pipelines can crop the source image directly.
4. Test Negative Disclaimers: Run retrieval tests with queries like "What are the documented limitations?" Verify that vector search retrieves the inlined footnote rather than dropping the qualification.