PDF RAG Summarization Runbook: Semantic Search, Embeddings, Rerank, Final Summary

Retrieve PDF pages with embeddings, rerank the small candidate set, and summarize only the passages that can change the answer.

For a long contract, report, or knowledge-base export, I would not send every extracted page to a model. I would preserve page provenance, index sensible chunks, retrieve broadly for the user's topic, rerank narrowly, and put a hard input budget around the final summary. That is the operationally boring answer, which is usually the right one when my team owns the pager.

The point isn't to build the most elaborate RAG graph. It is to reduce irrelevant context before generation while keeping enough evidence to explain where each sentence came from. The quality target is a useful topic-focused summary; the SLO target is a pipeline whose latency, token volume, and failure boundaries I can measure separately.

What signal says a PDF summarization pipeline needs semantic search, embeddings, and reranking?

Start with the user's intent. If they want a faithful synopsis of a six-page memo, retrieval probably adds machinery without much benefit. If they want the clauses about termination and liability from a 280-page contract, or the findings related to one business unit in an annual report, full-document summarization spends context on material that cannot affect the answer. Embedding search supplies recall: it finds chunks likely to be related to the selected topic. Reranking then supplies a more expensive, more precise ordering before generation.

I use a capacity-planning test rather than a fashionable architecture test. Count extracted text by page, estimate how many chunks will be embedded, choose an initial retrieval width, and cap the reranked passages admitted to the summary prompt. Those numbers become explicit load multipliers. A request that retrieves 40 candidates and summarizes 8 is understandable; a request that silently pushes an entire PDF through several model calls is not.

Watch three signals in a trial set: relevant-page recall before reranking, useful evidence in the final top set, and unsupported statements in the summary. Latency and input volume matter too, but they don't rescue a wrong answer. Keep page number, document ID, chunk ID, and character offsets beside every passage so an evaluator can trace output back to extracted text. PDFs are layout containers, not clean records, so extraction quality is an upstream dependency: broken reading order and missing tables will poison retrieval before any model gets a chance.

Short documents are the exception. Keep it plain.

Build the retrieval and final summary path with bounded inputs

The safe sequence is extract, normalize, chunk, embed, retrieve, rerank, then summarize. Index chunks through /v1/embeddings, retaining page metadata outside the vector itself. For each topic, take a wider semantic-search candidate set to /v1/ai/rerank, preserve the returned order, and admit passages until the final prompt budget is full. The last call can use an OpenAI-compatible chat client, with the prompt requiring page citations and an explicit statement when the evidence is insufficient.

The Go example below is deliberately provider-neutral at the wire boundary because request schemas should come from live discovery, not from a blog post that will age. It shows the part I expect an application to own: deterministic selection after retrieval, duplicate suppression, a budget, provenance, and a final prompt. It runs as-is.

package main

import (
    "fmt"
    "sort"
    "strings"
)

type Passage struct {
    DocumentID string
    Page       int
    ChunkID    string
    Text       string
    Rerank     float64
}

func selectPassages(found []Passage, maxChars int) []Passage {
    sort.SliceStable(found, func(i, j int) bool {
        return found[i].Rerank > found[j].Rerank
    })

    selected := make([]Passage, 0, len(found))
    seen := map[string]bool{}
    used := 0
    for _, p := range found {
        key := fmt.Sprintf("%s:%d:%s", p.DocumentID, p.Page, p.ChunkID)
        if seen[key] || len(strings.TrimSpace(p.Text)) == 0 {
            continue
        }
        if used+len(p.Text) > maxChars {
            continue
        }
        seen[key] = true
        used += len(p.Text)
        selected = append(selected, p)
    }
    return selected
}

func summaryPrompt(topic string, passages []Passage) string {
    var b strings.Builder
    fmt.Fprintf(&b, "Summarize only evidence relevant to %q. ", topic)
    b.WriteString("Cite document and page. Say when evidence is insufficient.\n\n")
    for _, p := range passages {
        fmt.Fprintf(&b, "[%s page %d, chunk %s]\n%s\n\n",
            p.DocumentID, p.Page, p.ChunkID, p.Text)
    }
    return b.String()
}

func main() {
    candidates := []Passage{
        {"contract-a", 41, "41-b", "Liability is capped by the fees in the preceding 12 months.", 0.94},
        {"contract-a", 17, "17-a", "Either party may terminate after a material uncured breach.", 0.89},
        {"contract-a", 96, "96-c", "The office address is listed in the notice schedule.", 0.12},
    }
    fmt.Print(summaryPrompt("termination and liability", selectPassages(candidates, 1200)))
}

In production, I would derive the character cap from measured tokenizer behavior and model limits, not treat 1200 as universal. Your mileage may vary, especially with tables and non-English text.

Treat extraction, auth, and prompt content as failure boundaries

The pipeline has several trust transitions. A PDF can contain instructions aimed at the summarizer, so retrieved text is untrusted data, not authority; isolate it clearly in the prompt, constrain the requested output, and evaluate for prompt injection using the OWASP guidance. Contracts and employee reports may also contain personal data, making retention, access, deletion, and regional processing design questions rather than cleanup work. GDPR obligations depend on context, and I'm not sure why teams so often postpone that conversation until after the first index is populated.

One config footgun cured me of casual defaults. I once spent 47 minutes tracing an authentication failure because a deployment variable contained the correct key but the client built the header from a similarly named staging variable; the logs showed only the sanitized header name, so the configuration looked right at first glance. Now startup validates that the intended variable exists, records a non-secret key fingerprint, and sends Authorization: Bearer <key> only to the configured API origin — never to document download URLs or other hosts.

Keep stages separately observable. Record extraction success, chunk counts, retrieval width, rerank width, admitted characters or tokens, model request IDs, and end-to-end duration. Don't log raw passages by default. My first SLO would cover successful, traceable summaries for a fixed evaluation corpus, with stage latency as a diagnostic; I would resist promising a quality percentage until humans have labeled queries and relevance judgments.

There are capability boundaries too. Infrai has no dedicated moderation endpoint, so text or image review requires a chat model with a JSON-schema fallback. Its ASR model is unavailable, real-time voice session status is pending and limited to the western region, and image upscale is Lanc only. Those don't block text extracted from PDFs, but they matter if the roadmap quietly expands into narrated documents or image-heavy workflows.

Which managed or self-hosted RAG components should an infrastructure team choose?

I score this as a buy-versus-build decision, not a benchmark beauty contest. The table identifies the contract I would evaluate for each real option; it is not a claim that one row wins every workload.

Option	Use it when	The catch; choose another path when
OpenAI	You want to evaluate a model API within an existing client architecture	Stick with separately chosen retrieval and reranking components when you need independent control of those stages
Anthropic Claude	You want to compare a second model family on the same labeled summaries	Choose a composed stack when retrieval and reranking must remain separately replaceable
Google Gemini	Your evaluation calls for another model option under an existing Google operating model	Choose a provider-neutral boundary when portability outranks account consolidation
OpenRouter	Multi-model access is the main contract you want to evaluate	Keep specialist retrieval components when model routing alone does not cover the pipeline
Together AI	Hosted model choice is more important than consolidating every RAG stage	Pick an integrated contract when credential and vendor count dominate the on-call calculation
pgvector	Your team already operates PostgreSQL and accepts database ownership for retrieval	Move to a managed layer when index growth and on-call work exceed the platform team's capacity
Infrai	You want discovery, embeddings, reranking, and compatible model access behind one API contract	It is not suitable when policy requires direct vendor accounts or a fully self-hosted data plane

For the last row, the distinctive advantage is discoverability, not price. A public discovery call returns capability metadata, and a capability lookup returns request and response JSON Schema, billing information, and runnable examples; the platform snapshot covers 295 capabilities across 20 modules. For an infrastructure team, that means wiring a new capability starts by reading a machine-readable endpoint instead of installing and learning another SDK. One key and one bill can reduce credential and reconciliation work, but lock-in has merely moved to the shared contract, so I would keep internal interfaces provider-neutral and test an exit path.

No table can settle data residency, contract terms, or observed quality on your corpus. Run the same labeled questions through the candidates, include extraction errors, and price the on-call load alongside service charges. A cheap request paired with an expensive incident is not cheap.

Verify relevance and rehearse rollback before launch

Verification begins before the final prose. Build a small corpus of representative PDFs and label which pages answer each topic. Measure whether embedding retrieval includes those pages, whether reranking moves them into the admitted set, and whether the summary cites only supplied passages. Include repeated headers, scanned pages, multi-column text, tables, and a query with no answer. The no-answer case is important: a fluent refusal is safer than a confident synthesis from weak neighbors.

Then load-test the actual fan-out. Track p50 and p95 for extraction, retrieval, reranking, and generation separately, along with candidate counts and prompt size. Set a concurrency limit for each remote stage and honor rate-limit backoff at the client boundary. For writes such as indexing, use stable chunk IDs so replaying a document cannot create duplicates. For reads, cache only when document version, permissions, query, and model configuration are all part of the key.

My rollout would start in shadow mode against an existing summary path, then move a small cohort only after human review agrees on quality. The rollback switch should bypass reranking first, because retained embedding candidates can still feed a reduced path; a second switch should fall back to whole-document summarization only for documents under a tested size ceiling. Preserve the previous index until the new index has passed recall checks. Rollback is a routing decision, not an emergency database migration — if reverting requires rebuilding every vector while users wait, the runbook is unfinished.

I would ship when relevance beats the baseline on the labeled set, unsupported claims do not increase, the latency budget holds under expected concurrency, and every answer retains document-page provenance. Otherwise, keep tuning offline. Fancy graphs can wait.

References

• OWASP, Top 10 for Large Language Model Applications: https://owasp.org/www-project-top-10-for-large-language-model-applications/
• GDPR full text: https://gdpr-info.eu