Building a Reliable Pipeline for Extracting Transaction Data from Financial PDFs

Financial documents look structured to humans, but they are often surprisingly difficult for software to process.

A bank statement may appear to contain a simple table:

Date	Description	Debit	Credit	Balance
01/15/2024	Office supplies	89.99		4,120.50

But internally, the PDF may contain:

• Text positioned independently on a page
• Separate text fragments for each character
• Wrapped descriptions across multiple lines
• Different layouts for each bank
• Headers repeated on every page
• Summary blocks mixed with transaction rows
• Scanned pages with no text layer at all

If the goal is to export the data into Excel or accounting software, simply extracting visible text is not enough.

This article explains the main engineering challenges and a practical architecture for building a more reliable financial document extraction pipeline.

The real problem is structure, not text

A basic PDF text extractor can often return something like this:

01/15/2024 Office supplies 89.99 4,120.50
01/16/2024 Wire transfer 5,000.00 9,120.50

That output is readable, but it does not tell us with certainty:

• Which value belongs to which column
• Whether 89.99 is a debit or a credit
• Whether the balance belongs to the same transaction
• Whether a line is a transaction or a summary value
• Whether a wrapped line is a new transaction

The extraction problem is therefore better represented as:

Document
  -> page structure
  -> table boundaries
  -> transaction rows
  -> normalized fields
  -> validated output

The most important design decision is to separate these stages instead of asking one parser to solve everything at once.

A practical extraction pipeline

A production-oriented pipeline can use the following stages:

1. Detect document type
2. Extract the text layer when available
3. Fall back to OCR for scanned pages
4. Identify the main transaction table
5. Reconstruct rows and columns
6. Normalize dates and amounts
7. Validate balances and totals
8. Export to the target format

Each stage has a different failure mode, which makes debugging and testing much easier.

1. Detect whether the PDF has a text layer

Not every PDF needs OCR.

There are two common types:

Native PDF

The document was generated digitally. It usually contains selectable text.

Bank statement generated by online banking
Invoice exported from accounting software
Digital receipt

For these files, direct text extraction is usually faster and cheaper than OCR.

Scanned PDF

The document is effectively a collection of images. Selecting text does not work because there is no usable text layer.

Scanned paper statement
Photographed receipt
Faxed document

These files require OCR before table extraction can begin.

A simplified TypeScript decision function might look like this:

type PdfInspection = {
  pageCount: number;
  textLength: number;
  hasTextLayer: boolean;
};

function shouldUseOcr(inspection: PdfInspection): boolean {
  return !inspection.hasTextLayer || inspection.textLength < 100;
}

The exact threshold depends on the document type. The important point is to make the decision explicitly rather than assuming every PDF can be handled the same way.

2. Use different prompts for different input types

A scanned image and a native PDF should not necessarily use the same extraction instructions.

For an image-based document, the model must:

• Read the visual layout
• Locate table boundaries
• Interpret column alignment
• Recognize low-quality characters
• Reconstruct multi-line rows

For a native PDF converted to text, the model should focus on:

• Identifying the transaction table
• Ignoring page headers and footers
• Ignoring account summaries
• Merging continuation pages
• Preserving the original values

A useful text-extraction instruction is:

Extract only the main transaction table.

Ignore:
- account holder information
- branch addresses
- account activity summaries
- daily balance summaries
- check listings
- loan information
- legal notices
- repeated page headers and footers

If the table continues on another page, merge the rows into one array.
Do not infer missing values.
Return an empty string when a value cannot be read confidently.

This is more reliable than asking for “all text on the page,” because financial statements often contain several different tables with unrelated totals.

3. Use a stable intermediate schema

Before exporting to Excel, QBO, or another accounting format, normalize the extraction result into a stable internal schema.

For example:

type Transaction = {
  date: string;
  description: string;
  amount?: string;
  debit?: string;
  credit?: string;
  balance?: string;
  reference?: string;
};

type ExtractedDocument = {
  header: Array<{
    label: string;
    value: string;
  }>;
  columns: string[];
  rows: string[][];
  summary: Array<{
    label: string;
    value: string;
  }>;
};

The original columns should be preserved initially. Normalization should happen in a separate step.

This is useful because different banks use different labels:

Withdrawal
Debit
Money Out
Payment
Outflow

These may all represent a similar concept, but they should not be blindly merged before the original data has been preserved.

4. Reconstruct wrapped descriptions carefully

Descriptions frequently wrap to a second line:

07/18/2024  ACME RETAIL SERVICES
            MONTHLY BUSINESS SUBSCRIPTION     49.00  2,811.42

A naïve line-based parser may treat this as two transactions.

A better approach is to detect whether a line begins with a date:

const dateAtStart = /^\d{1,2}[/-]\d{1,2}[/-]\d{2,4}\b/;

function isNewTransactionLine(line: string): boolean {
  return dateAtStart.test(line.trim());
}

If a line does not begin with a date, it may be a continuation of the previous description.

However, this is only a heuristic. Some statements use:

• Posting dates without transaction dates
• Transaction IDs at the beginning
• Dates in separate columns
• Non-date transaction identifiers

For that reason, the parser should preserve the original extracted rows and allow the user to review the result before export.

5. Normalize amounts without changing their meaning

Money values may appear as:

$1,234.56
(89.99)
-89.99
89.99 CR
89.99 DR

A normalization function should remove presentation characters while preserving the sign:

function parseAmount(value: string): number {
  const raw = value.trim();

  if (!raw) return 0;

  const isParenthesized = raw.startsWith("(") && raw.endsWith(")");
  const isCredit = /\bCR\b/i.test(raw);
  const isDebit = /\bDR\b/i.test(raw);

  const numeric = Number(
    raw
      .replace(/[,$£€¥]/g, "")
      .replace(/[()]/g, "")
      .replace(/\b(CR|DR)\b/gi, "")
      .trim()
  );

  if (!Number.isFinite(numeric)) return 0;

  if (isParenthesized || isDebit) return -Math.abs(numeric);
  if (isCredit) return Math.abs(numeric);

  return numeric;
}

The parser should not “correct” suspicious amounts automatically.

For example, if the source contains:

$1O0.00

where the character could be either O or 0, silently converting it to $100.00 creates false confidence. It is safer to leave the value empty or flag it for review.

6. Validate running balances

A running balance provides a useful consistency check.

If a statement has:

Opening balance: 1,000.00
Credit:           250.00
Debit:             80.00
Expected ending: 1,170.00

Then the final extracted balance should be approximately:

1,170.00

A basic validation function:

function expectedEndingBalance(
  opening: number,
  credits: number,
  debits: number
): number {
  return opening + credits - debits;
}

function amountsMatch(a: number, b: number, tolerance = 0.01): boolean {
  return Math.abs(a - b) <= tolerance;
}

This check does not prove that the extraction is correct. It only identifies cases that deserve review.

There are several reasons why a summary total may not match the visible transaction table:

• The summary covers a different period
• Check transactions are listed in a separate section
• The statement contains multiple account sections
• Some pages were excluded intentionally
• The PDF contains beginning and ending balance adjustments

Therefore, a mismatch should normally produce a review note, not an automatic rejection.

7. Keep validation warnings close to the risky action

Displaying every warning immediately after extraction can confuse users.

A user exporting to Excel may not need a warning that only matters for a Xero import.

A better user experience is:

General extraction result:
- Show the extracted rows
- Let the user review and edit them
- Avoid unrelated warnings

Xero export:
- Show date-format and accounting import notes
- Ask for confirmation if needed

Sage export:
- Show Sage-specific import notes
- Keep the CSV itself clean

This is an important product design principle:

Show a warning at the point where the user can act on it.

Warnings should also be kept out of machine-readable CSV files. Adding explanatory text before the CSV header may cause an accounting application to interpret the note as a transaction row.

Export should be a separate layer

The normalized transaction data should not be coupled directly to one output format.

For example:

const transactions = normalizeExtractedRows(document);

const excelFile = exportToExcel(transactions);
const csvFile = exportToCsv(transactions);
const qboFile = exportToQbo(transactions);
const xeroFile = exportToXero(transactions);

Each serializer can then enforce its own rules:

Excel:
- Preserve readable formatting
- Keep numeric values as numbers

CSV:
- Use a stable header
- Escape commas and quotes

Xero:
- Normalize dates
- Map signed amounts correctly
- Use the expected columns

Sage 50 UK:
- Normalize DD/MM/YYYY dates
- Sanitize descriptions
- Respect field length limits

This design makes it possible to improve one export format without changing the extraction engine.

What I built with CleanStmt

I built CleanStmt around this type of workflow.

It is a web-based tool for converting bank statements, credit card statements, invoices, receipts, and scanned financial documents into structured data.

The parts I consider most important are:

• A separate path for native PDFs and scanned documents
• OCR instructions focused on financial tables
• One transaction per output row
• No merged cells in the spreadsheet output
• Reviewable extracted values
• Export options for Excel, CSV, QBO, QIF, OFX, Xero, Sage 50 UK, and other accounting workflows

The tool is free to try. There is also a Professional plan for users who process more documents. The current pricing is available at cleanstmt.com/pricing.

Practical testing strategy

A financial extraction pipeline should be tested with more than one clean sample.

A useful test set includes:

1. Native digital bank statement
2. Scanned statement with low contrast
3. Multi-page statement
4. Statement with wrapped descriptions
5. Statement with separate debit and credit columns
6. Statement with one signed amount column
7. Credit card statement with fees and rewards sections
8. Invoice with line items and totals
9. Receipt photographed at an angle
10. Statement with ambiguous date formats

For every fixture, compare:

• Number of extracted rows
• Date values
• Debit and credit totals
• Ending balance
• Exported column names
• Import compatibility
• Handling of uncertain values

The goal should not be to claim that every document is perfect. The goal is to make errors visible, recoverable, and easy to review.

Final thoughts

Financial document extraction is not just an OCR problem.

It combines:

• Document inspection
• Table reconstruction
• Semantic classification
• Amount normalization
• Balance validation
• User review
• Format-specific export rules

The most reliable systems are conservative. They preserve source values, avoid silently guessing, separate extraction from export, and show warnings only when they are relevant to the user's next action.

That approach may produce slightly less “magical” marketing copy, but it creates a workflow that accounting users can actually trust.