Building an AI-Powered Invoice Processing Pipeline: OCR Meets LLMs

DHARANEE — Sat, 05 Sep 2026 06:57:48 +0000

Building an AI-Powered Invoice Processing Pipeline: OCR Meets LLMs

Invoice processing sounds simple until you actually deal with invoices coming from different vendors.

Different layouts, different field names, scanned documents, inconsistent formatting, missing information, duplicate invoices, and even suspicious transactions can make manual processing slow and error-prone.

As part of our project, we worked on an AI-powered invoice processing pipeline designed to automate much of this process — from extracting information from invoices to validating the data and identifying potential duplicates or fraud.

I worked as the Lead Frontend Developer, primarily responsible for the frontend experience and its integration with the backend services. At the same time, I worked closely with the team to understand the core processing pipeline and how the different services connected together.

This article walks through how the system works and some of the engineering decisions behind it.

1. The Problem

In a typical supply-chain environment, invoices can arrive in many different formats. Some may be digitally generated PDFs, while others may be scanned documents or images.

Processing them manually means employees have to read the documents, identify important fields, enter the information into a system, verify the calculations, and check whether the invoice has already been processed.

This becomes time-consuming and introduces opportunities for human error.

Our goal was to build a system that could automate as much of this workflow as possible while still keeping validation and human review in the loop when the system was uncertain.

2. The High-Level Idea

The basic idea behind our system was to create a pipeline where each component has a specific responsibility.

An invoice first enters the system as a PDF or image. Instead of sending the document directly to a language model, we first use OCR (Optical Character Recognition) to extract the text from the document.

That extracted text is then passed to an LLM, specifically Llama 3.3 70B, which understands the context of the invoice and converts the unstructured OCR output into structured information such as the vendor name, invoice number, dates, tax values, and total amount.

The structured result is then passed through validation and business checks. The system also performs duplicate detection and fraud-oriented checks before the final information is stored and made available through the application dashboard.

In simplified form:

Invoice
   ↓
OCR
   ↓
Raw Text
   ↓
Llama 3.3 70B
   ↓
Structured JSON
   ↓
Validation
   ↓
Duplicate & Fraud Checks
   ↓
Clean Structured Data
   ↓
Database + Dashboard

The important idea here is that the LLM isn't responsible for everything. It is one component inside a larger processing pipeline.

3. Tech Stack & Why

Tesseract 5 for OCR

The first major component of the pipeline is OCR.

We used Tesseract 5 to extract text from invoice documents.

You might wonder:

Why not simply send the invoice directly to an LLM?

That's a reasonable question, especially because modern vision-language models can process images directly.

For our architecture, separating OCR from language understanding gave us more control over the pipeline.

OCR converts the visual document into machine-readable text:

Invoice Image/PDF
       ↓
     Tesseract
       ↓
Extracted Text

Once we have the text, the language model can focus on understanding the content rather than performing the initial character recognition.

This separation also makes debugging easier. If something goes wrong, we can determine whether the problem came from the OCR stage or from the LLM's interpretation of the extracted text.

Llama 3.3 70B via Groq Cloud

Once OCR produces raw text, we need to understand what that text actually means.

This is where Llama 3.3 70B comes in.

Invoices are not standardized documents. One vendor might use:

Invoice Number

while another might use:

Bill No.

and another could use:

Reference ID

A purely rule-based parser would require us to continuously add rules for these variations.

An LLM provides a more flexible approach because it can understand the context and semantic meaning of the extracted text.

We used Llama 3.3 70B through Groq Cloud and designed prompts that instructed the model to return structured information in a predefined JSON format.

For example, instead of simply asking:

Extract the invoice details.

we could provide explicit instructions about the expected fields, formatting requirements, missing values, and output structure.

The result is something like:

{
  "invoice_number": "INV-1024",
  "vendor": "ABC Technologies",
  "subtotal": 20000,
  "tax": 3600,
  "total": 23600
}

The key point is that the LLM isn't treated as an unquestionable source of truth. Its output goes through additional validation before being accepted by the system.

FastAPI

For the backend API layer, we used FastAPI.

The frontend and backend were developed as separate layers, so the frontend communicates with the backend through APIs.

FastAPI was a good fit because the backend was primarily API-driven and needed to interact with several processing components, including OCR, the LLM service, database operations, and file storage.

It also provides automatic API documentation and request validation through Pydantic, which makes development and testing more convenient.

At a high level:

Next.js Frontend
       ↓
   REST API
       ↓
    FastAPI
       ↓
Processing Services
       ↓
Database / Storage

This separation also allowed the frontend and backend to evolve independently.

4. Architecture

One thing we wanted to avoid was building the entire application as one large script.

Instead, we designed the application as a modular, multi-tenant system with nine service layers, where different responsibilities were separated into appropriate components.

At a high level, the architecture can be viewed as:

                Frontend
                   ↓
              API Layer
                   ↓
          Authentication / RBAC
                   ↓
          Invoice Processing
                   ↓
          OCR + LLM Pipeline
                   ↓
       Validation / Fraud Checks
                   ↓
       Duplicate Detection Layer
                   ↓
          Database / Storage
                   ↓
        Notifications / Reporting

The exact responsibilities are separated across the application's service layers rather than putting everything into a single processing function.

The multi-tenant design is particularly important because invoices and users belonging to one organization should remain logically separated from another organization's data.

For example:

Organization A
 ├── Users
 ├── Invoices
 └── Reports

Organization B
 ├── Users
 ├── Invoices
 └── Reports

This makes the system more suitable for a SaaS-style environment where multiple organizations can use the same application.

5. The Hard Part #1: Duplicate Detection

One of the more interesting parts of the project was duplicate invoice detection.

At first glance, it might seem easy:

Just compare the invoice number.

But invoice numbers aren't globally unique.

Two different vendors can legitimately generate:

INV-1001

So invoice number alone isn't sufficient.

We therefore used a two-tier approach.

Tier 1: SHA-256 Hashing

The first level handles exact duplicate files.

When an invoice file is uploaded, we generate a SHA-256 hash for the file.

The same file will produce the same hash.

For example:

Invoice.pdf
     ↓
 SHA-256
     ↓
ABC123...

If exactly the same file is uploaded again, its SHA-256 value will match the existing record.

This allows us to detect exact file duplicates efficiently without having to compare the complete contents of every document.

Tier 2: Logical Triplet Matching

But there is a problem with relying only on file hashes.

Imagine the same invoice is downloaded and re-saved as another PDF.

Maybe:

the metadata changed,
the PDF was compressed,
the file was scanned again,
or the document was generated slightly differently.

The content may represent the same invoice, but the binary file itself is different.

In that case:

File A → SHA-256 → Hash A

File B → SHA-256 → Hash B

The hashes won't match.

That's where logical matching comes in.

We compare a combination of important invoice attributes:

Vendor Name
+
Invoice Number
+
Total Amount

This is our logical triplet.

If these values match an existing invoice, the system can flag it as a potential duplicate even when the physical files are different.

Why two methods?

Because each method solves a different problem.

SHA-256:

"Is this exactly the same file?"

Logical triplet matching:

"Does this appear to represent the same invoice?"

Using both gives us a stronger duplicate-detection mechanism than relying on either one alone.

It also illustrates an important engineering principle: sometimes the best solution isn't one sophisticated algorithm, but multiple simpler checks working together.

6. The Hard Part #2: Fraud Heuristics

Duplicate detection is only one part of invoice verification.

We also implemented rule-based checks to identify potentially suspicious invoices.

The purpose wasn't to claim that the system could definitively prove an invoice was fraudulent.

Instead, the system identifies anomalies and risk indicators that deserve additional attention.

Examples of checks include:

Duplicate invoice detection
Unusual invoice amounts
Suspicious patterns in invoice information
Missing or inconsistent fields
Mathematical inconsistencies
Policy violations
Unexpected invoice patterns

For example, if:

Subtotal = ₹10,000
Tax      = ₹1,800
Total    = ₹25,000

the numbers don't add up.

That doesn't automatically mean the invoice is fraudulent, but it is a strong reason to flag it for review.

This is why we treated fraud detection as a risk and decision-support mechanism, rather than an absolute "fraud/not fraud" verdict.

The system can assign risk indicators and allow suspicious invoices to move toward manual review.

7. What I'd Do Differently / Lessons Learned

Building the project taught me that integrating AI into an application is very different from simply calling an AI API.

1. Don't blindly trust model output

An LLM can produce convincing but incorrect information.

This is particularly dangerous when processing financial documents.

Our solution was to place validation after the LLM rather than treating its output as ground truth.

If something doesn't make sense, the system should be able to catch it.

2. Separate responsibilities between components

It is tempting to make one AI model responsible for everything.

But a better architecture is often:

OCR → Extraction
LLM → Understanding
Rules → Validation
Hashing → Exact Duplicate Detection
Database → Persistence

Each component has a clear responsibility.

This makes the system easier to debug and maintain.

3. The frontend is more than just UI

As the Lead Frontend Developer, one of my biggest takeaways was that frontend development in a system like this isn't only about creating screens.

The frontend has to understand:

API states
Authentication
Error handling
Loading states
Processing states
Role-based access
Backend responses

For an invoice-processing application, a user needs to know whether an invoice is:

Uploaded
   ↓
Processing
   ↓
Approved
   ↓
Rejected
   ↓
Or Requires Review

A good frontend needs to communicate that entire lifecycle clearly.

8. Wrap-up

Building this project gave us a practical look at how OCR, LLMs, APIs, cloud storage, databases, validation, and frontend systems can work together as one application.

The most important lesson for me was that AI doesn't replace traditional software engineering — it becomes one component within it.

In our case:

Computer Vision
      +
Generative AI
      +
Backend Engineering
      +
Database
      +
Cloud Storage
      +
Rule-Based Validation
      +
Frontend
      ↓
Complete Invoice Processing System

There is still plenty that could be improved, especially around more advanced fraud detection, near-duplicate matching, model evaluation, and asynchronous processing at larger scale.

But that's also what makes engineering projects interesting: the first working system is not the final system.

I'd love to hear how others would approach invoice automation, duplicate detection, or AI-assisted document processing. Feel free to share your thoughts or questions in the comments.

DEV Community: DHARANEE

AI is Evolving