DocuFlow: Turning Invoice PDFs into Structured Financial Data with Python and Docker

DocuFlow: Turning Invoice PDFs into Structured Financial Data

Invoices are easy to overlook when they’re just files sitting in a folder.

But once you need to extract information from hundreds of documents, identify duplicates, store financial records, and analyze spending, things get more interesting.

That’s the problem I wanted to tackle with DocuFlow — an intelligent, containerized invoice ingestion pipeline built with Python.

Shashank0701-byte / docuflow

DocuFlow is a containerized, event-driven pipeline that automates ingestion, OCR, parsing, and analytics for invoices and financial documents.

DocuFlow — Intelligent Invoice Ingestion Pipeline

DocuFlow is a containerized, event-driven pipeline that automates ingestion, OCR, parsing, and analytics for invoices and financial documents. It uses a lightweight microservice pattern so heavy OCR work runs asynchronously and does not block the UI.

Quick highlights:

• Hot-folder watcher for real-time ingestion
• Celery + Redis for asynchronous OCR and parsing
• Tesseract + Poppler for PDF/image text extraction
• PostgreSQL for persistent storage
• Streamlit dashboard for visualization and quick analysis

---

🏗️ Architecture

The system follows a microservices pattern and is orchestrated with Docker Compose.

If GitHub renders the diagram you will see the flow below. If it doesn't, a plain ASCII fallback is provided right after.

graph LR
  A[data_raw_hot_folder] -->|New PDF detected| B[watcher_service]
  B -->|Push task| C[redis_broker]
  C -->|Distribute task| D[celery_worker]
  D -->|Extract text| E[tesseract_poppler]
  D -->|Parse & save| F[postgres_db]
  F -->|Query| G[streamlit_dashboard]

Loading

Plain fallback (always visible):

Flow
data/raw (Hot Folder) -> Watcher Service

…

View on GitHub

What does it do?

DocuFlow watches a designated folder for new invoice files and processes them through an asynchronous pipeline.

The workflow looks like this:

New PDF -> File Watcher -> Redis + Celery -> OCR -> Data Parsing -> PostgreSQL -> Streamlit Dashboard

The goal is to turn unstructured invoice documents into structured, queryable financial data without making the dashboard wait for every document to finish processing.

Under the hood

Here are some of the engineering decisions behind the project:

Event-driven ingestion: A folder watcher detects incoming documents and queues them for processing.

Asynchronous processing: Celery and Redis separate the ingestion flow from the OCR workload, making the architecture easier to extend with additional workers.

OCR and parsing: Tesseract and Poppler help extract text from PDFs, while parsing logic identifies fields such as invoice number, date, vendor, and total amount.

Persistent storage: PostgreSQL stores the extracted records for downstream queries and analysis.

Data visibility: A Streamlit dashboard presents spending trends, vendor breakdowns, recent invoices, and processing information.

Containerized services: Docker Compose brings the watcher, worker, database, broker, and dashboard together in a reproducible local environment.

I also worked on duplicate detection so that the pipeline can avoid reprocessing the same invoice unnecessarily.

What I wanted to explore

This project was about more than extracting text from a PDF.

I wanted to work with the pieces that turn a script into a pipeline: asynchronous jobs, service boundaries, persistence, containerization, and visibility into the processing workflow.

It also made me think more carefully about what happens when a document is malformed, OCR produces noisy output, or a parser encounters an unexpected invoice format.

There’s still room to grow — including stronger test coverage, more robust parsing for varied invoice layouts, and better production-grade observability.

What's next?

• More resilient extraction for inconsistent invoice formats.
• Better automated testing and CI integration.
• Email-based invoice ingestion.
• Smarter parsing for complex financial documents.
• Improved operational visibility and failure recovery.

DocuFlow is an evolving project, and I’m looking forward to improving it further.

I’d especially love feedback from developers who have built document-processing pipelines: What would you improve first — the OCR and parsing layer, task reliability, or observability?