DEV Community: Fabien Kost

OCR is a step, not a guarantee (turning scanned archives into usable data)

Fabien Kost — Tue, 27 Oct 2026 14:00:00 +0000

Adapted from an article originally published in French on closiqode.com.

Running OCR over scanned PDFs, images or archives does not make them usable on its own. To get there, you need searchable text, a permanent link to the original, a structure for the information that matters, and a check on the uncertain cases.

OCR reads the document. The data only becomes useful once it finds its place in a structure.

A digital archive can be as opaque as a filing cabinet

Scanning a document protects the physical medium and makes it easier to share. It does not turn its content into data. An image sitting in a folder is still hard to search, compare or connect to a business application.

The confusion usually comes from four different operations being treated as one:

Digitizing creates an image.
OCR recognizes the characters.
Indexing makes the content searchable.
Structuring turns specific elements into usable fields.

Each step answers a different need.

You may already have thousands of well-named scans. Yet finding a reference, a date or a measurement still means opening files one by one. The knowledge exists, but it is stuck on the page. You see this gap in technical archives, customer files, field reports and old invoices: storage is solved, usage is not. So the project has to start from the next expected action, not from the volume to digitize.

OCR is a step, not a guarantee

The French National Library (BnF) describes a chain that combines preprocessing, segmentation and recognition. It also stresses the impact of contrast, orientation, smudges, columns and illustrations. A sound setup therefore keeps the source image and ties the extracted text to its page.

For business use, add sampling checks and consistency rules. An impossible date, an unknown unit or a total that does not add up must raise an alert rather than be imported silently. As an example, in Python:

from datetime import date

KNOWN_UNITS = {"mm", "cm", "m", "kg"}

def check_record(rec):
    """Return a list of issues; a non-empty list sends the record to review."""
    issues = []
    if rec.doc_date and rec.doc_date > date.today():
        issues.append("document date is in the future")
    if rec.unit not in KNOWN_UNITS:
        issues.append(f"unknown unit: {rec.unit!r}")
    if rec.lines and abs(sum(l.amount for l in rec.lines) - rec.total) > 0.01:
        issues.append("line amounts do not add up to the total")
    return issues

Each flagged record goes to a review queue with its file and page reference, so the reviewer looks at the original instead of trusting the extracted text.

Design the output before processing the backlog

Need	Useful output	Priority check
Find a file	Full-text index	OCR quality and metadata
Compare values	Structured table	Units and context
Detect an anomaly	Alert with evidence	Business rule and source page
Query the archive	Search engine or assistant	Access rights and citations

The system can process the normal case automatically and isolate the exceptions: unreadable page, incomplete document, handwriting, unusual table or contradiction. That human review queue makes the project more reliable and easier to improve over time.

Step 1: know what you are looking for

Digitizing everything without a precise goal can simply produce a bigger digital backlog. Decide:

which pieces of information have value;
how they will be used;
who needs access;
in what form they must come out.

Step 2: separate reading from understanding

Recognizing "12.4" in an image is one thing. Knowing that 12.4 is a specific measurement, in a specific context, with a defined tolerance, is another.

That layer of logic is exactly what a business application adds on top of OCR.

Step 3: plan for exceptions

Old documents. Poor quality. Missing pages. Different fields. Contradictory data.

The system must be able to report the situations it cannot handle, instead of guessing.

Step 4: make the data usable in your tools

The output can take several forms:

a spreadsheet;
a search engine;
an alert;
a structured database;
a dashboard;
an API;
a business assistant.

The best output always depends on what has to happen next.

From archived image to controlled data: five levels

Level	What becomes possible	What is still missing
1. File stored	Opening the document if you know where it is	Search in the content and reliable context
2. Text recognized	Searching for words after OCR	Structure, error correction, link between a value and its role
3. Document indexed	Filtering by type, date, folder or entity	Business fields comparable across documents
4. Structured data	Sorting, comparing, exporting, feeding an application	Quality rules, exceptions and proof of origin
5. Controlled data	Triggering a workflow with targeted review of uncertain cases	Continuous tracking of errors, versions and new formats

The right goal is not necessarily level 5 for every archive. Choose the level that enables the intended use, then keep the link to the original so any important value can be checked.

FAQ

Is OCR enough to query an archive?
No. It gives you text. You still need to index that text, manage metadata and access rights, and keep the link to the source.

Do all OCR errors need manual correction?
Not necessarily. You can focus review on critical fields and on results below a confidence threshold.

Can tables and forms be processed?
Yes, but their structure must be tested on every variant. A shifted row can change the meaning of a value.

Should the original files be deleted?
No. The original remains the reference, kept according to your legal obligations and archiving policy. The extracted data is an extra layer on top of it.

Storage is solved, usage is not

If your archives are digitized but still hard to use, the problem is probably less about storage and more about the missing tool between the page and the decision.

I am Fabien Kost, an independent developer in France (ClosiQode). I turn scans and archives into checked tables and databases that keep a link to every source page. More on how I approach document extraction (in French).

Sources (in French): BnF, text conversion techniques and formats and mass digitization, quality and formats.

How RAG lets an AI answer from your company documents (and what it does not fix)

Fabien Kost — Tue, 20 Oct 2026 14:00:00 +0000

Adapted from an article originally published in French on closiqode.com.

Retrieval-augmented generation (RAG) lets an AI answer from your own documents. Before writing anything, the system searches your document base for the relevant passages, then builds its answer from them and can say where it came from. The model learns nothing new: at every question, it is handed the excerpts that matter.

The term shows up in every demo and every quote, and it is rarely explained. The business question behind it is simple: how can a model talk about your procedures, technical sheets or terms of sale without ever having seen them? And what do you need to prepare so its answers are right?

RAG in one paragraph

RAG combines two building blocks: a search engine that finds the right passages in your documents, and a language model that writes an answer from those passages. The term comes from a 2020 paper (Lewis et al., arXiv) whose authors already wanted to solve two problems: showing where an answer comes from, and updating a model's knowledge.

The French Directorate General for Enterprise (DGE) sums it up in its November 2024 guide: RAG lets a model "consult" company data before answering. The same guide explains why this is needed: large language models are trained on general text, they know nothing about your company, and their knowledge is frozen at training time.

A useful picture: without RAG, the model answers from memory. With RAG, it answers open book, with the pages the search engine brought in front of it. Answer quality therefore depends as much on the pages retrieved as on the model reading them.

How a RAG pipeline works, step by step

A RAG system works in two phases: document preparation, done once and again at every update, and retrieval plus answer, done at every question.

Before any question: prepare and index

Conversion: every document becomes text. A scanned PDF goes through OCR, a slide deck is converted to text.
Cleaning: remove duplicates, repeated headers and footers, cover pages, and set aside documents that contradict each other.
Chunking: split documents into short segments, from a few sentences to a section. Chunk size is a decisive setting, to be chosen with the people who know the content.
Enrichment: attach context to every chunk (source document, chapter, date, department) for filtering and citation.
Indexing: convert every chunk into a vector that represents its meaning, and store those vectors in a vector database.

At every question: retrieve, then answer

Retrieval: the question is converted too, then compared to the chunks. The system keeps the closest ones by meaning, by shared keywords, or by a combination of both (hybrid search).
Assembly: the selected chunks go to the model along with the question and instructions such as "answer only from these excerpts".
Sourced answer: the model writes the answer and lists the documents used. If nothing relevant was found, a well-designed system says so instead of improvising.

In pseudo-code, the query-time half looks roughly like this:

# Pseudo-code, not a specific library
def answer(question, user):
    hits = index.hybrid_search(
        text=question,
        top_k=8,
        filter={"allowed_groups": user.groups},  # access rights at retrieval time
    )
    if not hits or hits[0].score < MIN_SCORE:
        return refuse("I cannot find this in the validated documents.")
    prompt = build_prompt(
        question,
        excerpts=hits,
        rules="Answer only from the excerpts. Cite the source for every claim.",
    )
    return llm.generate(prompt), [h.source for h in hits]

Typical use: a technician asks which setting to apply on a given machine model. The engine finds the right page of the manual, the model phrases the answer and points to the manual. If that manual is not in the base, the correct answer is "I cannot find this information", not a plausible value.

RAG versus a model trained on your data

RAG does not train the model on your documents. It shows them to the model at question time, and the model does not keep them. This is the most common confusion, and it has practical consequences for updates, confidentiality and traceability.

Fine-tuning changes the model's internal parameters. The knowledge gets blended in: you cannot tell which document a sentence came from, or cleanly remove an outdated fact. The French cybersecurity agency (ANSSI) notes in its April 2024 security recommendations for generative AI that user access rights cannot be managed on training data, whereas they can be on data added in production, such as a vector database.

Criterion	RAG	Fine-tuned model
Where the knowledge lives	In your documents, consulted at every question	In the model's parameters
Updating	Replace the document and reindex it	A new training run
Citing the source	Possible, passage by passage	No reliable link to a document
Access rights	Filterable per user	Not manageable, according to ANSSI
Good fit	Answering from a corpus that changes	Enforcing a very specific style, format or vocabulary

For a small or mid-sized company that wants to query its procedures, sheets or contracts, RAG is the logical starting point. Fine-tuning is rarely justified at that scale, and it fits poorly with information that changes often.

Which documents work, and which cause trouble

RAG performs well on written, current, non-contradictory text. It struggles with complex tables, poor scans and questions that require reading the whole corpus. The DGE guide notes that RAG is generally better on text than on tables, and less suited to summaries spanning many documents.

Document type	What happens	What to plan
Procedures, internal guides, written FAQs	Good fit: each passage stands on its own	Make sure only one version circulates
Technical sheets, manuals	Good fit if the text is machine-readable	Keep product references in the text, not only in an image
Terms and conditions, contract templates	Good for finding a clause	Date every version and remove old ones
Price or specification tables	Risky: chunking can separate a value from its header	Specific processing or a structured source
Old scans, handwritten documents	Depends on OCR quality	Check the extracted text before indexing
Email threads	Noisy: repetition, provisional opinions, decisions later reversed	Move final decisions into a reference document

A good starting scope is often a small set of reference documents: the ones that answer the questions asked every week.

The limits to know before you start

RAG reduces fabricated answers without eliminating them, and it is never better than the documents you give it. The DGE guide says explicitly that hallucinations can still occur with RAG. Four limits come up in practice:

Retrieval misses: only a limited number of chunks are kept. If the right information is phrased differently or spread across documents, it may not come up.
Documents contradict each other: two versions of a procedure give two possible answers, and the model may pick the wrong one or blend both.
Outdated versions stay indexed: a forgotten old price list keeps producing wrong answers, with a source to back them.
A citation does not prove everything: you still have to check that the cited passage actually supports the generated sentence.

Preparing your documents

Preparation matters as much as the technology. The DGE guide calls it an essential step and stresses that domain expertise remains indispensable, even when a contractor handles it.

List real questions for a few weeks, with the expected answer and the document that contains it.
Designate reference documents: one dated version per topic, with an owner.
Remove duplicates and old versions, or archive them outside the queried scope.
Spot what is written nowhere: answers that only exist in someone's head must be written down first.
Classify by access level.
Check for personal data: the DGE guide lists anonymization or pseudonymization among the cleaning steps.

The rule I set before developing anything: a useful company RAG answers from a validated corpus, cites the document used, and refuses explicitly when the base does not contain the information. It is tested with real questions before going live, including the ones it must refuse.

FAQ

Does RAG train the model on my documents?
No. Documents stay in a separate base and only a few excerpts are sent per question. Still check the model provider's terms: ANSSI notes that prompts and answers are sometimes reused for training.

How do you update it when a document changes?
Replace the document at the source, reindex it, remove the old version from the index, then replay the test questions.

Open book, not a better memory

RAG gives the model the right pages. Choosing, cleaning and testing those pages is still the real work.

I am Fabien Kost, an independent developer in France (ClosiQode). I build business AI assistants that answer from validated documents, cite their sources and say so when they cannot find the answer. More on how I design them (in French).

Sources: DGE, guide to retrieval-augmented generation (in French, PDF); ANSSI, security recommendations for a generative AI system (in French); Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv, 2020).

Extracting data from hundreds of PDFs into Excel (and why a converter is not enough)

Fabien Kost — Tue, 13 Oct 2026 14:00:00 +0000

Adapted from an article originally published in French on closiqode.com.

Getting data out of one PDF into Excel is a copy-paste job. Getting it out of hundreds of PDFs is a pipeline problem: identify each document, extract the text, structure the useful fields (one column per field), then check the result.

A plain PDF-to-Excel converter breaks down as soon as the files are scanned, badly framed or laid out differently. And there is one rule I never drop: every extracted value must stay linked to the document it came from.

A PDF is not necessarily a data source

Two documents can look identical on screen. Technically, the first one contains selectable text; the second one is just an image. A third one mixes text, tables, stamps and scanned pages. So processing starts by classifying the files, not by running the same extractor blindly over the whole folder.

OCR turns areas of an image into usable characters. The French National Library (BnF) points out that its quality depends on contrast, deskewing, print defects and layout complexity. And recognizing the words correctly does not guarantee that the right value was attached to the right field.

A first triage can be as simple as checking whether each page has a text layer. As an example, with PyMuPDF:

# Example: route each page to native extraction or OCR
import fitz  # PyMuPDF

def route_pages(path, min_chars=50):
    doc = fitz.open(path)
    for page in doc:
        text = page.get_text().strip()
        yield page.number, "native" if len(text) >= min_chars else "ocr"

The threshold is a heuristic, not a rule: a scanned page with a stamp may carry a few characters, and a native page may be almost empty. The point is to make the routing decision explicit and logged.

Where manual processing hits a wall

The problem rarely shows up with ten files. It becomes visible when a team opens hundreds of reports, invoices, technical sheets or regulatory files every week. People know exactly what to look for, but their time goes into navigating, copying, pasting and repeating the same checks.

At that point, the goal is not "read every PDF". You need to define a useful output: one row per document in a spreadsheet or a database, an alert on an inconsistency, a list of references, a status to review. Without that decision, the project automates noise.

So before talking about tools, write down:

the fields you need;
their possible formats;
the known exceptions;
the checks each value must pass;
the expected result.

A traceable pipeline, step by step

Inventory: formats, languages, scans, tables and known variants.
Route: native extraction for text, OCR for images, specific processing for complex tables.
Normalize: dates, units, references and missing values.
Apply the business rules: consistency checks, thresholds and exceptions.
Trace: keep the document, page and zone each value came from.
Send ambiguous cases to a person: anything below a confidence threshold goes to human review.

In practice, an extracted value is not just a cell. It is a small record, for example:

{
  "source_file": "report-0412.pdf",
  "page": 3,
  "field": "delivery_date",
  "raw_value": "12/04/26",
  "normalized_value": "2026-04-12",
  "method": "native_text",
  "status": "needs_review",
  "review_reason": "day/month order inferred from document locale"
}

The spreadsheet is then only a view over these records. When someone asks where a number comes from, the answer is the file, the page and the rule that accepted it.

Pick the processing path per document

Document observed	First approach	Risk to test	What to keep
PDF with selectable text and a stable layout	Native extraction with position or label rules	Mixed columns, repeated headers, value attached to the wrong block	Value, page, zone and source file
Clean scan of a known form	OCR, then extraction of the expected fields	Look-alike characters, misread checkbox, skewed page	Value, confidence level and image of the zone
Documents in varied formats	Classification first, then processing per family	One model works on frequent cases but fails on a rare variant	Detected family, processing version and reason for review
Complex table or degraded document	Specialized extraction with targeted review	Structure rebuilt in a plausible but wrong way	Extracted cells, coordinates and the untouched original

Test this routing on a sample that also contains the bad scans and the rare layouts. A good overall average must not hide the one document family where the errors concentrate.

Real example: a value present at the wrong level

Not all PDFs look alike. Some contain real text, others are scans, tables move around, and the same information can appear in several forms. But the harder problem is elsewhere: detecting a value is not always enough. You need to know what it means in the business.

In a project I built for an engineering firm, an important piece of regulatory information could be declared at the level of a component while remaining invisible in the main product information. The value was in the PDF, but reading it at the wrong level would have produced a wrong row in the spreadsheet.

The client deliberately included this case in a validation document, without telling me in advance. The tool caught it.

That is why document extraction is not just "reading text". The system has to reproduce the rules that make the information useful: read each file, then check each value against the business rules.

Measure quality in separate layers

A serious check distinguishes at least four results:

Was the document opened and every page processed?
Was the text recognized correctly?
Were the fields attached to the right context?
Did the business rule produce the right decision?

This separation avoids a false sense of precision. An OCR confidence score measures recognition; it does not prove that the extracted value answers the business question.

To validate, build a ground truth: a representative batch whose expected results have been checked by hand. Add difficult cases, compare the outputs, and measure the ability to find the information separately from the ability to interpret it correctly.

FAQ

Can scanned PDFs be processed?
Yes, with OCR. Scan quality remains decisive, and critical values must be checked.

Do you need generative AI?
Not systematically. Deterministic extraction is often the better choice for stable formats. AI can help with variable wording or layouts, as long as there is validation and traceability.

How do you handle errors?
The system must flag unread documents, missing fields and uncertain results. A silent gap is more dangerous than an alert.

Where do you start?
Pick a single document type, define the expected fields and prepare a representative sample. A first test then gives you a realistic view of feasibility and exceptions.

The spreadsheet is the easy part

The real work is being able to say, for every cell, which page it came from and which rule it passed.

I am Fabien Kost, an independent developer in France (ClosiQode). I build document data extraction tools that read every file, check every value against business rules and keep a link to the source page. More on how I approach document extraction (in French).

Sources (in French): BnF, text conversion techniques and formats and BnF digitization standards.

How to reduce hallucinations in a business AI assistant (and why "never make things up" in the prompt is not enough)

Fabien Kost — Tue, 06 Oct 2026 14:00:00 +0000

Adapted from an article originally published in French on closiqode.com.

You cannot remove every hallucination from a generative AI assistant. What you can do is make them rarer, and make sure an unverified answer never triggers a consequence that matters.

In practice, that means combining six things: a narrow scope, validated sources, an explicit refusal, hard tests, citations and human control.

The French data protection authority (CNIL) puts it simply: generative models are not knowledge bases. Their probabilistic logic can produce a result that is wrong but plausible. The real danger is not the visible error. It is the convincing answer nobody thinks of checking.

1. Narrow the scope

The more an assistant tries to answer everything, the further it drifts from its job.

Write down the allowed topics, the forbidden topics, who the users are and what the assistant is allowed to do. "Answer HR questions" is too broad. "Find the validated administrative steps for a leave request" is testable.

2. Ground it in verified sources

For a business assistant, the document base matters more than the model:

documents that are up to date,
identified versions,
content that someone has actually validated.

A RAG system connects the model to a search over a separate knowledge base. It can anchor answers in your own information, but a bad retrieval, an obsolete document or a wrong interpretation are still possible. Give every important source an owner and a review date.

3. Make "I don't know" a feature

The assistants I build are designed to refuse explicitly when the knowledge base does not support a reliable answer.

A good refusal is useful. It says what is missing, suggests which source to provide, or hands over to a person. A vague "I'm not sure" is not enough if the user can still take the answer at face value.

A minimal instruction looks like this, but remember that the instruction alone guarantees nothing: the tests in step 4 are what prove it works.

Answer only from the provided excerpts.
If the excerpts do not contain the answer, say so explicitly,
list what information is missing, and suggest who to ask.
Quote the source document and version for every factual claim.

4. Test the hard questions

Your test set should cover:

ambiguous questions,
missing information,
contradictory sources,
unusual wording,
edge cases.

A system tested only on obvious cases gives a false sense of reliability.

A minimal protocol

Collect real questions, each with the expected answer and the expected source.
Add impossible, out-of-scope or deliberately misleading questions.
Test several phrasings of every critical question.
Score retrieval relevance and answer faithfulness to the source separately.
Replay the whole test set after every change of model, prompt or corpus.

A test case can be as simple as:

{
  "question": "How many days in advance must a leave request be submitted?",
  "expected_source": "hr-procedures-v3.pdf, section 2",
  "expected_behavior": "answer_with_citation"
}

with a sibling case where expected_behavior is refuse because the corpus does not contain the answer.

5. Do not automate sensitive decisions blindly

The assistant can prepare, search, summarize or route. That does not mean every decision should happen without a human.

Match the control to the consequence:

Possible consequence	Example	Recommended control
Low and reversible	Drafting an internal note	Simple review
Moderate	Answering a customer about a procedure	Visible source and an escalation path
High	Committing spend or acting on a person	Mandatory human validation and an audit trail

6. Keep the sources visible

When the stakes justify it, let users check where the information came from.

The citation must point to the document that was actually used, with its version if needed. And check that the cited passage really supports the generated sentence: a reference being present is not proof that it is the right one.

7. Monitor errors in production

Launch is not the end of the work. Add a way to flag an answer, keep what you need for diagnosis while respecting data protection rules, and turn every incident into a new test case.

Track useful indicators: answers without a source, appropriate refusals, human corrections, out-of-corpus questions and error severity. A satisfaction score alone does not measure reliability.

Diagnose before you fix

Not every error comes from the model. To fix an assistant for good, find the step that failed:

Observed error	Diagnostic question	Useful check
The right source was not retrieved	Was the document present, current and accessible to this user?	Test retrieval on its own; check permissions, metadata and chunking
The source is right but the answer distorts it	Is the generated sentence really supported by the cited passage?	Compare claim and excerpt, limit rewriting, force a refusal when evidence is weak
The answer is fine but the action would be risky	What happens if it is misread?	Block automatic action, show the source, require proportionate human validation

This keeps you from "fixing" every incident by adding one more line to the prompt. Each error becomes a reproducible test.

What does not work on its own

Writing "never make things up" in the prompt with no other control.
Connecting every document without managing versions and permissions.
Testing only on prepared demos.
Confusing a fluent answer with a proven one.
Hiding the limits from users to make the tool look more impressive.

FAQ

Does RAG eliminate hallucinations?
No. It provides external, updatable context, which can reduce some errors. Retrieval and generation still have to be evaluated.

Is a more powerful model enough?
Not to guarantee the reliability of a business process. Sources, scope, tests and controls are still needed.

Should you put a warning everywhere?
A warning does not replace system design. Users should understand the limits, but sensitive decisions mainly need to be blocked or validated.

A good assistant does not always answer

It is the one that also knows when not to.

I am Fabien Kost, an independent developer in France (ClosiQode). I build business AI assistants that answer from validated documents and say so when they cannot find the answer. More on how I design them (in French).

Sources (in French): CNIL, questions and answers on using generative AI systems and first guidance on deploying generative AI.