DEV Community: Julie Love

Can you vibe code a DOCX editor for $200?

Julie Love — Thu, 10 Sep 2026 20:44:10 +0000

Quick Answer: A weekend with Claude will get you a browser-based editor that opens a .docx, renders the paragraphs, lets you type, and saves it back out. That part is real and it is genuinely impressive. What the $200 does not cover is the rest of the OOXML spec, the security work that comes with accepting files, the round-trip problem, and someone who understands the whole thing for the 2am fire drill. Generating code got cheap. Owning code did not.

I recently found this comment under a demo video for the Apryse DOCX editor:

"You can vibe code one better than that one with Claude so it will probably cost you 200 and you are better off doing that way since people are definitely gate keeping code that is technically free to make."

I think there are a lot of people excited about the possibilities of where we are with vibe coding. I hate to ruin a good buzz but can't resist bringing a bit of sobriety into this conversation. Sadly, the reality of owning code is never quite as fun as the imagination phase.

What does a weekend and $200 actually get you?

Quick Answer: Real code that runs. A .docx is a ZIP file full of XML, so pulling paragraph text out of document.xml is genuinely an afternoon's work, and what comes back opens real files and saves them again. But there is quite an expanse between "working" and great code. The distance between them is ECMA-376: several thousand pages, of which your weekend covered whatever your test file happened to use. Almost none of the rest is exotic.

Here is what is sitting past that quick vibe output, and none of it is unusual. This is just Tuesday for anyone who works on documents:

Numbering. Abstract definitions, concrete instances, per-level overrides, and restart semantics that nobody agrees on.
Styles. Inheritance running from document defaults, through linked and latent styles, down to direct formatting on a single run.
Section properties. Which can change halfway down the file, because of course they can.
Tables. Vertical merges expressed as continuation flags on the cells below rather than on the cell doing the merging.
Fields. Storing both a cached result and the recipe to recompute it, so you have to decide which one you believe.
Everything else. Footnotes, content controls, equations, bidirectional text, embedded objects.

None of that is hidden from you. It is just a whole lot more work than people are expecting to do when they "open a .docx".

The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time. - Tom Cargill, Bell Labs

Why does every "can it also do..." request cost more than it sounds?

Quick Answer: Because the cost of saying yes collapsed but the cost of the resulting system did not. Version 0.1 gets shown around, the requests come back, each one sounds like an afternoon, and each one lands somewhere much larger than the person asking realizes.

Where those small requests actually land:

"Can it do numbered lists?" Abstract versus concrete numbering, level overrides, restart rules, list styles.
"The indents look wrong." Indent resolution and tab stops. Congratulations, you are now maintaining a layout engine.
"Print it exactly like Word." Line breaking, font metric substitution, widow control, keep-with-next, rows splitting across pages.
"Keep the tracked changes." Insertion, deletion, move and format revisions, and every single edit your editor makes now has to be expressible as one of them.
"Where did the comments go?" See the round-trip section. This one is the mean one.

Six weeks later there are twenty thousand lines in the repo and you have read maybe three thousand of them. The three thousand that broke.

What if you don't control what is uploaded?

Quick Answer: The moment you accept uploads, you have an endpoint consuming arbitrary compressed XML sent by strangers. That is one of the more hostile input classes in computing.

A few of the vulnerability classes to consider:

XXE. Your XML parser will read files off your server unless somebody explicitly told it not to.
Decompression bombs. A few kilobytes of ZIP that expands to gigabytes.
Zip slip. Archive entries containing ../, writing wherever they like the second you extract to disk.

Every one of those appears in the CVE history of every mature document library, found the expensive way - by someone else, years ago.

What is the round-trip problem?

Quick Answer: The failure people brace for is a crash. The failure that actually happens is silent. If you parse a document into your own model and serialize it back out, it drops everything your model did not know about - with no error report in sight.

Someone opens a contract in your editor, changes one word, saves. The tracked changes are gone. So are the comments, the custom XML bindings a downstream system reads, the content controls, and the numbering that survived four rounds of legal review.

The file opens fine. Word does not complain. It simply is not the same document anymore, and nobody finds out until the moment it matters most.

Preserving the parts you do not understand is much harder than parsing the parts you do. No prompt is going to tell you which parts you did not understand, because by definition, you didn't ask.

Can you maintain code you have never read?

Quick Answer: Software cost lives in maintenance, and maintenance cost is a function of comprehension. Code generation drives the writing cost way lower but it does nothing for the comprehension cost. In fact it most likely makes comprehension worse, because writing the code is how you would otherwise have learned the code.

The question at the point of failure is never "can Claude fix this." It is "do I understand this system well enough to know whether that fix is right, or whether it just moved the bug somewhere I am not currently looking."

Writing scales with the model. Debugging still scales with you. Even after coding for many years, I still struggle to remember exactly what I meant in every line of code that I actually wrote - much less trying to decipher where the AI went wrong on something I had nothing to do with.

Is anyone actually gatekeeping this?

Quick Answer: No, and this is the part of the comment that holds up the least. Plenty of sources out there. LibreOffice is open. So are docx4j, python-docx and a dozen readable OOXML implementations. Nobody is hiding the code, because the code was never the scarce part.

Everyone deserves to be paid for their work. When you make the call to buy vs build, it's rarely because no one is capable of building it. You pay to keep your dev resources focused on solving your own business problems. You pay for that other company to be the expert for your team.

That expert brings years of experience with:

The zillions of real documents produced by 20 years of Word versions and every third-party generator that emits technically invalid XML which Word renders anyway.
The bug report from the customer whose merged table cells collapse only at 120% zoom.
Somebody to call when a filing renders wrong just before the deadline.
The unglamorous rest of it: security review, indemnity, accessibility conformance.

This is not about a goblin wildly guarding lines of code while sitting on a pile of your money. This is just the part you cannot prompt into existence because it is not code. It is lived, often traumatic, experiences that you are paying to skip entirely. ...and it's often worth every saved minute of your time and avoided suffering.

Toss this problem back over the fence? Yes please, and thank you.

So when should you build it yourself?

Quick Answer: Frequently. For a great many jobs, spending $200 and a weekend is exactly the right call. Not every tool needs to reach perfection or hold up under long term testing and maintenance stress.

Build it yourself when:

You control the whole workflow, use your documents, and you know exactly how the in and out points need to function.
Your company or workload is small, no need to over-engineer for something that doesn't need to scale.
"Wrong" costs a redo, not a lawsuit.
Nobody outside the building is uploading anything, or more specifically, you can entirely trust the users of the system.
You would be happy to delete the whole thing next quarter. Throw it up, try it out, and no one is upset when you scrap it for the next iteration.

Reach for something battle-tested when:

The inputs are other people's documents, arriving from anywhere, in any state.
The data is other people's data, with residency or compliance rules attached.
The deadline is other people's deadline, and the failure lands on them. ...or worse, lands on you.
You need enterprise level scaling. There is no point in starting over on trying to account for every edge case an enterprise-level roll out will turn up.

Where should you start?

Build the weekend version anyway. It is the fastest way to find out how much of the format your use case actually touches.
Round-trip a really complicated document early. Tracked changes, comments, content controls. Save it, reopen it in Word, and see what quietly vanished.
Read the code you shipped. All of it, or at least enough to know which parts you have not read.
Decide which 90% you are in for. The first 90% is a weekend now, which is legitimately worth being excited about. The second 90% is the whole rest of your job.

If you get to step 4 and only have interest in that first 90%, then that is when a document SDK earns its keep. You can try ours for 30 days at docs.apryse.com/guides/get-started.

If you're on step 4 and think, "I've got this," go forth and prompt away! Then come back and tell me what broke. I have no doubt the answers here will keep changing as things advance - I'm super interested to see where we are.

Disclosure: I work for a company that sells document SDKs, so weigh all of the above accordingly. The vulnerability classes and the round-trip problem are real either way. Go check them against whatever you build.

Self-hosted OCR without a cloud API: where Tesseract stops working

Julie Love — Wed, 09 Sep 2026 18:49:17 +0000

Cover image caption: Puppy with "free" sign and collar that reads "self-hosted OCR". Next to that a long receipt titled "Maintenance" with many cost items for dogs.

Quick Answer: Ask your favorite AI how to run OCR server-side without a cloud API and you will most likely hear about Tesseract. That is a reasonable answer for clean, straight, single-column English print. It stops being reasonable once you read the weak spots Tesseract's own documentation flags, and start counting developer time as the cost that it is. A production OCR SDK, like Apryse, earns its license fee in exactly those places.

I asked ChatGPT, Claude and Gemini that question in several forms. Every one said Tesseract, usually with PaddleOCR and EasyOCR as close seconds. Not one mentioned that Tesseract publishes a page called "Improving the quality of the output" which, if we are being honest, is a long list of what it just does not do well.

It's pretty easy to see why these top the lists. Because Tesseract is free, genuinely good, 30+ years old, and has more written about it than any other OCR engine on this lovely little planet. Models synthesize consensus, and people sure love to write and talk about things that are "free". But it's distinctly not the only answer, and you need to make sure it's actually the right answer to YOUR question.

Where does Tesseract stop working?

Quick Answer: Five places, all of them documented by the Tesseract project itself. Skewed pages, tables, uneven page backgrounds, tightly cropped or heavily bordered regions, and anything that is not a sentence of dictionary words. None of these throw an error, they just return bad data - and that is usually the worse outcome.

Straight from the Tesseract documentation:

Skew. "The quality of Tesseract's line segmentation reduces significantly if a page is too skewed." Deskew is YOUR problem to fix, not the engine's.
Tables. "It is known tesseract has a problem to recognize text/data from tables without custom segmentation/layout analysis." Their wording, and the suggested remedy is a GitHub thread.
Uneven backgrounds. Binarisation runs internally with Otsu, and the result "can be suboptimal, particularly if the page background is of uneven darkness." Which describes most scans older than a decade.
Borders and crops. Too little border causes problems. Too much on a small region returns an empty page. The fix is cropping to a 10 pixel margin, by hand, per document type.
Non-prose text. Tesseract is "optimized to recognize sentences of words," so for part numbers and receipts the advice is to disable the built-in dictionaries. Invoices are mostly not sentences.

Then you add the 14 page segmentation modes you have to specify with --psm, because Tesseract will not determine the layout for you. Tesseract is an excellent recognition engine wrapped in a pipeline you are expected to build yourself, out of Leptonica, OpenCV or ImageMagick. And things you build, you must maintain.

What does "free" OCR actually cost?

Quick Answer: The license does not cost money, but you pay in developer time. The pre-processing pipeline, the per-document-type tuning, the model curation and the retraining are all work somebody on your team does instead of working on your product. Free OCR moves the cost from pocketbook to time - and that's some of the most expensive currency you have.

One of our customers came to Apryse having already built a full, working Tesseract solution. It was running just fine. The problem was that it was costing them 2-3 developers to keep it running. 2-3 devs who were constantly having to try and learn new things about the OCR ecosystem and how to keep it working. 2-3 devs who were supposed to be improving the PDF software that was actually the company's business.

They did not switch because Tesseract could not read their documents. They switched so Apryse was their expert, and responsible for the OCR upkeep. It meant their team could get back to focusing on their expertise and the internal roadmap items.

Once you look for it, you can see where those developer-hours go.

The pre-processing pipeline. Deskew, despeckle, binarisation, border handling, alpha channel stripping.
Language bindings. Tesseract is a C++ program. Reaching it from Python, Java, .NET or Node.js means a third-party wrapper, and pytesseract works by shelling out to the command-line binary as a subprocess. Those wrappers are community-maintained on their own release cycles, so they are one more dependency sitting between you and your text.
Per-document-type tuning. Choosing among 14 page segmentation modes, toggling dictionaries off for invoices and part numbers, cropping to the right margin. Every new document type restarts this.
Model curation. Tesseract publishes three sets of model files. tessdata_best is the most accurate and the only one you can fine-tune from, and it is also the slowest. tessdata_fast is documented as the least accurate, and it is what ships with Linux distributions. Most people are running the least accurate models and were never told. Each language is a separate file you download and manage.
Retraining, when preprocessing runs out. The training guide is honest about the price: the shipped Latin models were built from roughly 400,000 text lines across about 4,500 fonts, a training run takes "a few days to a couple of weeks," and it officially "only works on Linux." Retraining from scratch is called "a daunting task," with the warning that you will likely end up with a model that does well on your training data and badly on your real data.
Support. There is no one to call. Tesseract's own training guide asks you not to file training problems as GitHub issues and directs you to a mailing list instead.

To their credit, the project tells you not to retrain: "unless you're using a very unusual font or a new language, retraining Tesseract is unlikely to help." Honest advice, and it also leaves you nowhere to go on the day preprocessing has not closed the gap and the invoices still have to be processed.

For comparison, the Apryse v12 default engine covers 80+ languages including CJK from one model set. No files to curate, no accuracy tier to pick, no training step, and a support contract when it misbehaves. (Not to mention tools like Smart Data Extraction that help you do more with your OCR.)

None of this makes Tesseract a bad choice. It makes it a choice with a staffing plan attached. Price both.

What does a production OCR SDK give you that Tesseract does not?

Quick Answer: Pre-processing built into the engine rather than bolted on in front of it, several recognition engines behind one SDK so you can switch per document type, structured output with coordinates, and a support contract. You are paying for the pipeline around the recognition, not the recognition alone.

With the Apryse Server SDK, deskew and despeckle happen inside the engine. Since v12 the default engine runs on deep learning neural networks, bringing roughly 16% better word recognition, 80+ languages including CJK, and no GPU requirement. There are other engines available when your pages are unusual, which my OCR article covered in detail, and swapping between them is a one-line change rather than a new integration.

The bindings are native rather than wrappers. The same call runs from C++, .NET, Java, Python, Node.js, Go, PHP and Ruby, on Windows, Linux and macOS:

opts = OCROptions()
opts.SetEngine("default")      # "default", "alternative" or "iris"
opts.AddLang("eng")

# Inspect the result as JSON before it becomes a searchable PDF
json = OCRModule.GetOCRJsonFromPDF(doc, opts)
OCRModule.ApplyOCRJsonToPDF(doc, json)

That JSON is worth pausing on. You get recognized text plus coordinates, which means you can filter low-confidence words, reject a page, or map any answer back to the exact spot it came from, before any of it reaches your database or your LLM.

With Apryse you also get a multitude of tools that are ready to handle most any document need - AND you have Support. When something isn't going right, there's real help and a team to back you up.

Does self-hosting OCR solve compliance?

Quick Answer: While it does remove a whole category of problems, that's not the same as solving compliance for you. Running recognition in your own environment means no third-party processor in scope, no data residency question and no per-page metering. The rest of your obligations under HIPAA or GDPR are still architecture you have to build.

You may still have ninety-nine problems, but data residency ain't one.

Worth being precise, because vendors are routinely vague about it. An SDK is not "HIPAA certified." HIPAA compliance, as I'm sure you're aware if this is your area, is a property of your whole system. What self-hosting changes is that documents never leave your environment, so there is no processor agreement to negotiate, no cross-border transfer to justify and no third party whose breach becomes your incident.

The safety department of a large city came to us having done real homework, testing a range of options against two hard requirements: no training with their data, and nothing leaves their systems. The AI-powered engine in v12 met both and gave them the best accuracy of everything they tested.

And while we're on security - for those who get to be part of those ever fun security audits, nice to know that Apryse holds ISO/IEC 27001:2022 certification since 2018 and completes annual SOC 2 Type II audits. Usually keeps those security form pain points to a minimum.

Where should you start?

Quick Answer: Fix your input, then benchmark Tesseract honestly against your own worst documents before you spend anything. If it clears your bar and you have the headcount to maintain it, you've got your answer. If it fails in the ways its own documentation predicts or your budget does not account for maintenance efforts, you have a business case rather than a preference.

Check your DPI before anything else. Below 300 and you are debugging the scanner, not the engine. On a rasterized PDF, divide image pixels by page size in inches.
Build a test set of your worst fifty pages. Rotated, stained, multi-column, tabular. Good scans don't prove out your real cases.
Run Tesseract first, with real effort. Deskew, strip the alpha channel, pick the right --psm. Make sure it's a solid test to compare against.
Measure Character Error Rate and Word Error Rate per document type. Accuracy and how you manage it matters.
Price the developers, not just the license. Make some real estimates on what it is going to take to keep up with this longterm. When you are your only support team, it makes a difference.
Test the Apryse engines on the same set. Every module is in the 30-day trial with no page cap.

Tesseract is not the wrong answer. It is a good answer to "what is the best free OCR engine." But what if what you really need to know is "what will do the best job on my docs with the least longterm cost to my dev team"?

Models will only answer a question explicitly asked. Make sure you're asking the question you need answered.

Happy Coding!

Julie Love is Director of Developer Experience at Apryse. She has spent an entire career in tech spanning everything from burning CD screen savers to mastering proprietary code, sales engineering and a variety of in between. Puns always intended.

ICR handwriting recognition in Python: when do you need self-hosted instead of a vision LLM?

Julie Love — Wed, 09 Sep 2026 18:41:44 +0000

Quick Answer: Self-hosted ICR is a recognition engine trained on handwriting that runs inside your own network and can be set to return text plus coordinates. A vision LLM reads handwriting using an enormous training set and usually returns a block of text from someone else's server. Pay close attention to your needs around deployment, cost and audit-ability to make the best choices for you.

I've heard learning cursive in school is a thing of the past, much like my spiral notebook and my handwritten to-do lists.

Yes, there are usually doodles in my notebooks as well. It is great for those of us with visual and spatial memories who need to put ideas together on paper before they make sense anywhere else. (At least that is my justification and I am sticking to it.)

Luckily, kids these days do not need to learn cursive while we have so many cool digital toys now.

Let's break it down. (Due to the limitations of the written word, you will need to mentally cue a great late-80s hip hop track and a dance of your choice.)

Why is handwriting so much harder to read than printed text?

Quick Answer: Printed text follows solid, definable, repeatable rules. Handwriting breaks five of them at once, which is why handwriting support is not just a quick add-on to a printed-text engine. It has to be a separate model trained on separate data.

Everyone writes differently. This may be one of the world's largest understatements. Fonts are finite. Handwriting variation is not. (in the uncountable infinity category, for my fellow math nerds.) Even the same person writes the same letter differently depending on the number of lattes consumed and how annoying the traffic was on their way to fill out the form.
Cursive has no gaps. No need to mind the gap here, because there are none. No reliable boundary between characters means nowhere obvious to cut.
Words wander. Print sits on the line. Handwriting drifts, slants, and takes a little trip to any nearby region that looks fun or available.
Training data is scarce. Labeled handwriting collections are far smaller and rarer than printed text. Side quest, handwriting sample set: IAM Handwriting Database
Every source looks different. A doctor's note has nothing in common with an 18th-century census record. And let's be real, you thought teaching a computer to beat a human at Go was hard? There is no quantity of doctors' notes large enough to solve that great mystery of life.

Then there are forms, which pile all of the challenges into one place. Typed labels that have to relate to handwritten answers, deciphered separately and reassembled correctly.

One practical note before you start building: If you have any influence over the forms themselves, use it. Grid-style boxes force people to space out their letters and define a usable area for each field. If that's too limiting, keep your field boxes black (non-scan colors like red, pink or cyan can drop out completely in the pre-process), and your field labels above or to the left of the fields (not inside or touching the boundaries). Blank lines invite the sprawl that breaks everything downstream. Design with scaffolding wherever you can.

What is ICR, and how is it different from OCR?

Quick Answer: Intelligent Character Recognition (ICR) is a neural network trained on handwriting examples rather than font definitions, so it can read writing it has never seen before. Optical Character Recognition (OCR) is based on matching the shape of a letter and expected characteristics - based on printed fonts.

Also of note, this is ever evolving. AI-powered OCR can sometimes pick up handwriting pretty well. As AI continues its expected exponential growth, will be interesting to see where this all ends up.

But as of 2026 as I write this, OCR and ICR each are the right tool for their different jobs, and there are many cases where you may need both. When you are building your self-hosted solution, make sure to review which engines are available. Tools like the Apryse Server SDK have lots of options (a few flavors of OCR and ICR), which make it easier to test multiple options out at one time.

Let's take a look at a simple Apryse SDK ICR implementation.

from apryse_sdk import *

PDFNet.Initialize(LicenseKey)
PDFNet.AddResourceSearchPath("../../../ICRModule/Lib/")

if not HandwritingICRModule.IsModuleAvailable():
    print("Handwriting ICR module not available. Download it from https://docs.apryse.com/core/guides/info/modules#handwriting-icr-module")
else:
    doc = PDFDoc(input_path + "intake_form.pdf")

    opts = HandwritingICROptions()
    opts.SetPages("2-3")

    # Run ICR and apply the result as hidden text
    HandwritingICRModule.ProcessPDF(doc, opts)

    doc.Save(output_path + "intake_form_searchable.pdf", SDFDoc.e_linearized)
    doc.Close()

PDFNet.Terminate()

Full and beautiful sample, complete with C#, C++, Java, PHP, and all the favorites: https://docs.apryse.com/core/samples/icrtest

SetPages is doing quiet work here. Running ICR only on the pages that contain handwriting is faster. Skip sections like the signature box and use OCR on typed pages. (Although, it might be an entertaining exercise to see if what signatures come out as. lol)

Developer note: Apryse ICR is a separate module you download from https://docs.apryse.com/core/guides/info/modules#handwriting-icr-module, and it maps to its own add-on to your Server SDK license. If you're using this, make sure to have it on your list of topics for sales.

When should you use a vision LLM instead?

Quick Answer: When volume is not huge and your documents are allowed to leave your network, then you can't beat the big players in the game. Vision LLMs are part of the AI ecosystem that is supposed to keep improving exponentially. Everything is subject to change quickly. Keep testing.

Because if the growth and change in the industry, nothing I type can be counted on for any real length of time. But here are a few high level considerations:

Reach for a vision LLM when:

Legal discovery, a few hundred pages. Low volume contains the cost, and the documents are public.
Historical archives and freeform cursive, where searching beats structuring. Broad training data copes with wildly unpredictable inputs, and you want a searchable layer rather than structured fields.

Reach for self-hosted recognition when:

Medical intake forms under HIPAA. Data cannot leave the network. That is the whole decision.
10,000 forms a night. Per-page costs become significant enough to be worth the extra setup.
Anything an auditor will review. Recognition is deterministic. Same document, same output, every time.

There is one comparison that needs no benchmark. A recognition engine is deterministic: same document, same output, every time. A generative model is not. If you have an auditor involved with your output, making sure you know the output is repeatable is important.

Healthcare keeps showing up for a reason. Handwritten intake forms, prescriptions and clinical notes are where patient data workflows pile up, and the time saving is measurable. A prospective multi-center study in Critical Care found OCR cut data entry time in intensive care units by a mean of 43.9%. Reported figures for ICR in claims and e-prescription workflows run as high as a 70% reduction.

In the end, there are a lot of numbers to throw around for these tools, and they are only as good as the time period they were printed. Technology is changing almost daily - usually improving these numbers for the better.

To sum up: manual data entry stinks. Make it easier, and both time needed and accuracy are better.

How do you get from a scanned form to structured JSON?

Quick Answer: ICR gets you a searchable PDF and a JSON file full of words and coordinates. It does not get you {"date_of_birth": "1985-03-14"}. Turning recognized words into named fields can take a few more steps.

That position data is the part people undervalue. Text alone is a wall of words. Working in the Apryse Server SDK, here's one approach to that work:

Find the fields. The Data Extraction module, run with the form engine, returns each field's position, type and a confidence score.
Convert those positions into zones. Field detection measures from the top-left. PDFs measure from the bottom-left. Subtract each Y value from the page height, or ICR will confidently read the wrong part of the page.
Run ICR on just those zones. Faster than the whole page, and it keeps the engine away from the signature box.
Apply the results to a clean template. Key-value extraction does not work on scanned documents, which is the gotcha that derails people. Apply the ICR output onto a blank copy of the form instead, at zero opacity.
Extract the pairs, then check the doubtful ones. Route anything below your confidence threshold to a person.

Roger Dunham's full worked example walks all of it in JavaScript, including the coordinate flip and the template trick.

It is a lot of steps for one form. That is the honest trade. You get a pipeline that runs entirely inside your own network, on documents that are not allowed to leave it. You pay for that in set up, but you also get to dial into exactly what you need.

What confidence threshold should you set?

Quick Answer: Every recognition engine returns a per-field confidence score alongside the text. Your threshold is the line below which a field stops going straight into the database and goes to a person instead. Around 90% is a common starting point, tuned against your own documents.

This is the mechanism behind human-in-the-loop, and it is worth being concrete about, because "we'll add review later" is how review never gets added.

Nothing is going to be 100% accurate. Set the threshold too high and you drown your reviewers in fields that were fine. Set it too low and wrong data lands in the record silently. The only way to find your number is to run a few hundred of your own documents, compare against ground truth, and look at where the errors actually cluster.

Two things that make the number easier to live with.

Set it before you build the pipeline, not after you have discovered the problem in production.
Measure your reviewers too, because people are not 100% accurate either.

Remember the goal is always "better", not perfect.

Where should you start?

Quick Answer: Test your worst handwriting first, decide the data residency question early because it usually settles the engine choice, and pick your confidence threshold before you build.

Start with your worst examples. The messy intake forms, not the tidy ones. Good scans are easy once the bad ones work.
Measure Character Error Rate and Word Error Rate on your own documents. Doesn't matter what anyone else's tests showed, only what you see on what you're working with at the time you are starting.
Check whether your data is legally allowed to leave your network. That usually decides between ICR and a vision model on its own.
Decide your confidence threshold before you build the pipeline, and design the review queue at the same time. Everyone will thank you later for planning this upfront.
Redesign the form if you can. Boxes beat blank lines.
Get a trial key. Why not start with testing Apryse? Every module is included in the 30-day trial with no page cap: docs.apryse.com/guides/get-started. Sample code for ICR is at docs.apryse.com/core/samples/icrtest.

With good recognition in place, all of those pieces of paper become sources your systems can actually use. Every visit your oldest client made is exhaustively documented and, better still, searchable. Your filing cabinets turn into a nostalgic history of useful data you can reach in seconds.

Happy Coding!

How does AI text recognition work? OCR, ICR and the Python pipeline that feeds your LLM

Julie Love — Wed, 09 Sep 2026 18:27:13 +0000

Quick Answer: AI text recognition converts images of characters into machine-readable text, and it runs before anything an LLM does. Classic OCR matches shapes against known fonts. AI-powered OCR replaces that template matching with a neural network. ICR is trained on handwriting. This is step 1 of 5, and arguably your most important step. Downstream tools can't repair what step 1 gets wrong.

AI is currently center stage, in the spotlight, on every single space anyone can half-correctly call a stage right now. The most shiny of new toys even us geeks have seen in ages.

So as is quite fitting with the time, the AI step in a document workflow gets all the attention. The recognition step gets glossed over, that is, until the answers start coming back wrong and it's time to dig into the why.

Here is what catches people up: they blame it on hallucinations. But the recognition step is just doing its job - reporting what the characters look most like on an imperfect document.

How does AI text recognition actually work?

Quick Answer: There are three major players in the game. Classic OCR compares pixels against known letter shapes and picks the closest match. AI-powered OCR swaps that template matching for a neural network, still aimed at print but far more forgiving of bad scans. ICR is trained on real handwriting, so has more capacity to correctly analyze every new handwritten letter instance.

Classic OCR. Optical Character Recognition assumes a standard font on a copy-machine-quality image. It expects the page to line up with its finite set of letter and font definitions. Think hall monitor with clipboard and a stopwatch - they get things done and keep it efficient, but you better fall in line with the exact expectations. With a clean print, it is better than 99% accurate - tough to beat for the job.
AI-powered OCR. Same target as classic OCR, but a neural network does the deciphering instead of a template. (think feature detection as opposed to rigid patterns) It handles the bad scans, complex layouts and unusual fonts that break a template matcher. So this is probably that hall monitor you liked a bit better - can look the other way on small exceptions, but largely still keeps things running as expected.
ICR. Intelligent Character Recognition learns character shapes from examples rather than matching templates. Lots and lots and lots of examples, which is what makes it good at educated guesses on handwriting. In the halls, this monitor knows that no two kids are exactly the same, leaving room for self-expression in an otherwise rigid rule book.

What works best? Only you, your examples and a bunch of testing can decide that. So having all of these options in one place, like the Apryse Server SDK, is definitely an advantage.

Apryse's default OCR module moved to deep learning neural networks in version 12.0, released July 2026. That brought roughly 16% better word recognition, much better tolerance of low-quality scans and complex layouts, no GPU requirement, and - my favorite part - a jump from 6 supported languages to more than 80.

Default OCR, AI-powered since v12. For print with poor scans, complex layouts, or non-English pages. 80+ languages and the most versatile of the set. Licensed under the OCR add-on.
Alternative OCR. For clean print, where speed and lean hardware matter most. Runs 2 to 3x faster. Also the OCR add-on.
IRIS OCR. For disconnected text regions like magazine covers and CAD drawings, where it handles fragmented layouts. Separate IRIS OCR add-on.
Handwriting ICR. Anything written by hand, and the best handwriting results of the four. Separate ICR add-on.

Developer note: these are each separate modules you download from https://docs.apryse.com/core/guides/info/modules, and they map to specific add-ons for your Server SDK license. When you're ready for the call with sales, make sure to mention which items you're using.

Where does recognition sit in an LLM pipeline?

Quick Answer: Right at the front, before anything interesting happens. The pattern people now search for as OCR LLMs is really a pipeline, and recognition is step 1 of 5. Everything after it inherits whatever it produced before.

Recognition. The page becomes text.
Chunking. That text gets split into passages.
Embedding. Passages become vectors.
Retrieval. A question pulls back the passages that look relevant.
Generation. The model answers using what it retrieved.

Nothing downstream can repair a problem in step one. An embedding model has no way to know that "rnedication" was "medication" before the scan blurred, and a retrieval step cannot find a passage that was never extracted.

What happens when recognition output is noisy?

Quick Answer: It fails quietly rather than loudly, which is the worse of the two options. Five failure modes account for most of it, and not one of them throws an error.

Character-level corruption. One misread letter creates a token the model has never seen, so the passage embeds strangely and stops matching the questions it should match.
Phantom whitespace. Words split or run together, breaking exact-match retrieval and keyword search.
Broken reading order. A two-column page read straight across interleaves two unrelated arguments into nonsense.
Lost tables. Rows and columns flatten into a stream of digits, so a figure detaches from whatever it was measuring.
Missing content entirely. Handwritten fields that classic OCR could not parse simply are not there, and nothing flags their absence.

That last one is the quietest failure of the set. An empty field looks identical to a field the patient left blank.

How do you get OCR output into an LLM?

Quick Answer: A tool like Apryse OCR returns structured JSON or XML containing the recognized text and its position on the page. OCR text alone is a wall of words. Text plus coordinates is something a database, a rules engine or a retrieval pipeline can act on, because you can map any answer back to the exact spot it came from.

There are always about as many options to accomplish the same task as stars in the sky (the reason I'd argue that coding is art), but let's take a look at the basic OCR code in Python with Apryse.

from apryse_sdk import *

PDFNet.Initialize(LicenseKey)
PDFNet.AddResourceSearchPath("../../../OCRModuleWindows/Lib/")

if not OCRModule.IsModuleAvailable():
    print("OCR module not available. Download it from https://dev.apryse.com/")
else:
    doc = PDFDoc(input_path + "scanned_invoice.pdf")

    opts = OCROptions()
    opts.AddLang("eng")

    # Get the recognition result as JSON before it goes anywhere
    json = OCRModule.GetOCRJsonFromPDF(doc, opts)

    # Inspect, filter or correct here, then apply it back to the document
    OCRModule.ApplyOCRJsonToPDF(doc, json)

    doc.Save(output_path + "scanned_invoice_searchable.pdf", 0)

PDFNet.Terminate()

For more detail and other languages, check out the formal OCR Sample code.

Important not to gloss over the notes here, particularly: "Inspect, filter or correct here," GetOCRJsonFromPDF gives you the recognition result before it becomes a searchable PDF, which means this is the spot to add some checks. Filter low-confidence words, strip special characters or reject a page outright before any of it reaches your chunker. If you are feeding an LLM, that inspection point is the best way to ensure higher quality or know when something needs more human review.

Why not just send the page to a multimodal model?

Quick Answer: Sometimes you should. If you have neither security nor cost-saving concerns, they are great. I've seen some really impressive accuracy. Unfortunately, the great result is not always repeatable and can get expensive. When deployment control, cost and predictability can't be compromised, that's the reason to run recognition yourself.

Your documents leave the building. For patient records or anything with a data residency requirement, that is where the conversation ends. No cloud API runs inside an air-gapped network.
Cost scales with pages. Fine for 50 documents. Budget line item at 50,000.
It may not give the same answer twice. Recognition engines are deterministic. Generative models are not, which matters when an auditor asks why one document produced two different records.
Latency. A round trip to someone else's server adds time you may not have.
No coordinates. A model hands you text. A recognition engine hands you text plus exactly where on the page it came from.

So just to be clear, this can be a big lift to put together. ChatGPT, Gemini or Claude tools will return results with less effort. But when you need to control everything, it's time to start building.

Where should you start?

Quick Answer: Test your worst documents first and measure the error rate on your own files. Settle any data residency questions before you pick an engine, because it may decide for you.

Start with your worst examples. Bad scans are the real test. The good ones are easy once worst-case is optimized.
Measure the error rate on your own documents. Character Error Rate and Word Error Rate are the standard measures, and no vendor's published figure tells you what yours will be.
Check whether your data is legally allowed to leave your network. That usually settles the engine question on its own.
Match the module to the page. Print, handwriting and collage layouts want different engines, and you may need several inside one project. Use the right tool for each job.
Read the recognition output before you read the model's answer. Most pipeline bugs live upstream of the part everyone is looking at.

If you've decided that you are building, definitely give Apryse a try. Get a trial key at docs.apryse.com/guides/get-started. The 30-day trial includes every module mentioned here, with no page cap - so you can put your worst scan through all four before you commit.

Happy Coding!

DEV Community: Julie Love

Can you vibe code a DOCX editor for $200?

What does a weekend and $200 actually get you?

Why does every "can it also do..." request cost more than it sounds?

What if you don't control what is uploaded?

What is the round-trip problem?

Can you maintain code you have never read?

Is anyone actually gatekeeping this?

So when should you build it yourself?

Where should you start?

Self-hosted OCR without a cloud API: where Tesseract stops working

Where does Tesseract stop working?

What does "free" OCR actually cost?

What does a production OCR SDK give you that Tesseract does not?

Does self-hosting OCR solve compliance?

Where should you start?

Further reading

ICR handwriting recognition in Python: when do you need self-hosted instead of a vision LLM?

Why is handwriting so much harder to read than printed text?

What is ICR, and how is it different from OCR?

When should you use a vision LLM instead?

How do you get from a scanned form to structured JSON?

What confidence threshold should you set?

Where should you start?

Further reading

How does AI text recognition work? OCR, ICR and the Python pipeline that feeds your LLM

How does AI text recognition actually work?

Where does recognition sit in an LLM pipeline?

What happens when recognition output is noisy?

How do you get OCR output into an LLM?

Why not just send the page to a multimodal model?

Where should you start?

Further reading