Optical Character Recognition has been a solved problem for about thirty years, except that it has not. Clean printed text on a white background? Any OCR engine handles that with near-perfect accuracy. A photo of a receipt taken at an angle in bad lighting with creases and shadows? That is where things get interesting, and where the gap between good and great OCR still matters.
At its core, OCR is a pipeline with distinct stages. Understanding each stage helps explain why some engines succeed where others fail.
The first stage is binarization: converting the image to pure black and white. This sounds trivial, but it is arguably the most important step. Simple global thresholding (everything above a certain brightness becomes white, everything below becomes black) fails badly on images with uneven lighting. Adaptive thresholding, where the threshold varies across different regions of the image, handles real-world conditions far better. Sauvola's method and Niblack's method are two commonly used adaptive approaches, each with different strengths depending on the input.
Next comes deskewing: detecting and correcting any rotation in the text. Even a 2-degree tilt can significantly degrade recognition accuracy because the character segmentation step expects roughly horizontal text lines. Most engines detect the skew angle by analyzing the horizontal projection profile of the binarized image or by using Hough line detection, then rotating the image to correct it.
After deskewing, the engine performs layout analysis and segmentation. It needs to identify text regions, separate them from images and graphics, determine reading order, detect columns, and isolate individual text lines. Within each line, it segments individual characters or, in more modern engines, groups of characters. This is where complex layouts like newspapers, forms with boxes, or documents with mixed orientations cause problems. An engine can have perfect character recognition but still produce garbage output if it reads the columns in the wrong order or merges text from adjacent regions.
Finally, the recognition step takes each segmented region and identifies the characters. This is where Tesseract's evolution tells an interesting story.
Tesseract started life at Hewlett-Packard's Bristol lab in the 1980s. HP developed it as a commercial OCR engine, and it was considered one of the most accurate available at the time. In 2005, HP open-sourced it. Google picked it up in 2006 and has been the primary sponsor of its development since.
Through version 3.x, Tesseract used a traditional approach: it matched character shapes against trained templates, analyzing features like the number and position of horizontal crossings, character topology, and geometric properties. This worked well for clean, well-segmented text but struggled with anything degraded or unusual.
Tesseract 4, released in 2017, added an LSTM (Long Short-Term Memory) neural network-based recognition engine. This was a fundamental shift. Instead of recognizing individual characters in isolation, the LSTM processes entire text lines as sequences, using context to resolve ambiguous characters. Is that an "l" or a "1" or an "I"? The LSTM uses surrounding characters and learned language patterns to make a better guess. The accuracy improvement was dramatic, particularly on degraded or noisy text.
Tesseract 5, the current version, refined the LSTM architecture and improved training tools but kept the same fundamental approach. It also maintained backward compatibility with the legacy engine, which you can still invoke for specific use cases.
Here is what surprises most people: Tesseract, a free open-source engine, still produces accuracy comparable to or better than many commercial OCR APIs for standard printed text recognition. I have run comparisons across document types, invoices, receipts, book pages, screenshots, and on clean to moderately degraded input, Tesseract consistently matches or beats services that charge per page. The commercial APIs tend to pull ahead on highly degraded images, handwriting, and complex multilingual documents where their larger training datasets and more sophisticated preprocessing pipelines give them an edge.
The key insight is that preprocessing is where most of the gains are. The same image can produce 60% accuracy or 95% accuracy depending on what you do before sending it to the OCR engine. Here is a practical preprocessing pipeline that dramatically improves Tesseract results.
Start with rescaling. Tesseract works best when text is roughly 30-33 pixels tall. If your image has small text, upscale it. Bicubic interpolation works well for this.
Apply contrast enhancement. Histogram equalization or CLAHE (Contrast Limited Adaptive Histogram Equalization) normalizes uneven lighting and makes text stand out from backgrounds.
Denoise with a median filter or Gaussian blur. This removes speckle noise and scanner artifacts without destroying text edges, as long as you keep the kernel size small, 3x3 or 5x5.
Deskew using the Hough transform to detect the dominant line angle, then rotate to correct it. Even if Tesseract has its own deskewing, doing it explicitly gives you more control.
Finally, binarize using adaptive thresholding. This gives Tesseract the cleanest possible input.
Where does Tesseract fail? Handwriting recognition is still weak. Complex page layouts with floating images, sidebars, and non-linear reading order can confuse the layout analysis. And documents in scripts with complex ligatures (Arabic, Devanagari) require properly trained language models that are not always well-optimized.
For browser-based applications, Tesseract.js is a pure JavaScript port that runs entirely in the browser using WebAssembly. It is slower than native Tesseract but eliminates server round-trips entirely. The image data stays on the user's device, which matters for privacy-sensitive documents. The Canvas API provides the preprocessing pipeline: draw the image to a canvas, manipulate pixel data for contrast and thresholding, and feed the result to Tesseract.js.
If you need to extract text from an image without installing anything, I built an image-to-text tool that runs OCR directly in your browser. No uploads, no server processing, everything stays local.
OCR in 2026 is not about finding the fanciest engine. It is about understanding the pipeline, preprocessing your images properly, and choosing the right tool for the specific type of document you are working with. For most printed text use cases, Tesseract, free and open-source, is still the answer.
I'm Michael Lip. I build free tools at zovo.one. 350+ tools, all private, all free.
Top comments (0)