OCR Preprocessing (Overview)

Overview

OCR preprocessing is the set of image-conditioning steps applied to a scanned or photographed page before character recognition runs, with the goal of presenting the recognition engine with the cleanest, most regular image possible. In a typical pipeline the raw capture is rescaled to an adequate resolution, converted to a clean two-tone (binary) image, straightened (deskewed), cleaned of speckle and other noise, and analyzed to find and order the regions of text (page/layout segmentation). Only then does the recognizer classify glyphs into characters.

Preprocessing matters because OCR engines are sensitive to the quality and geometry of their input. The Tesseract project states that "certain types of noise cannot be removed by Tesseract in the binarisation step, which can cause accuracy rates to drop," and that "the quality of Tesseract's line segmentation reduces significantly if a page is too skewed, which severely impacts the quality of the OCR." In other words, errors introduced upstream (skew, poor thresholding, residual noise, mis-segmented columns) propagate downstream and cannot always be recovered by the recognizer.

This page is an overview that frames the four main preprocessing families and links to detailed per-technique treatments:

• Deskew — detecting and correcting page/text-line rotation.
• Binarization / thresholding — converting greyscale or color to black-and-white text/background.
• Despeckle / denoise — removing speckle, scanner artifacts, and background texture.
• Layout / page segmentation — locating and ordering text blocks, lines, and words.

History

Preprocessing has been intrinsic to automated text recognition since document-image analysis became a research discipline. Several foundational threads are especially well documented.

• Global thresholding. Nobuyuki Otsu's 1979 paper "A Threshold Selection Method from Gray-Level Histograms" (IEEE Transactions on Systems, Man, and Cybernetics) introduced an automatic, non-parametric method for choosing a single global threshold from an image's grey-level histogram by maximizing between-class (equivalently, minimizing within-class) variance. Otsu's method remains one of the most widely used binarization baselines.
• Local adaptive thresholding. Because a single global threshold fails on pages with uneven illumination or aged backgrounds, local methods compute a per-pixel threshold from the local neighborhood's statistics. Niblack's approach (described in An Introduction to Digital Image Processing, commonly cited as 1986) computes thresholds from local mean and standard deviation but leaves substantial background noise; Sauvola and Pietikainen's "Adaptive document image binarization" (Pattern Recognition, vol. 33, 2000) refined this to suppress background noise, and it is now a standard reference method for document binarization.
• Layout analysis. The X-Y-tree / recursive X-Y-cut lineage of top-down page segmentation traces to Nagy and Seth's "Hierarchical representation of optically scanned documents" (1984); the full document-analysis system is often cited as Nagy, Seth and Viswanathan, "A Prototype Document Image Analysis System for Technical Journals" (IEEE Computer, 1992). Top-down (splitting) and bottom-up (connected-component grouping) segmentation strategies both trace to this era.

On the engine side, Tesseract was developed at Hewlett-Packard Labs between 1985 and 1994, released as open source by HP in 2005, and subsequently maintained with Google's sponsorship from 2006. Version 4 introduced an LSTM-based recognizer (v4.0.0 final released in 2018), and version 5.0.0 (released 30 November 2021) added two Leptonica-based binarization options — Adaptive Otsu and Sauvola — alongside its default Otsu step.

The DIBCO / H-DIBCO competition series, held in association with ICDAR/ICFHR beginning with the first competition at ICDAR 2009, established public benchmark datasets and ground-truth for evaluating document-image binarization, reflecting how central this preprocessing step is to the field.

How it works

A representative preprocessing sequence proceeds as follows.

1. Acquisition and rescaling. The page is captured (scanner or camera) and, if necessary, resampled. Tesseract's guidance is that it "works best on images which have a DPI of at least 300 dpi, so it may be beneficial to resize images." 2. Binarization / thresholding. The greyscale or color image is reduced to a binary image in which text pixels are separated from background. Tesseract performs this internally with Otsu by default, but "the result can be suboptimal, particularly if the page background is of uneven darkness," which is why performing binarization as a controlled, separate step is often recommended. 3. Deskew / rotation correction. The dominant text angle is estimated and the page rotated so text lines are horizontal. Tesseract's Orientation and Script Detection (OSD) can report orientation, writing direction, and a deskew angle. 4. Despeckle / denoise. Isolated speckles, scanner streaks, and background texture are removed. Related morphological operations (dilation/erosion) can also compensate for ink bleed or overly thin/bold strokes in degraded documents. 5. Border handling. Dark scan borders can be misrecognized as characters; conversely, missing whitespace margins can hurt recognition. Tesseract suggests adding roughly 10 pixels of border, while warning that overly large borders around a single glyph or word can also cause problems. 6. Layout / page segmentation. The engine identifies text regions, columns, lines, and words, and establishes reading order. Tesseract exposes 14 page-segmentation modes (PSM), numbered 0–13, via --psm, and layout analysis through APIs such as AnalyseLayout() and GetComponentImages() (block/line/word iterators).

Tesseract relies on the Leptonica image-processing library for much of this image handling.

Approach and techniques

Binarization / thresholding

• Global (one threshold for the whole page): Otsu's method is the canonical example — automatic and parameter-free, well suited to clean, uniformly-lit pages with a roughly bimodal histogram.
• Local / adaptive (per-pixel or per-region thresholds): Niblack, Sauvola, and Adaptive Otsu compute thresholds from local statistics; these are preferred for degraded, unevenly-lit, or historical documents where a single global threshold fails. Sauvola commonly uses parameters around k = 0.5 and R = 128.
• Trade-offs are documented: Otsu degrades on non-uniform backgrounds; Niblack leaves background noise; Sauvola can thin or break characters.

Deskew — estimate skew angle (for example via projection profiles, the Hough transform, or connected-component alignment) and rotate. Keeping text lines horizontal is a precondition for reliable line segmentation.

Despeckle / denoise — connected-component size filtering to drop tiny speckles, morphological opening/closing, and border/edge cleanup. Some noise types are explicitly outside what the recognizer can fix at binarization time.

Layout / page segmentation — top-down (recursive X-Y cut and other projection-profile splitting) versus bottom-up (connected-component grouping/merging), plus hybrid methods; the output is an ordered set of text regions, lines, and words feeding the recognizer.

Accuracy and quality factors

Factors documented to affect OCR accuracy through preprocessing include the following.

• Resolution / DPI — below roughly 300 dpi, recognition quality tends to fall (Tesseract guidance).
• Skew — excessive skew degrades line segmentation and therefore OCR output (Tesseract).
• Binarization quality — an uneven background can make the default Otsu threshold suboptimal; choosing an adaptive method for degraded pages materially changes downstream results.
• Residual noise — some noise cannot be removed at the binarization step and drops accuracy (Tesseract).
• Character rendering — very thin, very bold, or serifed characters, and ink bleed, affect recognition; morphological correction can help.
• Borders — dark scan borders can be misread as characters; inadequate margins can also hurt.

The DIBCO/H-DIBCO benchmarks exist specifically to quantify binarization quality against ground truth, underscoring that binarization is a measurable determinant of document-analysis outcomes. Precise, generalizable accuracy percentages are not cited here because they depend entirely on dataset, engine, and settings.

Advantages

• Improves the reliability of downstream recognition by regularizing input (straight, clean, well-thresholded, correctly ordered).
• Lets practitioners exert explicit control — for example, choosing and tuning a binarization method rather than accepting an engine's internal default — which the Tesseract documentation notes can yield better results on difficult pages.
• Modular: each step (deskew, binarize, denoise, segment) can be evaluated and swapped independently, and benchmark suites such as DIBCO exist for at least the binarization stage.

Limitations

• Not a cure-all. Certain noise cannot be removed at binarization and still lowers accuracy (Tesseract).
• Method-dependent artifacts. Global thresholding fails on uneven backgrounds; adaptive methods can introduce thinning or breakage (Niblack noise, Sauvola thinning).
• Parameter sensitivity. Adaptive methods depend on window size and parameters (for example Sauvola's k and R); poor choices degrade results.
• Error propagation. Mistakes in early steps (wrong skew correction, wrong column segmentation or reading order) are difficult for the recognizer to undo.
• No universal best method. The appropriate technique depends on document type — clean modern print versus degraded/historical versus camera-captured.

Relationship to scanning

Preprocessing begins where scanning ends: capture settings largely determine how much preprocessing can help. Adequate resolution (Tesseract's approximately 300 dpi guidance) and avoiding dark scan borders are capture-time choices that directly ease downstream binarization and segmentation. Digitization-standards guidance from national-library and digital-preservation programs emphasizes source-image quality precisely because OCR accuracy depends heavily on the scan. See the per-technique pages and the scanning reference for capture parameters.

Relationship to PDF and searchable PDF

A searchable ("sandwich") PDF keeps the original page image as the visible layer and stores the OCR-recognized text as a hidden, selectable/searchable text layer positioned beneath it — the page looks identical to the scan but its text can be found and copied. Tools such as OCRmyPDF implement exactly this "add an OCR text layer to scanned PDF" workflow, and can output archival PDF/A (an ISO-standardized subset of PDF, defined by ISO 19005 and designed for long-term preservation, omitting features such as embedded JavaScript, audio/video, and external font references).

Preprocessing quality feeds directly into the accuracy of the embedded text layer: a cleaner binarized, deskewed page yields a more reliable searchable-PDF text layer.

Relationship to workflows

Preprocessing is one stage in a document pipeline: capture/scan → preprocess (rescale, binarize, deskew, denoise, segment) → recognize (OCR) → post-process (spell/dictionary correction, layout reconstruction) → output (searchable PDF, plain text, structured data) → index/archive.

In batch digitization and document-management contexts, preprocessing is typically automated and tuned per document class, because it is the lever that most affects recognition accuracy without changing the recognizer itself.

Modern relevance

Preprocessing remains standard practice. Modern engines still expose and depend on it: Tesseract's default Otsu binarization plus the Leptonica-based Adaptive Otsu and Sauvola options (added in 5.0.0), its OSD-based deskew, and its page-segmentation modes are all current.

In parallel, research has moved toward learned approaches — for example, deep-learning binarization methods evaluated on DIBCO/H-DIBCO — but the underlying goals (clean thresholding, straight lines, denoising, correct segmentation) are unchanged. Even LSTM and neural recognizers benefit from well-conditioned input, so preprocessing continues to be a first-class concern rather than a legacy step.

Timeline

1. 1979

   Otsu publishes an automatic global-threshold method based on grey-level histograms (IEEE Trans. SMC).

2. 1984

   Nagy and Seth publish "Hierarchical representation of optically scanned documents," the origin of the recursive X-Y-cut lineage.

3. 1985–1994

   Tesseract OCR engine developed at Hewlett-Packard Labs.

4. 1986

   Niblack's local adaptive thresholding approach described (Niblack, An Introduction to Digital Image Processing).

5. 1992

   Nagy, Seth and Viswanathan describe a prototype document image analysis system (IEEE Computer).

6. 2000

   Sauvola and Pietikainen publish adaptive document-image binarization improving on Niblack (Pattern Recognition, vol. 33).

7. 2005

   Hewlett-Packard releases Tesseract as open source.

8. 2006 onward

   Tesseract development sponsored by Google.

9. 2009

   First DIBCO document-image binarization competition (ICDAR series).

10. 2018

    Tesseract 4 introduces LSTM-based recognition (v4.0.0 final).

11. 2021

    Tesseract 5.0.0 (released 30 November) adds Leptonica-based Adaptive Otsu and Sauvola binarization options.