Optical character recognition (OCR)

Overview

Optical character recognition (OCR) is the electronic or mechanical conversion of images of typed, handwritten, or printed text into machine-encoded (machine-readable) text. The input is a raster image — a scanned page, a photograph, or a page rendered from a document — and the output is a stream of characters, words, and, in richer formats, positional and layout data that software can search, index, edit, and reflow.

OCR sits at the boundary between imaging and text. A scanner or camera captures how a page looks (pixels); OCR determines what the page says (characters). Because the image itself carries no character codes, OCR is the step that makes scanned documents searchable and reusable rather than static pictures of text.

OCR is distinct from several neighboring technologies with which it is frequently confused:

• ICR (Intelligent Character Recognition) targets handwritten print-script or cursive text, typically one glyph at a time, and usually involves machine learning. In common industry usage, "OCR" tends to denote machine-printed text and "ICR" denotes handwriting, though the underlying methods now overlap heavily.
• IWR (Intelligent Word Recognition) processes handwritten text at the whole-word level rather than character by character, which is useful for cursive scripts where glyphs are not cleanly separated.
• OMR (Optical Mark Recognition) does not read characters at all. It detects the presence or absence of marks in predefined positions — filled bubbles, checkboxes, ballot marks — and is used for tests, surveys, and ballots. OMR is a data-collection technique that does not require a character-recognition engine.

History

Early reading machines predate digital computing. In 1914 Emanuel Goldberg developed a machine that read characters and converted them into standard telegraph code, and in the same era Edmund Fournier d'Albe built the Optophone, a handheld scanner that, when moved across a printed page, produced tones corresponding to specific letters — an aid for blind readers. Goldberg later worked on a microfilm "Statistical Machine" for searching archives; in 1931 he was granted US Patent 1,838,389 for the invention, and the patent was subsequently acquired by IBM.

In the 1920s the Austrian engineer Gustav Tauschek developed an early OCR "Reading Machine" and obtained a patent for it. These early devices were largely electromechanical and relied on template or mask comparison rather than programmable computation. (The specific patent date sometimes cited for Tauschek's device comes from secondary summaries and is not established here from a primary patent record.)

The modern era is usually dated to the 1950s and 1960s, when OCR moved onto digital computers and into large-scale operational use. The U.S. Postal Service pursued automated address reading and introduced optical character readers to sort mail in the mid-1960s; the first live-mail postal OCR entered service at the Detroit post office in November 1965. The USPS later moved to multiline optical-character readers (MLOCR) and a human-in-the-loop Remote Encoding process for mail that machines cannot read.

In 1974 Ray Kurzweil founded Kurzweil Computer Products and developed omni-font OCR, able to recognize many typefaces rather than a single fixed font. A reading machine for blind users was unveiled on January 13, 1976, and a commercial version followed; LexisNexis was one of the first customers, using the technology to digitize documents.

The period also produced purpose-designed OCR typefaces so that early, less capable readers could achieve reliable accuracy while remaining human-legible: OCR-A, standardized in the United States (its alphanumeric set dates to 1966, later revised as ANSI X3.17-1981 and aligned with ISO 1073-1:1976), and OCR-B, developed in 1968 by Adrian Frutiger and later standardized as ISO 1073-2:1976. Over subsequent decades OCR progressed from fixed-font, template-based systems to statistical and machine-learning methods and, most recently, to neural-network sequence models.

How it works

OCR is best understood as a pipeline. Exact stage names and boundaries vary by engine, but the conceptual flow is consistent.

1. Image acquisition. A page is captured as a raster image by a scanner or camera. The resolution, contrast, and cleanliness of this image strongly constrain everything downstream.

2. Preprocessing. The image is cleaned and normalized to make text easier to isolate. Commonly documented steps include:

• De-skewing — rotating a page scanned at a slight angle so text lines are horizontal.
• Despeckling — removing stray spots and smoothing edges.
• Binarization — converting the image to black-and-white, separating foreground ink from background paper; the chosen threshold method materially affects accuracy.
• Layout analysis and segmentation — detecting text regions, columns, and lines, then separating lines into words and, for some engines, words into characters, with baselines established.
• Normalization — adjusting the aspect ratio and scale of the units to be recognized.

3. Recognition. The prepared units — individual characters, or whole text lines in modern engines — are converted to character codes. Two classical families and one modern family dominate:

• Matrix or pattern matching compares an image against stored glyph templates pixel by pixel. It works best on clean, typewritten text in known fonts and degrades badly on unfamiliar fonts or noise.
• Feature extraction decomposes glyphs into abstract features — strokes, lines, closed loops, line direction, intersections — and classifies based on those features, which generalizes better across fonts.
• Neural and sequence models feed a whole text line into a recurrent neural network (commonly a bidirectional LSTM) and decode the output with a Connectionist Temporal Classification (CTC) layer to produce the most likely sequence of characters and spaces, without requiring the line to be pre-segmented into individual characters. Transformer-based recognizers are a further, newer development.

4. Post-processing and output. Recognized text is often corrected against a lexicon or language model, and confidence scores may be attached. Output can be plain text or structured formats that also carry position and layout.

Approach and techniques

• Fixed-font vs. omni-font. Early systems recognized a single, purpose-built typeface (hence OCR-A and OCR-B). Omni-font OCR, advanced commercially from the 1970s, recognizes many typefaces.
• Template matching (matrix matching). Pixel-level comparison to stored glyphs; simple and fast but brittle to font and noise variation.
• Feature-based classification. Extract geometric or topological features, then classify — historically with statistical classifiers.
• Neural line recognition. A bidirectional LSTM with CTC decoding, operating on whole lines, is the current mainstream approach for high-accuracy engines; transformer models are an active frontier. As a concrete, well-documented example, the open-source Tesseract engine added an LSTM-based line-recognition engine in version 4, retaining the older character-based engine as a selectable legacy mode.
• Handwriting. Reading handwriting — treated under labels such as ICR, IWR, or HTR (Handwritten Text Recognition) in libraries and archives — is a related but harder problem, generally requiring dedicated models.

Tesseract is used here only as an illustrative, publicly documented example; it is not presented as the sole or best engine.

Accuracy and quality factors

OCR accuracy is not a single fixed number; it depends heavily on inputs. Character-level accuracy is the common metric, and it is degraded by:

• Image quality — low resolution, low contrast, skew, noise, bleed-through, show-through, and compression artifacts.
• Source characteristics — unusual or decorative fonts, very small type, tightly spaced or touching characters, complex multi-column layouts, tables, and mixed scripts.
• Material type — clean modern print recognizes far better than degraded historical print, and both recognize far better than handwriting.
• Language and script coverage — accuracy depends on whether the engine has a trained model and lexicon for the language and script.

General accuracy figures vary widely by engine, corpus, and evaluation method and should not be treated as fixed constants. Where an operator reports figures for a specific, controlled application, those figures are attributable to that application rather than to OCR in general. The U.S. Postal Service, for example, reports reading nearly 98 percent of hand-addressed letters and about 99.5 percent of machine-printed letters in its own mail-sorting operations — figures specific to that context, not a universal benchmark.

Advantages

• Searchability — turns page images into full-text-searchable and indexable content.
• Reusability — enables copy, edit, reflow, translation, and data extraction from what was previously a static image.
• Automation — supports high-volume workflows such as mail sorting, forms and invoice capture, and document classification that would be infeasible manually.
• Accessibility — recognized text can be fed to screen readers and text-to-speech and reflowed for different displays.
• Storage and handling efficiency — text is far smaller and more tractable than page images and integrates with databases and search systems.

Limitations

• Not perfect, and errors are silent. OCR introduces substitution, insertion, and deletion errors that a downstream reader cannot detect without the original — a plausible-looking wrong character is worse than an obvious gap.
• Sensitive to input quality — degraded, skewed, or noisy scans and unusual fonts sharply reduce accuracy.
• Layout is hard — tables, multi-column pages, footnotes, and mixed text and graphics can be mis-segmented even when individual characters are read correctly.
• Handwriting remains difficult — cursive and idiosyncratic hands are error-prone relative to print.
• Language and script gaps — engines perform well only on the languages and scripts they are trained for.
• Requires review for high-stakes use — OCR is a text-extraction technology, not a source of truth; output intended for consequential reuse generally needs verification against the original.

Relationship to scanning

Scanning and OCR are separate, sequential steps. Scanning produces the image; OCR interprets it. A scan alone yields a picture of a page that a computer cannot search or edit.

Because OCR quality is bounded by image quality, scanning parameters — resolution, bit depth, contrast, flatness and skew, and lighting for camera capture — directly determine how well OCR can perform. Preprocessing (de-skew, despeckle, binarize) bridges the two, compensating for imperfect capture before recognition. Some devices combine both steps into a single scan-to-searchable-PDF operation, but conceptually the two remain distinct.

Relationship to PDF and searchable PDF

A searchable PDF is the most common consumer-facing OCR deliverable. It preserves the original scanned page image for display and adds a text layer — the OCR output positioned to align with the visible characters — that is typically rendered invisibly behind or over the image. The user sees the original page but can search, select, and copy text, because those actions operate on the hidden text layer rather than the image.

OCR engines expose this capability directly. Tesseract, for example, can output a normal PDF or an invisible-text-only PDF, the latter being the mechanism for producing a searchable text layer over an image. For long-term preservation, OCR'd documents are frequently saved as PDF/A, the ISO archival profile that requires embedding fonts and resources so the file renders consistently over time; the searchable text layer is retained within that profile.

Relationship to workflows

OCR is rarely an end in itself; it is a stage inside larger document pipelines.

• Digitization, libraries, and archives. OCR for print and HTR for handwriting convert scanned collections into searchable corpora. Output is often stored in structured formats such as ALTO (Analyzed Layout and Text Object) — an XML schema for OCR layout and text metadata, originating in the EU METAe project, with maintenance transferred to the Library of Congress in August 2009 and commonly paired with METS — and hOCR, an open representation carrying text, layout, and confidence data. Tesseract can emit plain text, hOCR, ALTO, PAGE, TSV, and PDF.
• Mail and logistics. Postal services use OCR and MLOCR to read addresses and apply routing, with human Remote Encoding as a fallback for unreadable pieces.
• Business document capture. Invoices, receipts, and forms are OCR'd and then parsed — often with ICR for handwritten fields — to extract structured data into downstream systems, while OMR handles checkbox and bubble fields.
• Accessibility and search indexing. OCR feeds screen readers and populates full-text search indexes for otherwise image-only content.

Modern relevance

OCR remains foundational and is actively evolving. The shift from character-based matching to neural line recognition (LSTM with CTC decoding) markedly improved accuracy on document images at the cost of greater compute, and transformer-based recognizers and vision-language models are extending the field toward more end-to-end recognition, broader multilingual coverage, and better handling of complex layouts and handwriting.

Widely used systems today span open-source engines such as Tesseract along with numerous commercial and research toolkits. For digitization of cultural heritage, printed-text OCR and dedicated handwriting-recognition platforms are treated as complementary baseline tools.

Timeline

1. 1914

   Emanuel Goldberg builds a machine that reads characters and outputs telegraph code; Fournier d'Albe develops the Optophone, a letter-to-tone reading aid.

2. 1920s

   Gustav Tauschek develops an early OCR "Reading Machine" and obtains a patent for it.

3. 1931

   Goldberg is granted US Patent 1,838,389 for a microfilm "Statistical Machine"; the patent is later acquired by IBM.

4. November 1965

   The U.S. Postal Service places the first live-mail optical character reader into service at the Detroit post office.

5. 1966

   The OCR-A alphanumeric character set is standardized in the United States (later revised as ANSI X3.17-1981; aligned with ISO 1073-1:1976).

6. 1968

   Adrian Frutiger designs the OCR-B typeface (later ISO 1073-2:1976).

7. 1974

   Ray Kurzweil founds Kurzweil Computer Products and develops omni-font OCR.

8. January 13, 1976

   The Kurzweil reading machine for blind users is unveiled; a commercial version follows, with LexisNexis among the first customers.

9. 1985–1994

   Tesseract is originally developed at Hewlett-Packard, with further changes in 1996.

10. 2005

    HP open-sources Tesseract.

11. 2006–2017

    Google develops Tesseract (through August 2017).

12. August 2009

    Maintenance of the ALTO OCR-metadata schema is transferred to the Library of Congress.

13. Version 4 (Tesseract)

    An LSTM neural-network line-recognition engine is added, with the earlier character-based engine retained as a selectable legacy mode.

14. November 30, 2021

    Tesseract 5.0.0 is released.