What is OCR: reading letters versus reading marks

OCR, short for optical character recognition, is the reading that turns the shapes of letters in an image into text. What settles the choice is the document rather than the name of a tool: free writing, a mark whose place is fixed in advance, or a region to be separated. In Vipkurum’s optical reading tool, what gets read is the mark on the form, not a letter.

What OCR is and what it turns a page into

Optical character recognition is what the abbreviation expands to. The expanded phrase describes a job rather than a product: it takes writing that is trapped inside an image and hands back characters a program can work with. The definition lives at both ends of that conversion. A picture arrives at one end and text leaves at the other.

Text sounds like a modest output until you list what it allows. Text can be searched, so you can find which line of a document a word appears on. Text can be copied, so moving it somewhere else does not mean typing it again. Text can be written into a field, which is how it reaches a form, a table or a record. None of that works while the page is still a picture. The sheet is in front of you, and software cannot touch a single word on it. Where the reading is needed follows from the same definition. If writing has ended up inside an image and software has to act on it, a conversion step stands in between. If the writing never entered an image, there is no step to add. The difference is in where the document came from: a text typed on a screen and a text printed and then scanned can sit inside the same file format and still hold different things.

One more thing is worth separating out. Stretch the abbreviation to cover document software in general and very different jobs end up inside it. Keeping the definition narrow has a practical payoff: a narrow definition lets you measure whether your own document fits it. A wide definition does not support measurement at all, only reassurance.

The term also carries its own noise. The same string of letters names things far outside software, and a drug name turns up in the search results next to it. Everything from here on concerns software that reads documents; the medical queries have no answer on this page.

A scan is a picture, not yet text

A scanner looks like a copying device and behaves more like a camera. It lights the paper, measures the light and dark areas across it, and lays those measurements out on a grid. The grid is what the file holds. Some of the dark areas look like letters to you; to software there are no letters there. There are dots, and the only thing each dot knows is its own colour.

Recognition is the word that fills that gap. The shapes the dots form are compared against the letter shapes the software already holds, and each shape is given a match. Once the matches are in, the page carries text for the first time. Text is not a by-product of scanning; it is the output of a separate step. If that step never ran, the file still holds a picture.

A misreading grows out of this. A file’s format and the kind of thing inside it are separate questions: a document in PDF form says nothing about writing being inside it, because the same format can just as easily carry a photograph of a page. To find out which one you have, open the file and look for a word you know appears in it. If nothing comes back, there is no text there yet, only a picture.

The business side of this distinction is concrete. A scanned archive looks digital: the files sit in folders, they are backed up, they open on screen. Search that archive for a name or a figure that appears somewhere inside it, though, and the box comes back empty. What went digital is the image of each page; the content of the page is still readable only by eye. The step that closes that gap is recognition.

How exact the matching gets depends on the shape of the letter. The blurrier the shape, the blurrier the match. A reading built for printed letters does not return the same result on handwriting, and reading handwriting is a separate problem that needs a separate setup. On the decision side that dependency is easy to state. Whoever fills in the document sets how hard the reading will be; a page off a printing press and a page filled in by hand do not go into the same recognition. So the question of whether your documents can be read cannot be answered without knowing whether they were printed or written.

The reverse case is worth stating. A business whose documents are created inside software has no use for this reading at all: the text was born as text and never went into an image. Letter recognition is only needed where writing has been shut inside a picture.

OCR and OMR do not read the same thing

A second abbreviation sits close enough to be swapped for the first, and the swap is expensive. OMR expands to optical mark recognition. A single letter divides them, and a single letter is not much of a warning sign. What they do not share is the thing each one reads.

Start with the unit, because everything else follows from it. Letter recognition works on letters, and a letter is self-contained: it carries its own shape and reads the same wherever it lands on the page. Mark recognition works on positions. A mark carries no shape worth reading; its meaning arrives from the place it occupies. The same pencil stroke stands for one answer in one box and for a different answer in the box beside it, and position is all that separates them.

What OMR actually reports is narrow. It checks an area that was pegged down beforehand and says whether a mark is there. No letter and no text leaves the step; the state of an area does. That state means nothing until you know which question the area belongs to on the form.

The word optical means the same thing wherever it appears here: light does the reading, not contact with the paper. So the input is shared and the extraction is not. Letter recognition converts shapes into characters. Mark recognition flags areas as filled or empty. One image yields information of different kinds, and neither kind can be converted into the other.

From here the consequence is easy to state. Run one sheet of paper through one scanner and the output depends entirely on which reading was built behind it: with letter recognition you hold the text of the writing on the page, with mark recognition you hold a list of which areas were marked. The scanner settles none of that. The definition of the reading settles all of it. And when the definition is wrong, the bill arrives later: a setup that reads letters will not recognise a marked form, and a setup that reads marks will not read free writing. The readings do not cancel each other either. A document carrying writing and marks at once needs each of them, and neither stands in for the other.

Which reading does your document call for

What sorts the table below is the thing being read, not a brand of tool. In one row there is free writing, in another a mark whose place has been fixed, in a third a region that has to be cut out. You can read the rows with your own document in hand: whichever row it falls into names the reading and says what its reliability depends on.

The last rows come from our own work. The optical reading tool we built for Vipkurum pulls the mark off a form; the question cropping tool separates a region from a page. The top row is the definition row: our portfolio holds no letter recognition setup, and this article does not write as though it did. The sections below take our own cases one at a time.

The last column is the easy one to skip. Choosing the name of the reading is one job; knowing what that reading depends on is another. In letter recognition reliability leans on the print quality of the document and the clarity of the writing. In mark recognition the weight shifts to the attention of whoever filled the form in. In separation it shifts again, to the geometry of the page. On no row does accuracy rest on the tool alone; on every row a property of the document weighs in.

Mark reading has to know where the mark will be

A reading that draws its meaning from position needs position to hold still. That requirement is the whole point of an optical form, where every box has a place before anyone picks up a pencil. Take that fixed layout away and the reading is left holding dark patches, with no way to say which patch answers which question.

Vipkurum’s optical reading tool is built for exactly that kind of document. OpenCV lifts the marks out of the form’s defined areas. What comes back is not text. It is a statement of which area carried a mark. Printed writing may share the sheet with those areas, and the tool does not read it.

Ownership of the layout turns into a live question here: the form moves with an institution’s own way of assessing, and the adaptation is made in the reading software rather than anywhere else. So the layout belongs to the reading’s own work, and what that adaptation costs is not this article’s subject; where the work goes when a form layout changes is covered separately.

Reliability moves with it. How cleanly a letter was formed has no bearing on a mark; what has a bearing is whether the mark was filled in properly. Half done, spilling over, spread across a neighbouring box: each of those makes the reading harder. What the tool took away was hardware, and the requirement that the form be filled in properly stayed where it was. For a buyer the practical upshot is that the design of the form has become a software question. Where the boxes sit, how many there are and in what order make up the map the reading consults, and owning that map is what lets the reading be built around you.

Receive that map from outside instead, with every incoming form arranged differently, and the arrangement has to be settled before any reading is built. Carry that further and the reading stops being possible at all: with no place pegged down in advance, there is no position to inspect. A freely completed sheet is asking for a different reading. None of that counts as a shortcoming; it is the definition doing its work.

Cutting a page into questions is not recognition

Picture a teacher at an image editor, cropping one question out of a booklet page, then the next, then the next. In Vipkurum’s question cropping tool, software does that cutting on a printed booklet page, and where the divisions fall is settled by image processing.

What the splitting step hands back is not text. Cropped images sit alongside the question records. A booklet that was sitting on a shelf becomes a pool divided question by question. What changed is the way the page is split. The writing on it was never turned into text.

Calling that step recognition empties the word out, because nothing on the page is being recognised: the step looks for a boundary, for the line where one part of the page ends and the next begins. Put most sharply: a job can leave human hands without letter recognition ever being involved, and cutting the page is one such job. If the output of a step stays an image, character recognition never entered it.

Handing a step to software does not always require meaning. In some steps what gets taken over is only a measurement: where something starts, where it ends. The moment the measurement can be made, the job can be done; reading the content is not a condition of that step.

So when the word reading comes up, one more question is worth asking: what is left in your hands when the reading ends. If text is left behind, recognition has taken place. If the state of an area is left, a mark was read. If an image is left, only a boundary was found. The same word covers all of those outputs, and the only way to see the difference is to look at the output.

Separation has a reverse side as well. Because boundary finding does not read the text, the separating step does not know what the piece says. In some jobs that is enough and in others it is not. What decides is what happens to the piece afterwards.

Jobs that need the meaning of the text do not stay on the vision side. This article does not argue where that line falls; which decisions are kept out of a model’s interpretation is written up in our piece on human oversight in AI systems.

What to settle before you have a document read

Open the conversation with the name of a tool and the document gets bent to fit it: forms are reprinted, pages are rescanned, a manual correction step slides into the flow. Open it with the document and the tool is the thing that has to follow. Which of those orders you pick is what decides who absorbs the cost.

So what is worth carrying into that conversation is not a tool name but a short list of questions. The order they come in matters. A tool name, if it comes at all, comes out at the end of the list.

Ask first what sits inside the document. A page of free writing and a form with one of its boxes ticked do not call for the same reading. A file you need a region cut out of calls for something else again. The name of the reading comes out of that answer, so nothing further can be settled until it is given.

Ask next who decides the layout, because mark reading becomes buildable only where the place the reading consults holds still. Own the layout and you can see where a change of yours will land; inherit a different arrangement with every incoming document and no fixed position is left to consult.

Ask last what shape the output has to take. Searchable text calls for letter recognition, and one value dropping into a field needs nothing more than the state of a mark. A cut-out image needs no letter read at all. Where that output then lands, and which rule it sets off when it lands, is separate work on the management software side.

One limit belongs in writing before any of that starts. Our portfolio holds work on the mark reading side and work on the page separation side, and no letter recognition setup. So this page is not a capability list; what it gives you is a question to put to your own document.

Is there still a sheet of paper, a form or a printed page sitting inside one of your steps? Then the place to start is the document itself: what in it is actually there to be read. Sorting which step belongs to image processing and which one needs a model is the first piece of work we do on the AI consulting side.