AI Transcription Accuracy: What CER Measures

This article explains what AI transcription accuracy actually measures, how the standard metrics are calculated, and where they mislead. For anyone working with historical handwriting, it also explains why fluent output is the most dangerous failure mode of all — and how to read an accuracy figure the way you would read a source.

AI transcription accuracy is the measured agreement between a system's output and a human reference transcription of the same page, most often reported as Character Error Rate (CER) — the proportion of characters that would have to be corrected to match the reference. It is not the same thing as fluency. A transcription can read smoothly, in confident modern prose, and still be wrong in ways that are hard to catch, because the language model producing it is built to sound plausible, not to report what is on the page. Accuracy is a conditional measurement, tied to script, language, period, image quality, and how the reference itself was built. It is never a single universal number.

That gap between looking right and being right is the whole problem, and it is worth being precise about. What follows explains what the accuracy figures actually measure, how the standard metrics are calculated, where they mislead, and why fluent output is the most dangerous failure mode for historical work.

What "accuracy" measures — and what it doesn't

Start with the thing everyone quotes: Character Error Rate. CER is built on edit distance — the minimum number of single-character edits (substitutions, insertions, deletions) needed to turn the system's output into the reference. The formula is:

CER = (S + D + I) / N

where S, D, and I are substitutions, deletions, and insertions, and N is the number of characters in the reference. Multiply by 100 and you have a percentage. Lower is better; zero is a perfect match. A 1,576-character reference with 312 total errors scores a CER of roughly 19.8%.

One consequence surprises people. Because the denominator is the reference length and not the number of edits, CER can exceed 100%. A system that produces far more characters than the page contains — a hallucinating one, say — can post an error rate above 100%. That is not a fault in the metric. It is the metric telling you the output has come loose from the source.

Word Error Rate (WER) runs the same calculation at word level. The two are not interchangeable, and the difference is instructive. Turning moon into noon is a single character substitution — a small CER — but a completely wrong word, and a full error at word level. Which metric matters depends on what you are going to do with the text. For a genealogist reading a surname, one wrong letter is the whole result.

CER's weakness is that it collapses three distinct error types into one number and treats every character as equally important. It says nothing about which characters were wrong. It can also look reassuring while word boundaries are quietly broken — missing spaces, merged lines, scrambled reading order. Metrics like bag-of-words error rate and flexible, reading-order-independent character accuracy were devised precisely because a clean CER can coexist with a structurally mangled transcription. This matters for anything tabular: a ledger or logbook can score well on plain-text CER while the row-and-column alignment that gives the numbers meaning has fallen apart. The field has not settled on a single agreed metric for structured accuracy, which is worth knowing before you trust a percentage on a table.

Ground truth is not found in nature

Every accuracy figure is measured against a reference — the ground truth. It is easy to treat that reference as a fixed yardstick. It is not.

Ground truth is produced by human transcribers working to a set of conventions: diplomatic (record exactly what is on the page) versus normalised (silently modernise), how to treat abbreviations, ligatures, damaged passages, and ambiguous glyphs. Different conventions produce different references, and therefore different "errors." On difficult historical hands, transcribers disagree with each other, and those disagreements have to be adjudicated. Inter-annotator agreement is finite. So "ground truth" is a constructed, contestable object rather than a natural fact.

This is not a pedantic point. Noisy ground truth degrades reported accuracy measurably — even a few percent of inconsistent characters in the reference moves the number and introduces systematic bias, because the gold standard is itself a product of human choices. When you read a CER figure, you are reading a comparison against one particular set of editorial decisions, made by particular people, on particular pages. The convention question is the same one you face in your own transcription work, and it is the reason source integrity and editorial method sit upstream of any accuracy claim.

Why one number never tells the story

The most common misconception about transcription accuracy is that one percentage describes a tool. It does not. Reported accuracy is conditional on every variable in the material.

The same model can differ in CER by an order of magnitude between clean nineteenth-century German handwriting and a damaged sixteenth-century French manuscript thick with marginalia. A figure quoted on modern English says almost nothing about early-modern secretary hand, and even less about a vernacular hand in a language for which independent benchmarks are thin. A sound figure is measured on a held-out test set — pages, scribes, or collections the system never saw in training — so that the number reflects generalisation rather than memorisation. This is the discipline the NIST evaluation model and the ICDAR competition track enforce: blind test sets, edit-distance scoring, explicit rules against training on test material.

Most vendor figures disclose none of this. They rarely state whether the test set was held out, its size and provenance, whether the reference was adjudicated, or which metric was used. There is no industry-wide requirement to report on a common benchmark. Without those conditions, a headline accuracy figure cannot be independently checked — it is marketing, not measurement. The practical upshot is a habit worth keeping: when you see a percentage, ask on what, held out from what, scored how. A number without those answers is a number to discount. This is a recurring theme in how to judge HTR accuracy claims, and it does not get easier the more impressive the figure looks.

There is also the question of what "good enough" means. At 99% character accuracy over a long document you still expect roughly one wrong character per hundred. For some tasks that is fine; for others it is not. Work on early-modern Fraktur found that below about 90% accuracy — a CER above roughly 10% — downstream tasks degrade non-linearly, the pipeline falling apart faster than the raw error rate would suggest. Usability is task-dependent. The threshold for a searchable index is not the threshold for a scholarly edition.

Confidence is not accuracy

A related trap: the confidence score. Many systems attach a number to each prediction. It is tempting to read a high-confidence flag as a low-risk one. Do not.

Confidence is calibrated only when it tracks the actual probability of being right — and modern neural networks are systematically over-confident, especially on out-of-distribution inputs, which is exactly what an unusual hand or a degraded page is. Calibration methods exist for classification problems but are not a default component of transcription products, and the per-character or per-line scores shipped to users are typically uncalibrated. A recogniser can output John Smith, b. 14 March 1824 with high confidence while having invented both the name and the date. Confidence tells you how sure the model is. It tells you nothing reliable about whether it is correct.

Why fluency is the failure mode, not the measure

Here is the heart of it. The most dangerous errors in AI transcription are not garbled ones. They are fluent ones.

When an image-first recogniser fails, it tends to fail visibly: a broken character, a dropped mark, a word that reads as nonsense. You can see the wound. When a language-model-driven system fails, it fails plausibly. It produces a smooth, grammatical, period-appropriate reading that is simply not what the page says — and because it reads well, it survives a casual check.

The mechanism is well documented outside the manuscript world. The Whisper speech-recognition model has been shown to generate fluent, confident phantom text on silence and non-speech audio — recurring inventions like "Thank you for watching" and "Subscribe" that appear nowhere in the source. The same behaviour operates in vision-language systems asked to transcribe an image: confronted with an ambiguous or illegible passage, the model's language prior fills the gap with something that sounds right. Fluent output is a signal that the model is confident, not that it read the page.

This is why "if it looks fluent, it's accurate" is exactly backwards for historical work, and why fluent-but-wrong LLM errors are harder to catch than the honest garble of older OCR. It is also why the assumption that a tool good at printed text will handle handwriting does not hold: even strong proprietary multimodal models still underperform specialised HTR baselines on historical benchmarks such as Bentham, IAM, ICDAR, and Saint Gall. That same benchmarking work found proprietary models competitive with supervised baselines in zero-shot settings on modern English handwriting — but systematically weaker on non-English languages and historical hands, on single corpora, and with self-correction loops that did not reliably improve matters and outputs that still require human verification. On the material historians and genealogists actually face, the general-purpose fluency that impresses is precisely the thing to distrust.

The error types line up on a spectrum of recoverability. Image-first tools err on characters — recoverable, because you can see the mistake against the page. General LLMs, leaning on a text prior with the image downsampled, err with plausible fabrications and long hallucinations — dangerous, because nothing on the surface flags them. A specialist recogniser that weighs high-resolution visual evidence against context sits at the recoverable end: it errs on characters and words, in ways a human checker can catch. This is the whole reason to keep a human in the loop rather than trusting a confident output — and the reason the choice of tool is not really about which posts the prettiest demo.

What this means for choosing and reading a tool

If accuracy is conditional and fluency is a trap, the useful question is not "which tool is most accurate" but "which tool is built to fail recoverably on the material I have, and does its reported figure survive scrutiny."

This is the ground on which purpose-built handwritten text recognition earns its place. Leo's model, ATR-1, is trained on images of historical documents and built to transcribe what is on the page rather than smooth it into modern prose — the long s stays a long s, an abbreviation mark is preserved rather than silently resolved, an archaic spelling survives rather than being "corrected." Where a general model's instinct is to normalise, a specialist model is trained to leave the source intact, which is what keeps its errors visible and checkable. Its cleared accuracy figure follows the discipline argued for here: on a randomised 97-image sample of early-modern English manuscripts from the Folger Shakespeare Library, at ATR-1's release, Leo scored roughly 5% CER against the next-best model's ~13% (Transkribus/Text Titan I), with several general LLMs far behind — the full benchmark, with model versions and per-item scores, is published here. That figure is stated with its conditions — a named corpus, a fixed sample, a stated metric — because a number without conditions is the very thing to be wary of. It is a strong result on early-modern English manuscripts. It is not a promise about every hand, language, or degraded page, and it does not remove the need to verify against the image.

For anyone weighing the tool classes more broadly — general OCR, LLMs, crowdsourcing, and specialist HTR — the honest comparison of handwriting transcription software works through where each one holds up and where it breaks.

Reading the number, not the demo

Accuracy in AI transcription is real, measurable, and useful — but only when you read it the way a specialist reads a source: with its provenance attached. A CER figure means something when you know the corpus, the held-out split, the metric, and the editorial convention behind the reference. It means very little as a bare percentage on a marketing page. And a fluent transcription tells you nothing on its own, because fluency is what a language model produces whether or not it read the page correctly.

The skill this asks of you is the same one paleography always has: judgement held against the evidence. Read the figure critically, check the output against the image, treat every confident reading as a claim to be verified rather than a fact to be filed. The machine can carry the first pass. Deciding whether it got the page right is still your work, and it is work worth doing well.

Frequently Asked Questions

What is a good AI transcription accuracy rate for historical handwriting?

There is no single "good" number, because AI transcription accuracy is conditional on script, language, period, image quality, and how the reference was built. A figure quoted on modern English says almost nothing about early-modern secretary hand or a vernacular hand. As a rough guide, work on early-modern Fraktur found that below about 90% character accuracy — a Character Error Rate above roughly 10% — downstream tasks degrade non-linearly, faster than the raw error rate suggests. Usability is also task-dependent: the threshold for a searchable index is not the threshold for a scholarly edition. Always ask on what material a figure was measured before trusting it.

How is Character Error Rate (CER) calculated?

Character Error Rate is calculated as CER = (S + D + I) / N, where S, D, and I are the substitutions, deletions, and insertions needed to turn a system's output into the human reference, and N is the number of characters in that reference. Multiply by 100 for a percentage; lower is better, and zero is a perfect match. It is built on edit distance — the minimum single-character edits required. Because the denominator is the reference length, CER can exceed 100% when a system produces far more characters than the page holds, which is what a hallucinating output looks like.

Does a high confidence score mean a transcription is accurate?

No. Confidence tells you how sure the model is, not whether it is correct. Modern neural networks are systematically over-confident, especially on out-of-distribution inputs — which is exactly what an unusual hand or a degraded page presents. Calibration methods exist for classification problems but are not a default part of transcription products, so the per-character or per-line scores shipped to users are typically uncalibrated. A recogniser can output a name and date with high confidence while having invented both. Treat every confident reading as a claim to verify against the image, not a fact to file.

Why are fluent AI transcription errors more dangerous than garbled ones?

Fluent errors are more dangerous because they survive a casual check. When an image-first recogniser fails, it fails visibly — a broken character or a word that reads as nonsense, so you can see the mistake. When a language-model-driven system fails, it produces a smooth, grammatical, period-appropriate reading that is simply not what the page says. Confronted with an ambiguous or illegible passage, the model's language prior fills the gap with something that sounds right. Fluent output signals that the model is confident, not that it read the page — which is why "if it looks fluent, it's accurate" is backwards for historical work.

What is ground truth in transcription, and why does it affect accuracy figures?

Ground truth is the human reference transcription a system's output is measured against — but it is a constructed, contestable object, not a natural fact. It is produced by transcribers working to conventions: diplomatic versus normalised, how to treat abbreviations, ligatures, damaged passages, and ambiguous glyphs. Different conventions produce different references, and therefore different "errors." On difficult hands, transcribers disagree, and those disagreements must be adjudicated. Noisy ground truth degrades reported accuracy measurably — even a few percent of inconsistent characters moves the number and introduces bias. So any CER figure is a comparison against one particular set of editorial decisions, made by particular people on particular pages.