Do You Need to Train AI to Read Handwriting?

Before you train AI to read handwriting from your own collection, test a ready model on a representative sample of your pages and measure what it actually gets wrong. This article sets out what training a custom recognizer involves, what the published evidence says it costs, and the narrow conditions under which it earns that cost — so the decision takes an afternoon rather than a term.

The short answer

Usually you do not need to train. The correct order of operations is to test a ready model on a representative sample of your own pages, measure the character error rate against a reference transcription you key yourself, and only then decide whether training is worth the ground-truth labour. Training a custom recognizer earns its cost in a narrow set of conditions: a large, homogeneous series in a single hand, a script or convention that no available model has seen, or an existing ground-truth set you have already paid for. For most historians working through a few hundred pages of heterogeneous archival material, the answer to "should I train AI to read handwriting?" is that someone already did, and your job is to verify the output rather than build the model.

That answer is only useful if you know what the alternative involves. So it is worth being precise about what training means, what it costs, and what the published evidence does and does not establish.

Three different things people mean by "training a model"

The vocabulary collapses three distinct choices into one word.

Training from scratch

The recognizer learns its parameters only from your project's ground truth — accurate, human-verified transcriptions paired with the manuscript images. There is no prior knowledge in the system. Everything the model knows about letterforms, it learns from your pages.

Fine-tuning

Fine-tuning, or transfer learning, starts from an existing base model and specializes it on your material. Transkribus documents the mechanism plainly: the new model learns from your ground truth and from the knowledge already in the base, which means less data and faster training. But the base has to be genuinely similar. Their own guidance warns that a base model too different from your documents will raise your error rate — a sixteenth-century Latin base applied to nineteenth-century French will confuse the new model rather than help it.

Using a ready model

This means applying a pretrained system with no project-specific fine-tuning at all. Operationally, this is what "zero-shot" describes.

These are three decisions with three cost structures, not synonyms. And underneath all of them sits a distinction that trips up people planning their first project: recognition and layout are separate problems. Segmentation finds text regions, baselines and reading order; recognition maps ordered line images to characters. Kraken's documentation is clear that these require separate ground truth — a "map" of lines and regions for layout, line-image-and-transcription pairs for text — and that getting reading order wrong on a multi-column or heavily annotated page produces disconnected text regardless of how well the recognizer reads individual letters. eScriptorium exposes Segmenter and Recognizer training as separate operations for exactly this reason. If your material has marginalia, columns, or tabular structure, "training a model" may quietly mean training two.

What training actually costs

The full mechanical workflow runs: select representative pages; transcribe them consistently to a fixed diplomatic standard; create or verify layout regions, baselines and reading order; train on line images plus transcriptions with validation data reserved; evaluate on a held-out test set; inspect the errors; correct recognized pages into new ground truth; retrain; stop when further iterations no longer improve the held-out result.

Every step in that list is your time, or your research assistant's.

The ground-truth question has no single answer

You will find confident thresholds quoted. They do not agree with each other, because they come from different engines under different conditions.

Transkribus says a first handwritten model generally needs at least 10,000 transcribed words, while noting the figure varies by material and model.
A Journal of Documentation account of HTR in archival scholarship gives around 15,000 words, or 75 pages of a single script, as typical for applying a model at volume.
Kraken's training documentation puts a single-script model at a minimum of 800 lines — upwards of 30 pages at 25–40 western lines per page — while allowing that a clean, specific manuscript may need only a couple of dozen pages and a genuinely generalized model may need tens of thousands of examples.

Treat these as starting points from different toolchains, not as a law. What they collectively establish is the order of magnitude: producing a first custom model means hand-transcribing somewhere between twenty and a hundred pages to a publication-grade standard before the model reads anything for you. Then you iterate. Transkribus's own retraining example — an 800-page English diary, fifty manually transcribed pages for version one, then additional twenty-page batches, with a batch correction taking a few hours — is an illustrative practitioner workflow rather than a controlled labour benchmark, but it gives the shape of the commitment honestly.

Machine time is real but secondary. Kraken notes that accuracy seldom improves after fifty epochs, reached between eight and twenty-four hours. eScriptorium reports that training can take several hours and that self-hosting wants Linux or macOS, 8 GB of RAM minimum with 16 GB or more for training, Docker, and — optionally but very helpfully — an Nvidia GPU. Open source removes licensing dependence; it does not remove server administration, data preparation, or compute.

One more cost, easy to miss until you are deep in a funded project: outsourcing training to a commercial provider often means paying not only for the model but for every subsequent batch of transcription and ground truth, and providers frequently do not release models trained on client data — a genuine problem where a funder requires public outputs.

More ground truth is not automatically better

This is the finding most likely to surprise someone planning a training budget. Hodel et al.'s German Kurrent study assembled a very large model from the State Archives of Zürich — many pages, but written by a relatively small number of scribes with comparable training. The result was over-specialization: recognition of the known hands improved, while quality on similar but not identical hands declined. They had to deliberately rebalance the training set with more varied hands.

Representativeness, transcription consistency, hand coverage and layout diversity matter more than raw volume. Twenty carefully corrected pages will outperform a hundred pages with errors in the reference. And because there is no published cross-tool curve showing where accuracy flattens as a function of data, you cannot plan the endpoint in advance. You can only measure as you go.

The diagnostic: five questions before you train anything

1. Have you tested a ready model on your own pages yet?

Not on a showcase example — on a random sample of the material you will actually work with, including the worst pages. This is the step most often skipped. There is a practical protocol for testing an AI transcription tool on your own manuscripts in an afternoon: sample randomly, key a reference transcription, score it, then read the errors rather than only the number.

2. What accuracy does your actual use require?

Character error rate is the normalized character-level edit distance — substitutions plus deletions plus insertions over reference characters. It is a corpus metric, not a measure of paleographic correctness or of how long correction will take. Transkribus's vendor-indicative bands are useful as orientation: full-text search often works at 5–8% CER, scholarly editions may need under 2%, keyword spotting can tolerate 10–15%. These are guidance ranges, not consensus. But the question they force is the right one — a finding-aid index and a critical edition are not the same project, and interpreting accuracy figures properly is what turns a benchmark into a decision.

3. How homogeneous is your corpus?

Training amortizes over volume and uniformity. A single scribe's letter-book of two thousand pages is the strong case. Forty documents from twelve archives in four hands is the weak one. There is no controlled Latin-script study measuring the break-even page count, so treat this as a rational decision rule to test rather than a settled result.

4. Is there evidence for your script and language specifically?

The independent comparative evidence is uneven. German Kurrent is the best-documented case; Dutch and English have direct multilingual-study evidence. French, Spanish and Italian historical hands should be treated as test-first rather than assigned a confident maturity label — vendor claims about community-model counts are availability evidence, not accuracy evidence. This is the general problem with judging which languages and scripts HTR can actually read.

5. Do you already have ground truth?

If a predecessor project left you fifty diplomatically transcribed pages, fine-tuning a reasonably similar base becomes cheap. If you would be starting the transcription from zero, you are paying the full price.

It is worth saying that fine-tuning demonstrably helps in some cases: in a multilingual, multi-authored collection, Capurro et al. found that specializing existing models on project data improved English CER from 18.7% to 11.1% and Dutch from 21.7% to 18.9% — a real gain, though on one heterogeneous corpus, with both post-training figures still far above what a scholarly edition would accept, and with the Dutch improvement marginal. It argues for fine-tuning when your baseline is genuinely poor, not for training as a default first move.

The case for not training at all

The reason "test first" has become good advice, rather than merely cautious advice, is that ready models improved. That is where Leo sits in this decision, and it is the position we argue for: a specialist model that is applied, not built.

Leo's transcription engine, ATR-1, is a zero-shot Transformer trained on images of historical documents. There is no per-corpus training step, no ground-truth production, no segmentation pass to configure — you upload page images and it reads them. On a randomized 97-image sample of early-modern English manuscripts from the Folger Shakespeare Library, at ATR-1's release, it scored roughly 5% character error rate against Transkribus Text Titan I at about 13%, Claude Opus at 23.3%, Gemini 2.5 Pro at 24.8% and GPT-4.1 at 56.7% — 61% fewer errors than the next-best model, with the full comparison data published here. One sample, one language, early-modern English: run it on your own pages before you trust it on ours.

The scope is the writing system, not the language. Any language written in the Latin alphabet is in scope — English wills, French notarial records, Dutch registers, German parish books, Latin treatises alike. Greek, Cyrillic, Hebrew, Arabic, Indic and East Asian scripts are not. Transcription is not translation: Leo transcribes what is on the page, and translating it is a separate one-click Transformation that writes to a new tab, leaving the base transcription untouched. That separation is the point. Strikethroughs, insertions, marginal notes and archaic orthography survive into the transcript rather than being silently modernized.

The honest limits: complex tabular layouts vary, and pages mixing dominant printed structure with dense handwriting — pre-printed ledger and deed-book forms — are the known weak spot. As users correct transcriptions in the app, those corrections feed back into training, so the model you test this year is not the one you tested last year. That is the practical argument, and it belongs in your comparison alongside the other classes of HTR software and their alternatives.

When training still makes sense

Some projects should train, and it is not a failure of tooling when they do.

Train when your corpus is large, homogeneous, and a ready model demonstrably fails on it — measured on a held-out sample, not judged by eye. Train when your material is in a script or a set of conventions no available model has seen. Train when your transcription norms are themselves a research output and must be reproduced exactly. Train when the model is a deliverable your funder expects to be published. And if you do, eScriptorium and Kraken give you the most control over the pipeline, while OCR4all's historical-print orientation is worth understanding before you point it at manuscript hands.

What you should not do is treat training as the entry fee. It is one option among several, with a real fixed cost, justified by evidence you can gather in a day.

The output is a draft, whichever route you take

The error rate tells you how much of the text differs from a reference. It tells you nothing about where the errors fall, and names, dates, sums and place-names are exactly the tokens where a plausible misreading does the most damage downstream. That is why verification against the image remains the historian's work, and why paleographic skill has not been retired by better recognition — it has been repositioned. You are no longer deciphering every letter. You are adjudicating a reading, which requires knowing what the hand can and cannot have written. Time spent on that judgment compounds across a career. Time spent producing ground truth for a model you did not need does not.

Frequently Asked Questions

Do you need to train AI to read handwriting from your own collection?

Usually not. Test a ready model on a representative sample of your own pages first, key a reference transcription yourself, measure the character error rate, and read where the errors fall. Only then decide whether training is worth the ground-truth labour. Training earns its cost in narrow conditions: a large, homogeneous series in a single hand, a script or set of conventions no available model has seen, transcription norms that are themselves a research output, or ground truth a predecessor project already paid for. For a few hundred pages of heterogeneous archival material, verifying output usually beats building a model.

How much ground truth do you need to train a handwriting recognition model?

There is no single number, because published thresholds come from different engines under different conditions. Transkribus suggests a first handwritten model generally needs at least 10,000 transcribed words. A Journal of Documentation account of archival HTR gives around 15,000 words, or 75 pages of a single script. Kraken's documentation puts a single-script model at a minimum of 800 lines — upwards of 30 pages — while noting a clean, specific manuscript may need only a couple of dozen. The order of magnitude is consistent: roughly twenty to a hundred pages hand-transcribed to publication standard, then iteration.

What is the difference between fine-tuning and training a model from scratch?

Training from scratch means the recognizer learns everything from your project's ground truth alone — no prior knowledge of letterforms enters the system. Fine-tuning, or transfer learning, starts from an existing base model and specializes it on your material, so it learns from your ground truth and from what the base already knows. That means less data and faster training, but only if the base is genuinely similar; Transkribus warns that a base too different from your documents will raise your error rate. A sixteenth-century Latin base applied to nineteenth-century French confuses the new model rather than helping it.

Does more training data always improve handwriting recognition accuracy?

No. Hodel et al.'s German Kurrent study assembled a very large model from the State Archives of Zürich — many pages, but written by a relatively small number of scribes with comparable training. The result was over-specialization: recognition of the known hands improved while quality on similar but not identical hands declined, and the training set had to be deliberately rebalanced with more varied hands. Representativeness, transcription consistency, hand coverage and layout diversity matter more than raw volume. Twenty carefully corrected pages will outperform a hundred pages with errors in the reference transcription.

Can an AI transcribe handwriting without being trained on my documents?

Yes — that is what zero-shot means. Leo's transcription engine, ATR-1, is a zero-shot Transformer trained on images of historical documents: no per-corpus training step, no ground-truth production, no segmentation pass to configure. On a randomized 97-image sample of early-modern English manuscripts from the Folger Shakespeare Library, at release it scored roughly 5% character error rate, ahead of the next-best model tested by a wide margin. That is one sample in one language, so run it on your own pages. Scope is the Latin alphabet; pre-printed ledger and deed-book forms remain the known weak spot.