Why ChatGPT Struggles with Historical Handwriting

This is a plain account of why ChatGPT and its peers falter when asked to transcribe handwriting from old manuscripts — and where they still earn their place. If you rely on general AI to read archival hands, understanding how it fails tells you exactly when it is safe to use and when it will quietly corrupt your work.

ChatGPT struggles with historical handwriting for two structural reasons. It downsamples the image before its language model ever sees the page, so fine marks — a macron, a hairline abbreviation stroke, the difference between a long s and an f — are averaged out before reading begins. And because it is a text generator at heart, when it cannot read something it produces the most statistically plausible words rather than admitting the gap, so its errors arrive as confident, fluent fabrications that are hard to catch. On clean modern handwriting it often does well. On degraded or archaic hands, specialist recognition wins decisively — and the gap is not small.

That is the short version. The longer version is worth understanding, because why ChatGPT fails tells you exactly when it is safe to use and when it will quietly corrupt your work.

What ChatGPT Actually Is, for This Task

ChatGPT and its peers — Claude, Gemini — are vision-language models (VLMs): a large language model bolted to a vision encoder. The encoder turns the image into a sequence of embedding vectors; the language decoder then generates text one token at a time, each token conditioned on the image embeddings and on everything it has already written.

That last clause is the whole problem. A dedicated handwritten text recognition (HTR) engine is image-first: it produces per-character probabilities grounded in pixels, and when the ink is ambiguous it misreads a letter. A VLM is text-first. Its decoder carries a strong learned prior about what English — or French, or Latin — usually looks like, and that prior pulls the output toward common, well-formed sequences. When the visual evidence is thin, the prior fills the vacuum. This is the mechanism behind the tempting but wrong claim that transcription is "just OCR." The failure modes differ in kind. OCR and HTR miss letters. VLMs invent text. For the fuller version of that distinction, we've written it up separately in HTR vs OCR; it belongs to the broader question of which HTR software actually fits historical work.

The Resolution Problem: What the Model Never Sees

The first reason ChatGPT struggles is physical, and it happens before any "reading" occurs.

VLMs do not consume raw pixels. They tile the image into fixed-size patches and project each patch into a fixed token budget. OpenAI's own vision documentation describes the geometry: in high-detail mode an image is fit inside a 2048×2048 envelope, split into 512×512 tiles, and each tile is encoded at roughly 170 vision tokens. Claude and Gemini apply comparable per-image token ceilings.

For a photograph of your dog this is plenty. For a densely written folio it is not. A single 512-pixel tile may have to represent a whole cluster of words. Anything below the effective per-patch resolution is averaged into its neighbours before the language model receives a single token. This is why the intuition "just zoom in and it'll see the detail" does not hold: the model is not looking at your zoomed image, it is looking at a compressed embedding of it.

The marks that vanish first are precisely the ones that carry scholarly weight:

The long s (ſ), used word-initially and medially from the late eighth century to the early nineteenth. A faint descender is exactly the kind of hairline detail that survives downsampling poorly — and it is routinely misread as f even by systems built for the job.
The macron or titulus — a bar over a vowel standing for an omitted n or m. It is a thin horizontal stroke. Average it out and dñi silently becomes dni, or worse, gets "corrected" to something plausible.
Brevigraphs and contraction marks generally, the dense scribal shorthand that fills legal and administrative hands.

None of this is exotic to a paleographer. It is the ordinary furniture of the page. But it lives at the spatial frequencies a VLM discards first. We treat this shorthand at length in our guide to manuscript abbreviations and ligatures, because reading it correctly is often the whole task.

The Confabulation Problem: Fluent, Plausible, Wrong

The second reason is more dangerous, because it does not look like a failure.

When an HTR engine cannot read a word, it produces garbage or a low-confidence guess that looks like garbage — a misfired character, a nonsense string. You can see the wound. A VLM, faced with the same illegible word, does what a language model is built to do: it emits the most probable continuation. The result is a grammatical, period-appropriate, entirely invented reading. Put plainly: OCR errs recoverably; VLMs confabulate.

This is not a matter of prompting harder. He et al.'s 2025 study, Seeing is Believing?, built a benchmark specifically because GPT-4o-class models fail to abstain when they cannot read degraded text — they "see" plausible content instead. A more detailed prompt reduces the rate but does not remove the behaviour; the tendency is baked into next-token generation, not into the instructions.

Worse, "self-correction" can make things worse. In the head-to-head benchmark by Crosilla, Klic and Colavizza, Benchmarking Large Language Models for Handwritten Text Recognition, asking GPT-4o to correct its own transcription of historical Italian pushed its character error rate up — from about 61% to 77% — as the model reasoned its way further from the page toward what "should" be there.

For anyone building a genealogical database, this is the quiet catastrophe. A silent u/v substitution, a long-s misread, an abbreviation expanded wrongly — each changes a name, a place, or a date, and then propagates into a searchable index where the error is now invisible. There is no confidence signal telling you which characters were seen and which were guessed. If you work with baptism and burial entries, our method for transcribing parish records faithfully treats exactly this risk.

What the Numbers Actually Say

The temptation to reach for ChatGPT is understandable, because on modern material it genuinely is strong. The confusion comes from generalising that experience to the archive.

In the Crosilla, Klic and Colavizza study — the closest thing the field has to a systematic head-to-head, though it remains a single paper — the pattern is consistent. On modern English (the IAM dataset), GPT-4o and Claude 3.5 Sonnet score around 1.7% CER, beating the specialist baseline comfortably. Then the material gets older, and the picture inverts. On historical Italian (ICDAR-2017), zero-shot GPT-4o reached roughly 61% CER; Transkribus's specialist Text Titan I model reached about 26%. On Early Modern German (READ-2016), every general model tested exceeded 70% CER while the specialist sat near 41%.

Two honest caveats travel with those figures. First, on some corpora the gap narrows or even reverses: on the English historical Bentham set GPT-4o did well, and on one Norwegian historical set a general model was marginally ahead of the specialist in some configurations — so "specialist always wins" would overstate it. Second, model versions turn over fast; the exact numbers above predate the current GPT-4.1 and Claude 4 generations and should be read as a pattern, not a leaderboard. What the pattern shows is robust across datasets: as script grows older, more degraded, and less English, general VLM error rates climb by roughly an order of magnitude, and the specialist tools built on curated historical corpora pull ahead.

This is also why the story that general AI is quietly replacing specialist HTR has it backwards. The specialist models hold proprietary, curated training data on exactly the hands that matter — data the general providers have little incentive to chase, since historical transcription is a very low-resource task.

The Practice Worth Keeping

None of this means the general models are useless to a historian. It means they belong at the right stage.

The evidence-based rule from the benchmark literature is clean: use a specialist reader for the image, and reserve general LLMs for downstream work on a transcription you have already verified — translation, summarisation, glossary-building, named-entity extraction. The moment the model is asked to read a degraded hand, its architecture works against you. The moment it is asked to process text that is already correct, that same architecture is genuinely strong.

This is where a purpose-built platform earns its place. Leo's transcription engine, ATR-1, is a Transformer trained on images of historical documents in the Latin alphabet — English, French, German, Dutch, Spanish, Italian, Latin, and the other languages that alphabet records. It weighs high-resolution visual evidence against context rather than defaulting to a language prior, and it is trained to transcribe what is on the page rather than to normalise it: the long s stays a long s, the macron is not silently resolved, the archaic spelling survives. On a randomized 97-image sample of early-modern English manuscripts from the Folger Shakespeare Library, ATR-1 scored about 5% character error rate — roughly 61% fewer errors than the next-best model tested, with Transkribus around 13%, Claude Opus around 23%, and GPT-4.1 around 57% (full comparison here). No per-collection model training is required, and it reads out of the box.

Leo keeps the two jobs separate by design. The base transcription stays faithful to the page. Its AI Transform layer — where you can run a model to translate, modernise, or summarise — writes to a new tab and never overwrites the original. That is the safe version of what people are reaching for when they paste a manuscript into ChatGPT: the language model's fluency applied after the reading is trustworthy, not instead of it. Even a low character error rate is too high for a critical edition that asserts every reading, so the base text is a draft for a human to verify — not an oracle. But it is a draft grounded in the page rather than in a plausible guess about it, which is the difference that matters.

The Judgment to Carry Back to the Archive

The deeper point survives any particular model or version number. A tool that generates fluent text will always give you fluent text, whether or not it could read the page — and fluent text is the hardest kind of error to catch, because it reads exactly like a correct transcription. The skill worth keeping sharp is the old one: knowing your hand, knowing your abbreviations, and reading the machine's output against the image rather than instead of it. Whatever reads the page for you, the page is still the authority. The moment you stop checking it, you have stopped doing history and started trusting a very confident guess.

Frequently Asked Questions

Can ChatGPT transcribe handwriting from historical documents?

ChatGPT can transcribe clean modern handwriting reasonably well, but it struggles with historical or degraded hands for two structural reasons. It downsamples the image before its language model reads the page, so fine marks like a macron or a long s are averaged out before reading begins. And because it generates the most statistically plausible text, it invents confident, fluent readings when it cannot see a word rather than flagging the gap. On older, more degraded, less-English scripts, specialist recognition tools built on curated historical corpora pull decisively ahead, often by roughly an order of magnitude in error rate.

Why does ChatGPT make up words when reading old manuscripts?

ChatGPT invents words because it is a text generator at heart: its decoder carries a strong learned prior about what English, French, or Latin usually looks like, and when the visual evidence is thin, that prior fills the vacuum. Faced with an illegible word, it emits the most probable continuation — a grammatical, period-appropriate, entirely invented reading. This differs in kind from OCR and HTR errors, which produce visible garbage or low-confidence guesses you can catch. A VLM's fabrications look exactly like correct transcriptions, and there is no confidence signal telling you which characters were seen and which were guessed.

Does zooming in help ChatGPT read fine details on a manuscript?

No, zooming in does not help, because ChatGPT never looks at your zoomed image directly. Vision-language models tile the image into fixed-size patches and project each into a fixed token budget; in high-detail mode an image is fit inside a 2048×2048 envelope, split into 512×512 tiles. A densely written folio may have to represent a whole cluster of words in a single tile, and anything below the effective per-patch resolution is averaged into its neighbours before the language model receives a single token. The model reads a compressed embedding, not your high-resolution scan.

Can I ask ChatGPT to correct its own transcription to improve accuracy?

Asking ChatGPT to self-correct can make results worse, not better. In one head-to-head benchmark, asking GPT-4o to correct its own transcription of historical Italian pushed its character error rate up from about 61% to 77%, as the model reasoned its way further from the page toward what "should" be there. This is not fixable by prompting harder; the tendency to generate plausible text rather than abstain when it cannot read is baked into next-token generation, not into the instructions. A more detailed prompt reduces the rate but does not remove the behaviour.

When is it safe to use ChatGPT with historical documents?

It is safe to use ChatGPT for downstream work on a transcription you have already verified — translation, summarisation, glossary-building, or named-entity extraction — but not for reading a degraded hand directly. The moment the model is asked to read an image, its text-first architecture works against you. The moment it is asked to process text that is already correct, that same architecture is genuinely strong. The evidence-based rule is clean: use a specialist reader for the image, then apply the language model's fluency after the reading is trustworthy, not instead of it. The page remains the authority.