Climate Data Rescue: Recovering the World's Weather from Handwritten Records

Climate data rescue is the work of recovering pre-industrial weather observations from ships' logbooks, station registers, and observatory ledgers before they are lost. This article explains what the work involves, why turning handwritten records into structured data is the stubborn bottleneck, and where the field stands on automating it — written for the data-rescue teams, meteorological archivists, and volunteers who do it.

Climate data rescue is the recovery, imaging, digitisation, and validation of historical environmental observations at risk of being lost — the ships' logbooks, station registers, and observatory ledgers that hold the only surviving record of pre-industrial weather. The WMO's 2024 Guidelines on Good Practices for Data Rescue frame it as a seven-stage pipeline running from locating records through transcription to quality control and dissemination. The hardest, most labour-intensive stage is turning handwritten observations into structured, machine-readable data. It is also the stage that gates everything downstream, because a reanalysis is only as good as the observations fed into it.

This article works through what climate data rescue actually involves, why the transcription stage is so stubborn, and where the field currently stands on automating it. It is written for the people doing the work: data-rescue teams, meteorological archivists, and the volunteers who keep these projects alive.

Why the pre-industrial record is worth recovering

Modern climate reanalysis depends on long, continuous observational series. The NOAA-CIRES-DOE 20th Century Reanalysis (20CR v3) is an 80-member ensemble spanning 1806–2015 at roughly 80 km resolution, and it assimilates only surface pressure observations to reconstruct four-dimensional weather maps with uncertainty estimates. The further back that record extends, the better we understand natural variability against which recent change is measured.

The problem is supply. Direct instrumental observations from before about 1900 are sparse, and most of what survives has never been digitised. Even ICOADS — the International Comprehensive Ocean–Atmosphere Data Set, the world's most extensive archive of surface marine observations, spanning 1662–2014 in the self-describing IMMA format — contains only a fraction of the observations believed to survive in ship logbooks. Vast portions of pre-1900 land-station records remain undigitised and physically at risk in archives. Each rescued page is potentially an observation that no reanalysis has ever seen.

It helps to be precise about what a rescued observation is and is not. Reanalysis products like 20CR v3 are gridded model fields constrained by assimilated observations — model output informed by rescued data, not the rescued data themselves. The rescued observation is the primary source. That distinction matters for the same reason source integrity matters throughout the recovery of historical climate and scientific data: once a misread digit enters the pipeline, it can propagate silently into a physically plausible field that no downstream check will flag.

The seven stages, and where the work concentrates

The WMO stages are, in sequence: search and locate; inventory; preservation and storage; imaging and digitisation; keying and transcription; validation and quality control; archiving and dissemination. Coordinating bodies exist to reduce duplication across this pipeline — ACRE facilitates data rescue in support of long reanalyses, and the Copernicus C3S Data Rescue Service consolidates metadata catalogues and guidelines across more than 50 countries.

In practice, imaging is now the easy part. A modern archive can photograph a logbook quickly and safely. The bottleneck sits at stage five: turning those page images into structured numbers. This is where a project lives or dies, and it is where the character of climate records makes the work distinctively hard.

What makes weather records harder than prose

Most historical transcription concerns running text. Weather records are tabular: dense ruled grids of dates, coordinates, barometric pressures, and temperatures, often with idiosyncratic column conventions that shift between observers and decades. Three properties compound the difficulty.

First, layout is load-bearing. A value means nothing without its row and column. Table-structure analysis is frequently the first point of failure in any automated approach — get the grid wrong and every value lands in the wrong field.

Second, digit-level precision is non-negotiable. In prose, a single wrong character is usually recoverable from context; a reader infers the intended word. In a pressure column, there is no context to recover a misread 3-for-8. The error simply becomes a false observation, and false observations are what climate trend detection is least able to tolerate.

Third, the physical page fights you. Faded iron-gall ink, foxing, show-through from the verso, water damage, and the varied hands of successive observers all degrade legibility — and the abbreviations are local, invented by whoever kept the book.

A note on language: these records are vernacular

A persistent misconception is that old scientific records are "mostly in Latin." They are not. Historical instrumental observations are overwhelmingly in vernacular European languages — English, Dutch, French, Spanish, German, Portuguese — with Latin appearing, if at all, only in early metadata headers, not in the substantive columns of readings. The CLIWOC project captured this directly: it digitised 287,114 logbook records from 1,674 ships across British, Dutch, French, and Spanish fleets for 1750–1854, producing the first large multilingual marine climatology.

This matters for tooling. What these records have in common is not a language but a writing system: they are almost entirely Latin-script. Any transcription approach for climate data rescue has to read the Latin alphabet across several languages and several national handwriting conventions — a question of script and hand, not of language count, and one worth understanding before choosing a method. Our companion guide to which languages and scripts HTR can read develops that distinction.

The three approaches on the table

Citizen-science transcription

The current default for large backlogs is crowdsourced transcription, and its successes are real and substantial. Rainfall Rescue, led by Ed Hawkins at the University of Reading, launched on 26 March 2020 and — with more than 16,000 volunteers — produced approximately 5.3 million monthly rainfall totals from roughly 66,000 scanned pages of UK Met Office observer forms in sixteen days, extending the official UK precipitation series back to 1836. Old Weather on Zooniverse has fed millions of marine observations from Navy and whaling logbooks into ICOADS, and McGill's DRAW project has done the same for Canadian observer forms.

The strength of the crowd is also its constraint. Volunteer campaigns are bursty and campaign-dependent. Rainfall Rescue's sixteen-day sprint drew on a surge of pandemic-era attention; sustaining that engagement for slower, more complex, multilingual, or narrative records is not established. Double-keying — two independent transcriptions reconciled — is the quality norm, which effectively doubles the human effort per page. The approach is accurate and community-building, but it does not scale predictably to the size of the surviving backlog.

General OCR and general-purpose AI

The reflex is to reach for OCR, or lately for a general chatbot. Both struggle here, for different reasons, and the failure mode of the second is the more dangerous.

Conventional OCR is engineered for clean, modern, high-contrast type. It segments the page and classifies glyphs against that expectation, and it is good at what it was built for. Historical weather forms violate every assumption in it: ruled tables it cannot parse, mixed hands, faded ink, and abbreviations it has no model for.

General large language models fail differently. Because they lean heavily on textual priors and downsample the high-resolution image, they tend to produce fluent, plausible readings — output that looks like a well-kept logbook whether or not it matches the page. In prose that is bad; in a data column it is corrosive, because a fabricated-but-plausible pressure value is exactly the kind of error no reader catches by eye. Our detailed treatment of why LLMs produce fluent but wrong transcriptions unpacks the mechanism, and why ChatGPT struggles with historical handwriting walks through it on real material. The short version: an error you can see is recoverable; an error dressed as a confident number is a liability.

Specialist and tabular-aware HTR

The third route is specialist handwritten text recognition — models built for historical hands rather than modern type. This is the fastest-moving area, and it is where the honest picture requires care.

Recent tabular-aware models are genuinely improving. Singh et al. (2025) report a 0.035 character error rate on benchmark data, outperforming prior methods by up to 41 percent. But those figures come from curated benchmarks, and performance degrades substantially on heterogeneous real-world forms — dense grids, mixed handwriting, faded ink, local abbreviations. The most sober recent assessment, Zhang's 2025 review of automation in historical weather data rescue, concludes that automatic transcription "has not been reliable or sufficiently accurate on handwritten data" for climate-quality work — which is why most rescued records still depend on human transcription or human verification of machine output. That caveat should travel with every benchmark number you read in this field: a strong score on a clean set is not a promise on a foxed 1840s logbook.

The emerging consensus is therefore hybrid: a machine first pass, followed by targeted human verification concentrated on the high-stakes tokens — the digits and coordinates where a single error corrupts a data point. Rigorous comparative studies quantifying the throughput-versus-accuracy trade-off across pure-human, pure-machine, and hybrid workflows remain sparse, so the exact balance is still being worked out project by project.

Where a specialist transcription tool fits the pipeline

The hybrid model is where a purpose-built transcription instrument earns its place — as the first pass whose defining virtue is that it does not smooth the record. This is the stage Leo is built for. Its transcription model, ATR-1, reads Latin-script material — handwritten and printed, across the vernacular European languages these logbooks are actually written in — with a commitment to source integrity: it transcribes what is on the page, preserving strikethroughs, marginal additions, tables, and archaic conventions rather than normalising them into modern, plausible-sounding output. For tabular material that is the whole point, because the danger in this work is precisely the confident, fluent correction. Results on complex tabular layouts can vary, which is exactly why the pass is a first pass: the workflow keeps the base transcription intact and separate from any later analysis, so a human verifier always checks output against the original image, and corrections feed back rather than overwrite.

Two properties matter for a data-rescue team specifically. It is zero-shot — there is no per-corpus model to train before you can start reading a new logbook series — and finished transcriptions export to structured formats, including TEI XML and Word, ready for the validation and quality-control stage. It reads modern print as readily as manuscript, which matters when a series mixes printed observer forms with handwritten entries. What it does not do is capture images or run your downstream assimilation; it occupies the transcription stage of the WMO pipeline, before archiving and after imaging, and is honest about that boundary. On accuracy claims for this specific material it stays qualitative — the mechanism, not a promised number, is the argument, and the number that matters is the one your own verification produces on your own records. Teams running large openly-licensed rescue projects can also apply for free transcription credits, in keeping with the field's culture of open data.

Building a rescue workflow you can defend

Whichever tools you choose, the shape of a defensible workflow is consistent. Image at high enough resolution that a verifier can resolve individual digits. Preserve layout, because a value without its cell is not an observation. Run a first pass — human, machine, or both — and then concentrate scarce verification effort where errors are unrecoverable: the numbers, the coordinates, the dates. Record provenance and QC flags in a self-describing format so that a future user knows not just the value but how confident to be in it, the way IMMA carries per-element flags. Keep the raw transcription distinct from any correction or interpretation, so the chain back to the page is never broken.

None of this is quick, and none of it is finished. No reliable global estimate exists for what fraction of surviving observations has even been digitised, let alone validated. The backlog in the world's meteorological archives is large enough that it will take machines and people together, working carefully, for a long time. What the field cannot afford is a shortcut that trades a visible error for an invisible one — because the whole value of a rescued observation lies in being able to trust it a century after the hand that wrote it has gone.

Frequently Asked Questions

What is climate data rescue?

Climate data rescue is the recovery, imaging, digitisation, and validation of historical environmental observations at risk of being lost — the ships' logbooks, station registers, and observatory ledgers that hold the only surviving record of pre-industrial weather. The WMO frames it as a seven-stage pipeline running from locating records through transcription to quality control and dissemination. The hardest, most labour-intensive stage is turning handwritten observations into structured, machine-readable data. It also gates everything downstream, because a climate reanalysis is only as good as the observations fed into it. The work matters because instrumental observations from before about 1900 are sparse, and most that survive have never been digitised.

Why is transcribing weather records harder than transcribing ordinary handwriting?

Weather records are harder because they are tabular, not prose, and three properties compound the difficulty. First, layout is load-bearing: a value means nothing without its row and column, so getting the grid wrong sends every reading into the wrong field. Second, digit-level precision is non-negotiable — in a pressure column there is no surrounding context to recover a misread 3-for-8, so the error becomes a false observation. Third, the physical page fights you: faded iron-gall ink, foxing, show-through from the verso, water damage, varied hands, and locally invented abbreviations all degrade legibility. False observations are exactly what climate trend detection is least able to tolerate.

Are old scientific weather records written in Latin?

No. Historical instrumental observations are overwhelmingly in vernacular European languages — English, Dutch, French, Spanish, German, Portuguese — with Latin appearing, if at all, only in early metadata headers rather than in the substantive columns of readings. The CLIWOC project captured this by digitising 287,114 logbook records from 1,674 ships across British, Dutch, French, and Spanish fleets for 1750–1854, producing the first large multilingual marine climatology. What these records share is not a language but a writing system: they are almost entirely Latin-script. Any transcription method has to read the Latin alphabet across several languages and several national handwriting conventions — a question of script and hand, not language count.

Can AI or OCR transcribe historical weather logbooks accurately?

Not reliably enough for climate-quality work, at least not on its own. Conventional OCR is engineered for clean modern type and fails on ruled tables, mixed hands, and faded ink. General large language models fail more dangerously: they lean on textual priors and downsample the image, producing fluent, plausible readings that may not match the page — a fabricated-but-plausible pressure value is exactly the error no reader catches by eye. Recent tabular-aware HTR models score well on curated benchmarks but degrade on heterogeneous real-world forms. The most sober recent assessment concludes automatic transcription is not yet reliable or accurate enough, which is why most rescued records still depend on human verification.

What does a defensible climate data rescue workflow look like?

A defensible workflow follows a consistent shape regardless of tools. Image at high enough resolution that a verifier can resolve individual digits. Preserve layout, because a value without its cell is not an observation. Run a first pass — human, machine, or both — then concentrate scarce verification effort where errors are unrecoverable: the numbers, coordinates, and dates. Record provenance and quality-control flags in a self-describing format, so a future user knows not just the value but how much to trust it, the way IMMA carries per-element flags. Keep the raw transcription distinct from any correction or interpretation, so the chain back to the original page is never broken.