← Back to all posts
News

Hugging Face's Van Strien Benchmarks Open OCR 5x Cheaper, $0.095 per 1,000 Pages

November 20, 2026 · 00:11 UTC · News
On this page 8 sections
Hugging Face's Van Strien Benchmarks Open OCR 5x Cheaper, $0.095 per 1,000 Pages

TL;DR

Daniel van Strien of Hugging Face ran eight open OCR models, shipped between April 2025 and October 2026, over the same 850 pages on one rented GPU. Reading 1,000 pages fell from $0.51 with the oldest model to $0.095 with LightOnOCR-3-0.8B, and a million pages now runs $65-130 in GPU time, about a day and a half on one card. The cheaper models read ordinary pages as well as the old ones did, and feeding the server properly saved about as much as switching to a newer model.


At a glance
The cost of reading a page
$0.095
per 1,000 pages with the october 2026 model
5x
cheaper than the april 2025 model, same image size
$65-130
gpu time to ocr a million pages
8
open models, same 850 pages and same gpu
0.44-0.82%
median character error span across all eight
$6
total gpu bill for the entire comparison

What the benchmark ran

The set is 850 pages: 350 from historical natural history books, 250 from the 1771 Encyclopaedia Britannica, and 250 typed and printed documents from an archive, mostly late 20th century. Of the book pages, 282 have human-checked transcriptions, so accuracy can be scored directly. 63 of them are blank, which matters later.

The eight models, in release order: RolmOCR (April 2025), dots.ocr (July 2025), olmOCR-2, PaddleOCR-VL and DeepSeek-OCR (October 2025), GLM-OCR (January 2026), OvisOCR2 (July 2026) and LightOnOCR-3-0.8B (October 2026). Each ran behind a vLLM server on one Nvidia RTX PRO 6000 rented from Hugging Face Jobs at $2.75 an hour, with every page fitted to 2,048 pixels. The full comparison, every model on every page, cost about $6.

Two steps and a plateau

The curve is two steps with a plateau between them. The two earliest models cost $0.45-0.51 per 1,000 pages. Five models released between October 2025 and July 2026 land between $0.23 and $0.28, close enough that ranking them is noise. The October 2026 model comes in at $0.095.

Cost per 1,000 pages
Two steps and a plateau
earlier modelslightonocr-3
rolmocr apr 25
$0.51
5 models 25-26
$0.23-0.28
lightonocr-3
$0.095
Same 850 pages at 2,048 pixels on one RTX PRO 6000 at $2.75 an hour. Source: Daniel van Strien.

Two honesty checks come with the headline. Fed smaller 1,288-pixel images, the April 2025 model nearly halves its own cost, which shrinks the gap from five times to about three, and a vision-model judge then preferred the larger-image readings on most pages it compared. The fitted trend line halves every 9.6 months and would put a million pages under $50 by the end of 2027, but it rests on eight points, one of which does most of the work. Van Strien calls the projection a toy.

Cheap did not get worse

On the 282 checked book pages, median character error spans 0.44-0.82% across all eight models, and the newest makes no more errors than the oldest. Every model gets more than 99% of characters right on a typical book page, not counting running headers. Error scoring uses the rules behind the BHL OCR leaderboard.

A vision-model judge compared outputs side by side, 50 pages per page type, and preferred the newest model on book pages 35 to 7. That judge is wobblier than the error rates: it flipped its verdict with display order on one page in ten to one in five. Van Strien blind-read pairs himself and cross-checked the judge with four other models on 40 pairs.

Book pages, side by side
The cheapest model wins on quality too
rolmocr apr 2025lightonocr-3
lightonocr-3
35/50
rolmocr
7/50
Vision-model judge over 50 book pages, cross-checked blind by the author and four other models. Source: Daniel van Strien.

Dense 18th-century print is the exception. On the two-column Britannica pages the April 2025 model still wins against every later model except the newest, which ties it 18-19. Several newer models read the long s as an f, and the preference for the old model survives even when those mistakes are excluded.

Half the saving was plumbing

The most useful result for anyone who already serves models has no new weights in it. Van Strien took GLM-OCR from January 2026 and the Britannica pages, and reran his February setup. That setup sent fixed batches of 64 pages and processed between batches, leaving the GPU idle about a quarter of the time, a kitchen that only cooks when a party of 64 is seated and closes the stove between parties.

Switching to a saturate client that keeps the vLLM server fed continuously, moving from an L4 to the RTX PRO 6000, fitting pages to 2,048 pixels and updating vLLM brought the same model from $1.57 to $0.40 per 1,000 pages. Swapping in LightOnOCR-3-0.8B, which is smaller and faster, brought it to $0.13.

Same model, same pages
Where the saving came from
feb recipe
GLM-OCR as run
$1.57 per 1,000 pages
run better
Fed server, big gpu
$0.40, vllm never idle
swap model
LightOnOCR-3-0.8B
$0.13, smaller and faster
1771 Britannica pages. Serving improvements cut the cost about 4x, the newer model about 3x more. Source: Daniel van Strien.

Running it better cut the cost about four times. The newer model cut it about three times more. He puts the queueing, the image size and the engine together at roughly 2.5x and the bigger GPU at 1.5x, and the current vLLM alone runs the two 2025 models 1.2 to 1.5 times faster than the versions that shipped with them. If you serve from a naive loop script, the queue is the cheap upgrade, and it applies to models you already run.

Where the models invent things

Blank pages catch the improvisers. Three models return nothing, which is correct. GLM-OCR, run without its layout step, wrote a sentence on all 63 blanks, usually that the page was too blurry to read. OvisOCR2 answered 24 blank pages with "The quick brown fox", the pangram finally finding steady work as filler. RolmOCR writes made-up tables on most blank pages and repeats an empty table until it hits its output limit on almost half of them.

Ordinary pathologies are worse. On a tax form full of dot leaders, four of the eight models looped, a different four each run, and none finished the page cleanly twice; the readings that did finish ticked empty boxes or dropped lines. On a typed page with a black blotch over the last few lines, three models each invented a different final entry, in both runs, and nothing in their output marks the invention. Output-limit hits ranged from under 1% to 26% of pages depending on model and page type.

Failure modes
What eight models do with bad input
63
blank book pages; three models return nothing
all 63
blanks GLM-OCR called too blurry to read
24
blank pages OvisOCR2 answered with a pangram
26%
worst share of pages hitting an output limit
Every model ran twice; loops and invented entries appeared in both runs. Source: Daniel van Strien.

What a million pages buys you

With the newest model, a million pages runs $65-130 in GPU time, about a day and a half on one rented card for a mixed set, and splitting the work across several GPUs cuts the wall-clock while the bill stays about the same. The big card is optional now: LightOnOCR-3-0.8B ran on a 16 GB T4 at $0.40 an hour for roughly $0.25-0.41 per 1,000 pages. The April 2025 model on a 24 GB card cost $2.10-2.80 per 1,000, because its weights leave room for only a handful of pages at a time.

His advice from February got cheaper to follow: run a few models on a sample of your own pages and read the output, which now costs pocket change at benchmark scale. For anyone sitting on archives, parish records, a back catalog or a scanned corpus they want searchable, a 100,000-page collection crossed from project to errand. Spot-check the blanks and the blotchy pages before you index anything.

Key Takeaways

  • Reading 1,000 pages with open OCR fell from $0.51 (RolmOCR, April 2025) to $0.095 (LightOnOCR-3-0.8B, October 2026) on identical pages, hardware and image size, about 5x cheaper in 18 months, or about 3x against the old model at 1,288-pixel images.
  • A million pages costs $65-130 in GPU rental, roughly a day and a half on one RTX PRO 6000, and the newest model also runs on a 16 GB T4 for $0.25-0.41 per 1,000 pages.
  • Quality held: median character error spans 0.44-0.82% across all eight models on 282 checked book pages, and a vision judge preferred the newest model 35-7 over the April 2025 one on book pages.
  • Serving practice rivaled model progress: a fed vLLM server, a bigger GPU, 2,048-pixel pages and newer vLLM cut GLM-OCR from $1.57 to $0.40 per 1,000 pages; the newer model then took it to $0.13.
  • Dense 18th-century print still favors the April 2025 model, with the newest tying it 18-19, so sample your own pages before committing.
  • Failure modes are silent: invented sentences and tables on blanks, loops on dot-leader forms, fabricated entries under a blotch, and output-limit hits on up to 26% of pages.

Sources: Daniel van Strien, OCR with open models got about 5x cheaper in 18 months, The Decoder, van Strien on X

AIOCRHugging FaceDaniel van StrienLightOnOCR-3Open SourceBenchmarksDigitization
CONSOLE
$