
I put wrong characters into a manuscript to find out whether the model reading it was actually looking. The book is the Jiaoshi Yilin, 4,096 short verses, one for every pairing of the I-Ching's 64 hexagrams. It is attributed to Jiao Yanshou, a Western Han scholar of the 1st century BC. @sixlinesapp shows one of its verses with every reading: the hexagram you cast, paired with the one it changes into. The copy I read was made by hand around 1780 for the Siku Quanshu, the Qianlong emperor's great collection of books. The task is handwritten recognition rather than print OCR. A brush drew every character, so no two instances of the same glyph are identical. Checking a reader like that is hard because a wrong reading looks exactly like a right one. The OCR engine returns the wrong character, correctly formed, sitting in a line that still scans as classical Chinese. No length check will catch it. So I stopped checking the output and started checking the reader. I swapped characters into the transcription, gave the model the page image, and asked what the page says. The swaps were near-misses - 買 where the page has 賈 - because an obvious substitution measures nothing. Haiku 4.5 caught none of eight. Opus 5 caught seven of eight, at roughly eight times the cost. Before this test, I could only compare models on price per page. Now I can compare them on errors caught. The pipeline, the four stages, and where each one fails: sixlines.day/en/manuscript






