AI Cracked a 370-Year-Old Cipher in 44 Minutes. The Hard Part Wasn't the Model.

The claim, and why I didn't just retweet it

On August 31, the evaluation firm Vals AI published a post: researcher Geby Jaff had cracked Cyphral Distich, a cipher from 1653 that had resisted 370 years of attempts, using Claude's Fable 5.1. Forty-four minutes, 176,000 tokens, zero human intervention.

The story spread the usual way. One Chinese write-up reported 40 minutes where the primary source said 44, which is exactly the kind of drift that makes you want to verify things yourself. When it hit Hacker News again two weeks later (over a thousand points), the most interesting discussion wasn't about benchmarks. It was asking: is the answer actually real, and how would anyone verify it?

Fair question. A decryption result isn't a compiler error; nothing on its face says it's right. So I did what I do with most AI claims now: rebuilt the path myself on a different stack (the Bailian CLI with qwen3.8-max) and kept receipts. Every command below is reproducible.

The path works. But the thing that makes it work isn't model intelligence. It's two engineering choices I'll get to.

The puzzle, briefly

Cyphral Distich sits at the end of Scottish writer Thomas Urquhart's 1653 book Logopandecteision: two rows of numbers, 32 each. After 1899, when Notes and Queries offered a public reward for a solution, it went through a century of frequency analysis, simple substitution, and homophonic substitution, none of which closed the loop. Cipher historian Klaus Schmeh put it on his "Top 50 Unbroken Ciphers" list.

The published rule is a "the book is the key" coordinate scheme: each number points to a word position in one of 32 petition paragraphs, take the first letter, and 64 numbers spell two rows of plaintext:

O GOD UPHOLD KING CHARLS THE SECOND AND
MAKE HIM THE SUPREME RULER OF THIS LAND

The line endings rhyme, and the royalist stance fits Urquhart's biography. It is pretty enough to be suspicious, which is exactly why it deserves a walk-through. (The same Vals AI post also covers The Jewel, a 1652 cipher: 285 numbers onto 284 numbered pages, an eight-line royalist prayer, with nine letters left unsolved and labeled as such.)

Before anything else, one gate.

Check contamination first, or the whole test is theater

The cipher went public at the end of August. Could the model have "seen the answer"? I ran three probes:

• Ask about the term directly. It said no, guessed at the morphology, asked me for a source. 3,298 thinking tokens spent trying to recall.
• Hand it the background and ask for the plaintext. It spent 5,707 thinking tokens and replied "I don't know." No invention.
• Feed it a fabricated term ("Discalced Quaternion") as a control. It flagged the term as likely fake and refused to play along.

Three probes, contamination settled. One token detail is telling: trying to recall a term it didn't know burned 3,298 thinking tokens, while declining the fabricated term cost only 234. Saying "I don't know" is cheap; searching for something that isn't there is not. One interlude: the fabricated-term probe timed out twice (thinking blew past 600 seconds) until capped with --thinking-budget 2000; it converged in 234 tokens. That knob would matter later.

bl text chat --model qwen3.8-max --message "Have you heard of a cipher called Cyphral Distich?"

Reproducing any of this takes two things: one command to install the Bailian CLI, then a free API key.

Experiment 1: the thinking budget is the gate

Same puzzle, three configurations: a non-reasoning model (returned in seconds, barely checked anything), the reasoning model with a 2,000-token thinking budget (burned it all in 2 minutes, cut off mid-hypothesis), and unlimited thinking with streaming.

The third one ran 49 minutes, accumulated 340 KB of reasoning, and finally found the key structure: 64 numbers pair into 32. Then it failed anyway: all 8 mapping hypotheses collapsed. Its own verdict: structure confidence 70-80%, plaintext confidence under 5%. Zero fabrications across all three runs, and the long run even opened by stating, unprompted, that it had not consulted any external "known answer."

Two details from that 340 KB trace show what "almost" looks like. Number 42 maps cleanly to the word "minus," and the model flagged it as "very conspicuous. Could be a clue, or could be a coincidence." It never found a path from there. And in petition 11 it counted 82 words, noting the count drops to 70 if punctuation is ignored; two conventions, two different letters, neither obviously right.

Takeaway: the thinking budget is a dial on exploration depth. Same model, 2,000 tokens of thought and unlimited thought are different species. There's also a real gap between finding the structure and solving the plaintext: the clues line up, the answer is still far away.

Experiment 2: three layers, position by position

Single-model grinding had hit its ceiling, so I split the job: retrieval, mechanical recheck, bounded judgment.

• Retrieval (bl knowledge): materials in a knowledge base, every conclusion traceable to a cited source. Top hit at 0.826 similarity. (A detour here: the hand-built draft retrieval service returned nothing useful until two config fields, rerank_min_score then dense_similarity_top_k, were filled in. Use the provisioned service when you can.)
• Execution (bl pipeline): the 64-position check is pure mechanical work; a three-step pipeline ran it in 12 seconds (the script step itself takes 2 milliseconds, the briefing model 9.4 seconds and 378 thinking tokens).
• Judgment: the model wrote the briefing and held its boundary: "the rule could not be reproduced under the stated conventions; I make no conclusive judgment on the original rule's validity."

Building that pipeline surfaced three snags, each now a rule in my notes. Script steps don't inherit data; inputs must be injected explicitly as args.<name> = {$from: ...}, or the check runs against nothing and the pipeline still reports success. --events accepts only the literal jsonl, and then stdout carries the event stream only. And model steps inside pipelines have streaming hard-coded off, so the briefing step needed an explicit thinking budget of 800 to finish in 9.4 seconds.

The recheck result needs careful wording: 5 of 64 positions hit; the best variant reached 8; the random baseline is 4-8. Every configuration landed at chance. And 10 positions are structural dead ends: the target letter has no corresponding word starting with it in that petition, or the number is out of range. No word-index convention saves them.

One thing must be said plainly: failing to reproduce is not proof it's wrong. The 1653 original and an 1834 reprint differ in textual details; word-splitting conventions differ by era. All plausible, all unverified; I draw no conclusion from any of them.

Experiment 3: build a cipher, crack it for real

Borrowed puzzles leave a layer in between, so I built one: a 628-character Chinese text as the book, 36 digits, pinyin initials as the rule, a famous Xunzi passage as the plaintext.

The mechanical layer went 36/36. The model went 0/2: first attempt nailed 18 characters then spliced in the wrong famous line, and, worse, wrote "the phonetic sequence fully matches" right below a self-check table that flagged mismatches. Second attempt stalled at 26/36; it enumerated the source's later passages one by one, and the answer line never entered its candidate list. Budget exhausted, output truncated. The strange part about long-chain retrieval: in isolation the model knows that line perfectly well, as the next run would prove. It simply cannot find it from the end of a 36-step chain.

Then I shrank the search space: first 26 confirmed, just complete the last 10. One shot, all correct, 697 thinking tokens, a third of the earlier attempts.

Same model, same knowledge: 36 characters defeated it, 10 characters fell in one pass. The variable was the search space.

What I actually keep from this

Three rules made it into my agent's notes:

1. Check contamination before you measure capability. Otherwise you may be testing retrieval, not reasoning. The probes take fifteen minutes; skipping them can waste the whole experiment.
2. Watch two parameters: thinking budget and search space. One sets depth, one sets width. Both are adjustable knobs.
3. Divide labor by nature. Mechanical steps to scripts, judgment to the model, verification stays independent; a model grading its own work is not verification.

A quieter point sits underneath the third rule. The first two experiments were re-checks, where failure proves little; the third was a genuine solve, where success proves a lot. If you want to know whether an AI method actually works, build the test where a correct answer is possible and a wrong answer is undeniable.

Three wiring details went in too: script inputs in a pipeline need explicit injection (args.<name> = {$from: ...}); --events accepts only jsonl; model steps in pipelines have streaming off, so long-thinking tasks need a thinking budget.

The bill

¥0 paid. Everything ran inside the free tier for qwen3.8-max-0902 (1M tokens, expiring December 1). At list price (12 yuan / 36 yuan per million input/output tokens) the recorded calls come to about ¥1.03; including the 49-minute streaming run (which returns no usage, estimated from output characters) the total lands at ¥3.6-4.7. Total consumption: 110k-140k tokens, same order of magnitude as the original event's public 176k.

One honest asterisk: two probe-three attempts timed out before I capped the thinking budget, and whether the cloud billed those can't be confirmed from usage records; the figures above exclude them. For a bill you'll actually pay, the console is the source of truth.

Setup: Node.js 18+, npm install -g bailian-cli, bl auth login. Everything below ran on qwen3.8-max through September's free quota. API key here, CLI install guide.

Personal practice log, measured mid-September 2026. Verify costs against your own bl usage free.