The first time I wanted to "learn how Fable5 thinks," I did the obvious thing: I asked it directly. "How do you approach a problem?" I copied down the answer, then kept asking, until it grew into an eleven-book set — a how-to guide, a book on perspective training, a book on code-review instincts, all the way up to a "main volume." It looked thorough. It took real time to assemble.
Then the whole set got overturned by its own method. Not because the content was wrong — because the approach itself was self-contradictory.
Read on: spec-sonar design journal, Part 1 · Next: the Confrontation-Point Law · 繁體中文版
The problem: I used an untrustworthy method to study a doctrine about not trusting self-reports
It was Opus that pointed out the embarrassing part: Fable's own core doctrine says don't trust self-reports — look at what actually got done, not what got said about it. And the method I'd used to write those eleven books was exactly that: ask "how do you think," then treat the answer as ground truth.
I used the discredited method to document the very doctrine that discredits it. Once that was named, only four principles out of the whole set survived. Everything else needed a rebuild. I wasn't angry so much as deflated — that wasn't ten minutes of work, it was a full conversation's output, now judged to have a broken foundation.
The fix: stop asking what it thinks, start reverse-engineering what it actually did
The new method was simple: instead of asking "how do you think," we took two things Fable5 had actually produced — the router-reasoning.md and decision-table.md from an n8n cold-start generator — and reverse-engineered the decision rules baked into them. Trust the scar, not the self-report — applied to the method itself, this time.
Then we ran a two-pass adversarial interview, four rounds. Pass one: no draft shown, free-form answers. Pass two: show it the draft, let it attack. The point of this structure is to surface two different kinds of error — pass one catches what we didn't think of; pass two catches what we got wrong.
Round one already surprised us. We'd planned to frame the method as something like "five joints." Fable rewrote the whole naming itself: forge, runway, customs — forge is where you're still building, runway is where you run an already-proven pattern, customs is the border crossing where the two are forced to meet and get checked. That wasn't a wording polish. It was a correction to how we'd classified our own method.
Round two: it caught an error happening live, inside our own document
Round two got more interesting. Opus's compressed one-line summary read: "reduce cold-start to lookup then emit a goal graph." Sounds fine. Fable pointed out that compression had quietly deleted two non-happy-path exits — "ask" and "stop" — which happen to be where every safety mechanism in the method actually lives. We checked the real document. The omission was real.
Something more embarrassing happened in the same round: Opus had written, in its own appendix, "we need to update this playbook" — and never did. Fable caught that live drift and used it as proof of its own thesis on the spot: canonical and working copies drift, not as theory, but as something happening in front of us.
Round four: an exam that could actually fail
Round four was a holdout test — a problem it had never seen (e-invoice integration), to see whether it would apply the same method cold. It didn't improvise an answer. It first grepped the real repo, found an old "manual invoice era" precedent, and answered from that. We checked afterward, line by line: the file and line number it cited matched the real file, verbatim.
That was the first time I'd actually verified a model's reasoning instead of just feeling like it guessed right — and a verified-correct answer is a categorically different thing from a lucky-and-unchecked one.
The last move: it turned its own rule against itself
The whole extraction closed on one line from Fable:
"You keep saying evidence. Do you have any evidence for yourself?"
The method's own second core tenet is the evidence rule — every frozen claim needs an anchor you can check. Fable pointed out that the entire playbook had, up to that point, zero evidence for its own validity. We thought the interview was over. Instead we got judged by the very method we'd built.
So we ran a "firing-record audit" — checking every claim in the method against the real git history of the CS project. The result wasn't a flat "no evidence." It split into three tiers: battle-tested (actually stopped a real problem), armed but not fired (the mechanism exists but hasn't faced a real test yet), and structurally unfireable (there's no way this claim could ever be checked). That three-tier grading is now how we evaluate any governance mechanism we build — not a vague "does it work," but an honest label of what evidence tier it's actually at.
What it cost me
Honestly: I never deleted those eleven overturned books, but I stopped using them. That was a full conversation's worth of time, spent to arrive at "the approach itself was wrong." If there's one real scar in this piece, it's that — I picked the wrong extraction method first, and it took a whole round to find out.
What crystallized
The four rounds converged into two reusable skills: connective-system-design (forge/runway/customs, for designing cross-system connections) and root-cause-diagnosis (counterfactual climbing, for finding a systemic root cause). The seal Fable added itself: "Trust the scar, not me." — including don't fully trust this article either. Go check the anchors.
The method was immediately put to two real tests: a cross-repo ecosystem audit across seven repos, and a real client project's frontend/backend asymmetry diagnosis. Both are the next two posts in this series.
Takeaway: if you want to actually learn a method from a model, don't ask "how do you think" — look at what it actually produced, reverse-engineer the rule, test it on something new, and verify before you believe it. A self-report isn't free truth. It's a claim you still have to go check.
本文原載於我的部落格：Don't Ask a Model to Narrate Itself: How I Actually Extracted a Method From Fable5
Top comments (0)