Ask GPT-4 or Claude to play a real game of chess and it will, with total confidence, try to move a knight like a bishop. Or capture its own queen. Or castle out of check. I've watched a frontier model announce "checkmate" on a board where the king had four legal escape squares.
This is the whole problem I ran into building aichess.guru, an AI coach that tells you why your move was bad in plain English. Turns out the "plain English" part is easy. The "not lying to the user about the position" part is where everything breaks.
LLMs don't have a board
Here's the thing people miss. A language model doesn't see a chessboard. It sees a string like rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR. It has read millions of games in that notation, so it can autocomplete plausible-looking moves. Plausible is not legal. Plausible is not correct.
The failure mode is nasty because it's confident. The model won't say "I'm not sure where the rook is." It'll tell you the rook on a1 forks your queen and king when there is no rook on a1. A beginner reading that has no way to know it's nonsense. That's worse than saying nothing.
So rule one for me became: the LLM is never allowed to decide anything about chess.
Split the job in two
Every chess claim in the product comes from real software. Not the model.
- Legality, checks, checkmate, what piece is where: chess.js. It's a boring, deterministic library.
game.move("Nf3")returns null if the move is illegal.game.isCheckmate()is a fact, not a vibe. - Evaluation, best move, "how bad was this blunder": a real engine (Stockfish). It gives you a number. -3.4 means you're down a piece and change. It doesn't hallucinate the number.
The LLM only ever gets handed verified facts and asked to phrase them like a patient human coach. "You dropped your knight on f5 for free" instead of "eval swung from +0.2 to -3.1." The model is a translation layer, nothing more.
I have a hard rule in the codebase, written in a comment so I don't forget it at 2am: AI generates feelings, never chess. If a number appears in the output, an engine produced it. If a board appears, chess.js rendered it from a legal move sequence. The model gets zero authority over the game state.
The part that surprised me
I assumed the hard bit would be prompt engineering the explanations. It wasn't. Give a decent model the real eval, the real best move, and the real refutation line, and it writes a genuinely good explanation on the first try. Models are great at "explain this fact simply." They're terrible at "produce this fact."
The hard bit was FEN plumbing and trusting nothing. Every position that gets shown to a user is computed from a starting position plus a list of moves, each one validated. I don't hand-write FENs anymore. I did once, shipped a smothered-mate example with the rook and king swapped, and it rendered a perfectly legal-looking board that was the wrong position. Both parse. Only one is the mate you're talking about. That bug is why the "compute, never type" rule exists.
Why this matters beyond chess
Chess is just a clean test case for a pattern that shows up everywhere you point an LLM at a domain with actual rules. Law. Medicine. Accounting. Anything with a source of truth.
The move that works: let the deterministic system own correctness, let the model own communication. Don't ask the model to be the calculator. Ask it to be the friend who explains what the calculator said.
People keep trying to make the LLM do the whole job because it looks like it can. It'll happily produce an answer. It just won't tell you when the answer is invented.
Try it
The product is live and free to start: aichess.guru. Play a game, blunder something, and it'll tell you exactly where it went wrong in words a human understands, with the engine backing every claim.
If you've built LLM stuff on top of a rules engine and hit different walls, I want to hear about it. What did you let the model own, and what did you rip out of its hands?
Top comments (0)