DEV Community

Emmanuel Yator
Emmanuel Yator

Posted on

FineChess...!!

#ai #chess

Stockfish can tell you that 17...Bh6 dropped your win chance from 72% to 31%.

What it won't tell you is the part that actually makes you better:

You were trying to create a kingside attack. That idea is fair but Black's reply Bxd5 wins a pawn immediately, and the cleaner try was Nf3, checking every check before you commit.

That gap between engine truth and human understanding is why I built FineChess.

Try it live

The problem I wanted to solve

If you've ever clicked Game Review after a Chess.com or Lichess game, you know the ritual:

  1. Eval bar jumps
  2. Moves light up: Brilliant, Mistake, Blunder
  3. A principal variation appears: Nf3 Bxd5 cxd5 Qe2
  4. You nod politely
  5. You queue the next game with the exact same blind spot

The engine is brilliant at measuring decisions. Humans need help interpreting them.

Club players (roughly 800 to 1800, the range FineChess coaches for) don't lose because they can't read an eval graph. They lose because:

  • They don't know what the move was trying to do
  • They don't see how the opponent punishes it
  • They don't leave with a reusable habit for next time

Existing tools give you a verdict. I wanted a coach sitting across the board.

So FineChess does three things traditional analysis usually separates:

  1. Stockfish scores every move and finds the best lines
  2. An LLM coach ("Coach Rex") turns those facts into plain-language lessons
  3. RAG pulls principles from chess literature so advice isn't just vibes. It's grounded in real chess knowledge

How Stockfish feeds the AI coaching pipeline

FineChess doesn't ask the LLM to "look at a board and think." That would be asking a language model to do an engine's job.

Instead the pipeline is deliberately asymmetric:

PGN / Chess.com / Lichess
        │
        ▼
   Stockfish analysis
   (eval, multipv lines, mate scores)
        │
        ▼
   Move classifier
   (brilliant → blunder, win% loss)
        │
        ▼
   MoveFacts sheet  ← ground truth
        │
        ▼
   Coach Rex (LLM)  ← may only talk about these facts
Enter fullscreen mode Exit fullscreen mode

What Stockfish actually produces

For each position we run Stockfish (python-chess and a cached engine wrapper) and keep:

  • Best move (SAN and UCI)
  • Score in centipawns or mate distance
  • Principal variation (the "if both sides play well" line)
  • MultiPV alternatives (2nd and 3rd best) so explanations can compare ideas and not just recite move one

Results are cached by FEN and depth so replaying a game doesn't recompute the same positions forever.

Classification that matches how humans feel swings

Raw centipawn drops lie in unbalanced positions. Going from +600 to +400 looks huge on paper and often feels like "still completely winning" over the board.

So FineChess classifies with win-probability loss from the mover's perspective, then labels moves across the familiar spectrum: brilliant, great, best, excellent, good, book, inaccuracy, miss, mistake, blunder.

Brilliant detection is its own mini-pipeline: sacrifice candidates, SEE filters, PV lookahead, and guards against calling something "brilliant" when you were already completely winning.

The MoveFacts sheet: the contract with the LLM

Before Coach Rex speaks, the backend builds a strict JSON fact sheet, roughly:

  • Played move and label
  • Eval or mate before and after
  • Win % before and after
  • Best move and best line (SAN)
  • Played continuation line
  • Material delta along best vs played lines
  • Whether the best move is a check or capture
  • Game phase (opening, middlegame or endgame)

That sheet is the coach's only allowed chess reality.

Rex's system prompt is blunt about it:

ENGINE FACTS are ground truth. Never invent moves, evaluations, or lines that are not in them.

And the answer shape is coaching, not commentary:

  1. Intent: what the played move was trying to do (be fair)
  2. Missed idea: name the best move and walk the line in human language
  3. Cost: describe the damage from facts (material, advantage, mate) and never guess
  4. Habit: one reusable thinking rule tied to this position

No "centipawns." No lecture. Roughly club-player English in about 120 words.

That is the whole bet of FineChess:

Stockfish decides what is true. The LLM decides how to teach it.

How RAG grounds the broader coaching voice

Move-by-move analysis is only half the product.

FineChess also has a global coach that remembers patterns across your games: recurring weaknesses, opening mastery gaps, fix plans, recent accuracy. When you ask something like:

"I keep hanging pieces after I castle queenside. What should I watch for?"

the coach shouldn't answer from generic LLM chess folklore.

So we retrieve.

A deliberately lightweight knowledge base

Under backend/knowledge/ we keep public-domain chess literature (starting with Capablanca's Chess Fundamentals from Project Gutenberg). At startup the service:

  1. Loads every .txt book
  2. Chunks by concept boundaries (chapters, parts, numbered titles) instead of dumb fixed windows
  3. Indexes chunks with TF-IDF (1 to 2 grams)
  4. Retrieves the top passages for the player's question
  5. Injects them into the system prompt as CHESS KNOWLEDGE REFERENCE

The instruction to the model is intentionally narrow:

Use them to ground your explanation. Explain in your own coaching voice but let these principles guide your reasoning.

Why TF-IDF instead of a fancy vector database on day one?

Because chess coaching retrieval is often lexical and conceptual ("isolated pawn", "opposition", "back rank"), the corpus is small and high-signal, and a local sklearn index is fast, free, and easy to reason about. We can graduate to embeddings later. The product win was already having literature in the loop.

So FineChess uses grounding at two layers:

Layer Source of truth Purpose
Move coach (Rex) Stockfish MoveFacts Don't invent tactics
Global coach Player memory and RAG literature Don't invent principles

Both are anti-hallucination strategies. They just fight different failure modes.

The technical challenge: LLMs love inventing chess moves

Here's the uncomfortable truth about putting an LLM next to a chessboard:

It will confidently invent a move that isn't legal, wasn't played, and wasn't in the engine line, and it will sound great doing it.

Early prototypes of FineChess did exactly that.

Rex would say you "dropped a rook" when the PV only lost a pawn.
Or cite Qh5 as the refutation when Stockfish's best line never touched the queen.
Or end a sentence on 17... like a broken PGN printer.

That is not a tone problem. That is a trust problem. One fake move and the coach is dead.

The fix: verify, retry, then refuse to freestyle

Coach Rex now runs a cheap but ruthless grounding check after generation:

  1. Parse every SAN-looking token in the reply
  2. Allow only moves that appear in played_san, best_san, best_line_san, or played_line_san
  3. Block dramatic material phrases ("drops a rook", "loses the exchange", "drops a piece") unless the material deltas in the fact sheet actually support them
  4. If the reply fails, regenerate once with the failure note appended
  5. If still bad, try the fallback LLM provider
  6. If still bad or timeout (about 22s), serve a templated answer built only from facts

There is also a separate claim verifier path for structured tactical claims (legality, hanging pieces, simple SEE-style material checks) used when generating richer mistake explanations, with verifier outcomes logged so we can see whether an explanation passed, retried, or fell back to facts.

In other words:

The model is allowed to be eloquent.
It is not allowed to be creative about the board.

That single constraint changed FineChess from "fun demo" into something I'd trust with my own blunders.

A second, quieter challenge was latency. Users click a move and expect a coach, not a research paper. So Rex asks for short answers, uses tight timeouts, and always has a fact-sheet fallback. Correct and slightly plain beats poetic and wrong every time.

What it feels like in the product

  1. Import a game from Chess.com, Lichess, or paste a PGN
  2. Stockfish analyzes in the background while you watch progress
  3. Browse the board with move quality, eval graph, and key moments
  4. Ask Coach Rex about a specific move or chat about patterns across your games
  5. Leave with a habit, not just a color-coded move list

Under the hood it's FastAPI, Stockfish, Gemini/Groq coaching, Supabase auth/data and a React analysis UI. On the surface it should feel like a sharp human coach who somehow watched every game you've uploaded.

Try FineChess

If you've ever stared at an eval bar and thought "okay but what was I supposed to think?" that's the product.

Live app: https://finechess.vercel.app

Import a recent loss. Click the move that hurt. Ask Rex why.

I'd love feedback from other builders working on grounded LLM systems, especially if you've fought the same "confident nonsense" problem in a domain where facts are non-negotiable.

Quick architecture takeaway 

Engine truth  →  Fact sheet  →  LLM narration  →  Grounding check
Literature    →  TF-IDF RAG  →  Coach principles
Player memory →  Pattern context → Personalized habits
Enter fullscreen mode Exit fullscreen mode

Chess improvement doesn't need more numbers.

It needs numbers that can explain themselves.

Top comments (0)