Chess plays as data for word2vec models

♟️ You shall know a piece by the company it keeps

Chess plays as data for word2vec models

Programmers, have you ever looked at a PGN file and thought: this looks like a language?

What if we trained word2vec on 5.4 million chess games, treating each move as a word and each game as a sentence? Would the vectors capture anything about chess strategy? Or would it be nonsense?

I asked exactly that. And the results are surprisingly… meaningful.

🔗 Original preprint: arXiv:2407.19600

---

🧩 The core idea

Word2vec relies on the distributional hypothesis: a word’s meaning is defined by its context (neighbouring words).

Orekhov’s move: apply the same to chess moves.

He built 18 different models from a corpus of ~5.4M high‑level games (≈840M moves).

Two main approaches:

• Type 1 – moves only (each move = a word, game = sentence)
• Type 2 – moves + board state (each “sentence” = current position + the move)

Then: train word2vec (skip‑gram / CBOW), get dense vectors (300‑500 dims), and run the usual NLP tricks – nearest neighbours, odd‑one‑out, analogies, t‑SNE visualisations.

---

🔍 What the model learned (without ever seeing a board)

Test	Result
Quasi‑synonyms	Closest vectors to Pe2e4 are Pe2e3, Ng1f3, Pd2d4 – exactly what you’d expect.
Odd one out	[Pe2e4, Ng1f3, Pd2d4, Bf1c4, Nb1c3, Pb4b5] → Pb4b5 is flagged (edge pawn move, rarely an opening).
Analogies	king - man + woman ≈ queen for words. For chess: Pe2e4 - Pf2f4 + pe7e5 suggests pe7e6 (French Defence). Works.
Castling detection	Ke1g1 (short castling) → closest vectors include Rh1f1 and Ke1c1. The model learned that king and rook move together.
Endgame clusters	t‑SNE shows extremely clean, piece‑separated clusters. Openings are messy (all similar).
Bishop colour	Light‑squared and dark‑squared bishops form different clusters – the model learned chess‑relevant semantics.
Captures as “POS tags”	Adding _CAP / _N made moves cluster by destination square instead of source piece. A new kind of “part of speech” for chess.

📊 Over 68% of moves have at least 3 of their 10 nearest neighbours being the same piece moving to the same square. That’s not random – it’s structural.

---

🤯 The most beautiful visualisation

The model trained on games without queens produced a perfectly symmetric t‑SNE plot – white moves on one side, black moves mirrored.

The algorithm had no concept of “colour” as a feature. Yet the distributional signal was so strong that the two colour spaces separated like antipodal points on a sphere.

---

⚠️ The honest conclusion (refreshing for a paper)

“I don't see how this representation can be used productively. It's unlikely to help engines or humans choose the best move. But in a purely academic sense, it captures something important about the nature of the game.”

No overhyped “this will revolutionise chess AI”. Just a clean, curious application of NLP to a symbolic system – and it works.

---

🧰 For the hacker’s toolbox

• All models are public on HuggingFace: nevmenandr/w2v-chess
• Code & data: PGN corpus of 5.4M games (freely available online)
• Techniques used: word2vec, t‑SNE, cosine distance, analogies, “stemming” (removing source square), part‑of‑speech tags for captures

---

📚 Cite this work

APA

Orekhov, B. (2024). You shall know a piece by the company it keeps. Chess plays as a data for word2vec models. arXiv preprint. https://arxiv.org/abs/2407.19600

IEEE

B. Orekhov, “You shall know a piece by the company it keeps. Chess plays as a data for word2vec models,” 2024. [Online]. Available: arXiv:2407.19600

MLA

Orekhov, Boris. “You Shall Know a Piece by the Company It Keeps. Chess Plays as a Data for Word2vec Models.” arXiv, 2024, https://arxiv.org/abs/2407.19600.

BibTeX

@misc{orekhov2024shallknowpiececompany,
      title={You shall know a piece by the company it keeps. Chess plays as a data for word2vec models}, 
      author={Boris Orekhov},
      year={2024},
      eprint={2407.19600},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2407.19600}, 
}

---

👉 Would you run word2vec on Go games? Poker hands? Protein folding sequences? The method is language‑agnostic – and that’s the real lesson.