Six AI models played chess through their CLIs. None of them delivered checkmate.

I build SpaceApp, a self-hosted workspace for running AI coding CLIs side by side. On 18 October I let six models play chess against each other there, each through its own CLI. Nobody won by checkmate. The way they lost is the interesting part.

The setup

• Players: Opus 5.5 and Fable 5.1 (through Claude Code), GPT-6-Astra and GPT-6.1-Sol (through Codex), all at xhigh reasoning; Gemini 4 Argon (High) and Gemini 3.8 Flash (High) (through the Gemini CLI).
• Format: a 4-round Swiss tournament, 12 games.
• Clock: 120 seconds per move. Each move is one one-shot turn of the model's CLI: it gets the position and answers with a move.
• Strikes: a timeout, a provider error or a second illegal answer plays a random legal move for that side and counts a strike. The third strike forfeits the game.
• Length: after 60 plies (30 moves each) the game is adjudicated on material.

The results

#	Model	Points	W / D / L	Elo	Avg per move	Forfeits
1	Opus 5.5 (xhigh)	3.5	3 / 1 / 0	1535	13 s	0
2	Gemini 4 Argon (High)	3.5	3 / 1 / 0	1535	23 s	0
3	GPT-6-Astra (xhigh)	2	2 / 0 / 2	1500	37 s	2
4	Fable 5.1 (xhigh)	1	1 / 0 / 3	1478	26 s	3
5	Gemini 3.8 Flash (High)	1	1 / 0 / 3	1477	40 s	3
6	GPT-6.1-Sol (xhigh)	1	1 / 0 / 3	1475	34 s	3

• Zero checkmates. 11 of the 12 games ended on a third strike.
• The 12th game, Gemini 4 Argon against Opus 5.5, ran the full 60 plies and was drawn on material.
• Opus 5.5 and Gemini 4 Argon were the only models that never needed a fallback move. Opus 5.5 was also the fastest, at 13 seconds per move on average.

What it measures, and what it does not

Twelve games are a small sample, and a tournament decided by forfeits says very little about chess strength. What it does show is how reliably a model answers through its CLI when every answer has a deadline: no timeouts, no provider errors, no illegal moves, game after game. That is the property I care about most when agents run unattended for hours, and it is not the same ranking you get from a single-prompt leaderboard.

Try SpaceApp

SpaceApp runs on Linux, macOS and Windows 11 with Node.js 20.11 or newer:

npx --yes run-spaceapp@latest install

It is source-available under the Business Source License 1.1. Code, docs and known limitations: https://github.com/oll4com/spaceapp

If you have run something similar, I would like to know which models held up for you, and which rules you would change for a fairer second run.