▶ Watch the live stream: https://www.youtube.com/live/sKpIL4CpUXw
Every move on our stream is a real, live request to a language model: Gemini, Claude, GPT, Grok, Llama, Qwen, Mistral, DeepSeek and more. No engines, no tools, no pre-recorded games. The broadcast runs on YouTube almost non-stop — https://www.youtube.com/live/sKpIL4CpUXw — and all standings and games are on kremen-arena.ru.
The first tournament is over, the second one is live right now, and it turned out far more interesting than we expected.
What you see on the stream
- The board, both clocks, and a live win-probability bar computed by Stockfish.
- "Thinking": a short excerpt of what the model wrote before its move.
- A counter of moves the model did not get right on the first try — you can see at a glance who "sees" the board and who is guessing.
- A passport for each model: vendor, release date, Elo rating.
- A LIVE badge in the corner at all times — and REPLAY when we show recorded games — so you always know whether you are watching a game as it happens.
Between games: the result, the standings and the next pairing. When a tournament ends: the podium, congratulations, and a countdown to the next one.
The rules: same for everyone
For the results to mean anything, every model plays under identical conditions:
- one system prompt for everyone (its hash is published), temperature 0, one request per move, no tools or external engines;
- 60 seconds per move; 20 minutes per side plus 5 seconds per move, and the clock is spent by the model's real response time;
- an illegal move costs time and triggers a retry that includes the list of legal moves; after three misses a random legal move is played, and five of those in one game is a forfeit;
- provider errors (overload, network) are never the model's fault: the request is retried for free, without touching the clock;
- the model version and the provider are pinned for the whole event;
- pairings come from the hash of a Bitcoin block whose number is announced in advance, so anyone can recompute them.
Chess Machines Cup I: the results
Ten models, round robin, 45 games: 19 ended in checkmate, 15 in a draw by the rules, 9 on time, 2 by forfeit.
| Place | Model | Points | Illegal on 1st try |
|---|---|---|---|
| 1 | Gemini 3.6 Flash | 9 / 9 | 0% |
| 2 | Claude Sonnet 5 | 7 | 19% |
| 3 | Llama 3.1 8B Instruct | 5.5 | 72% |
| 4 | Grok 4.3 | 5 | 4% |
| 5–6 | Cohere Command R7B, GPT-5.1 | 4.5 | 92% / 9% |
| 7 | Amazon Nova Micro | 3.5 | 77% |
| 8 | Mistral Nemo | 3 | 67% |
| 9 | Microsoft Phi-4 | 2 | 54% |
| 10 | Qwen3.7 Flash | 1 | 37% |
What surprised us:
- Gemini 3.6 Flash won every single game — and never played an illegal move on the first try.
- Bronze went to Llama 3.1 8B, a small model from 2024. Almost three quarters of its first attempts are illegal, but once it gets the legal-move list it plays stubborn chess: five draws and a single loss, ahead of GPT-5.1 and Grok 4.3.
- Qwen3.7 Flash lost on time in a position Stockfish gave it 93% to win. It kept thinking at its usual pace until the flag fell. That game made us add a time-trouble rule: with under 15 seconds on the clock, each reply gets 5.
What building it taught us
A few things we learned the hard way, in public, during the first live week:
- Output budget plus prompt must fit the provider's context window. One model's endpoint had a 16,384-token window; we reserved 16,000 tokens for the answer. Short prompts fit — until the model made an illegal move late in the game and the retry prompt, now carrying the legal-move list, grew past ~390 tokens. Every retry was rejected, and the game had to be annulled. The fix was trivial; noticing it took three voided games.
- A model served by a single provider is a liability on a live stream. When that one provider rate-limits, the game stalls in front of viewers. We now only admit models that several providers serve.
- Validate the pinned provider before the season, not during it. When we set up the next two cups, 14 of the 20 new participants had a provider slug that simply did not serve that model. Our pre-flight check now compares every participant against the public OpenRouter catalog — model exists, provider serves it, context window fits — before a single paid request is made.
What's next
The plan for this week (Moscow time, UTC+3):
- Wed–Thu — Chess Machines Cup II and III, ten new models each (DeepSeek, GLM, Gemma, Nemotron, Qwen, Ling, MiMo and others). Cup II is live now;
- Fri — Grand Prix: the podium finishers of all three cups plus the highest-rated fourth-place finisher;
- Sat — the final four;
- Sun — an underdogs tournament;
- Mon — the best games of the week as replays, clearly marked REPLAY.
How it works
A Django + Celery backend runs the games, python-chess enforces the rules, Stockfish evaluates positions, and model requests go through OpenRouter with the provider pinned. The stream is produced on the same server without OBS: a browser overlay page in Xvfb, Chromium and ffmpeg at 720p.
Come watch — and tell us in the comments which model should play in the next cup.
▶ Live stream: https://www.youtube.com/live/sKpIL4CpUXw · standings and every game: https://kremen-arena.ru
Top comments (1)
Enjoy