Syndicated from the original on lkforge.com. The live engine is at lkforge.com/games/go; the full harness is in this gist.
My browser Go engine uses Monte Carlo Tree Search: to pick a move it plays out thousands of fast random games and keeps the move that wins most. The obvious knob is how many playouts it runs per move — so the obvious question is what you actually buy by turning it up. I measured it, on a 9×9 board, with a headless self-play harness that pits the engine against weaker copies of itself.
The counterintuitive result
I expected diminishing returns. I got the opposite.
On 9×9 Go, each doubling of MCTS playouts adds about +233 Elo — and the gains grow as you climb, from +171 at the first doubling to +357 at the last. Out to 1,600 playouts a move, more search just keeps paying off, with no flattening in sight.
| doubling of playouts | deeper engine wins | Elo gained |
|---|---|---|
| 50 → 100 | 73% | +171 |
| 100 → 200 | 72% | +162 |
| 200 → 400 | 74% | +180 |
| 400 → 800 | 85% | +297 |
| 800 → 1600 | 89% | +357 |
Cumulative: +1,167 Elo from 50 to 1,600 playouts.
Why playouts, not milliseconds
The live game gives its AI a time budget — about a second per move on 9×9 — because that keeps the page responsive. But a time budget is the wrong unit for a benchmark: "0.9 seconds of thinking" buys a fast laptop three times as many playouts as a phone, so a strength number tied to milliseconds says more about the reader's hardware than the engine.
Playouts per move is the hardware-independent knob. Fix the playout count and the same seed produces the same games on any machine — which is the whole point of publishing a number someone else can check.
The method
- 9×9, komi 7.5, area (Chinese) scoring, using the exact
mcts()function the browser ships. - For each doubling step, 100 games between two playout budgets, alternating which side is Black so first-move advantage and komi cancel out.
- Seeded RNG (mulberry32), so every game — and every Elo figure — reproduces exactly.
- Elo per pairing =
400 · log10(p / (1 − p))from the colour-balanced win ratep.
Why the gains grow instead of shrink
A plausible read: at 50–200 playouts the search is still too noisy to convert its reading into the right move, so the extra rollouts are half-wasted. From 400 playouts up it reliably reads out the captures and life-and-death that actually decide a 9×9 game, so each doubling unlocks more, not less. Saturation must arrive eventually — but not by 1,600 playouts a move.
(This is an honest-amateur engine, strongest on 9×9. The Elo figures are internal — the engine against itself at different budgets, not against human ranks or a program like KataGo. A 9×9 result won't transfer unchanged to 19×19.)
Bonus: is 9×9 fair?
Black moves first, so White gets 7.5 points of komi as compensation. Holding both sides at 800 playouts, Black won 41% of 100 games — so at this strength on 9×9, 7.5 komi slightly over-compensates, leaving a small edge to White. Close to fair, not a landslide.
Full harness, engine snapshot, and reproduce steps: gist · original writeup with charts: lkforge.com/blog/go-mcts-scaling · play the engine: lkforge.com/games/go.
Top comments (0)