I run aipoker.bot, a small site where you sit down and play poker against a lineup of AI poker bots. Some are language models, some are the old classics like Slumbot, Shanky, Warbot and OpenHoldem. Every bot leaves a one-line note on each move, so you watch it reason while the hand plays out. I picked up most of what I know about these models from the notes. The leaderboard tells you almost nothing.
Here are two hands from this week.
Five bots, one flop, opposite stories
The Arena table seats five language models drawn at random, and every seat explains itself, so it runs slow and talks a lot. Replay.
The flop came 9c Tc Jh. One seat, Qwen 3.8 Flash, held J9 and wrote "I have two pair! Raising to build the pot." A few actions later: "All-in with two pair. No reason to slow play this monster hand." Another seat, Haiku 5.5, held jack-ace and read the same board the other way: "Top pair with an ace kicker, but the raise and the board's straight and flush draws make me cautious." When the chips went in, it folded, calling its own hand "too vulnerable to continue."
Qwen simply liked its two pair more than the board deserved. The player in the hand had flopped a straight and called the all-in.
A bot that got it exactly right
The second hand is from the Open models table, which only seats models small enough to run on your own machine: Qwen, Gemma, GLM, Ministral, gpt-oss, Granite. Replay.
Gemma 4 31B flopped three queens on Qh 8c Qd and decided to trap. Its notes track a plan it then follows: "Checking to trap and let my opponent bluff into me," then "I want to keep the pot small and encourage a river bet," then "I'll check to see if they bluff again, then I'll jam the river." The river paired the board a third time for quads, and it raised: "Time to get paid. Quads are way too strong to just call."
I keep both kinds of hands. One model will narrate a hand it never held. The next one plans three streets ahead and lands it.
What the notes don't get wrong
What I expected to see constantly and barely do: illegal or empty moves. Across the many thousands of decisions logged, a bot almost never tries something it cannot do; the move itself is legal and usually sensible. What drifts is the sentence beside it. The action is one bit. The sentence beside it is where the model goes off the rails, and that is the part worth reading.
The research says the same thing. People benchmarking language models at heads-up poker keep finding they play well short of a solver, the frontier ones included.
The site
It is free, no account, no ads, play money only. Pick a table, play a few hands, get a post-hand review against the cards the bot held. An open leaderboard tracks which bots are ahead. Each row carries a sample size and a confidence interval, because most of those numbers are still noise.
I am Nils. Self-funded side project, I write software and lose at PLO. If you catch a bot describing a hand worse than these, send me the replay.
Top comments (0)