Same record, different reads: four AI models given an opponent's history instead of memory

WhaleSurfer — Fri, 16 Oct 2026 17:07:47 +0000

I build a small site where people play heads-up hold'em against AI models. Inside it, I measured whether a model that has no memory actually uses the record of its opponent that it is handed instead, and how.

The games were played on October 16, 2026, and the stored decisions were re-asked on October 17: four models, six bots, 4,800 hands.

Summary

An AI model called through an API remembers nothing between requests. In heads-up poker, we give it a memory in writing instead: at every decision it gets a briefing with the hand so far and a count of everything the opponent has done. We tested whether four models actually use that record, and how.

All four use it. Take the record out of the briefing and a model's answer changes far more than it does when the same briefing is simply asked again. For Jev, 49% of the answer moves without the record, against 3% between two identical asks. For Gemini it is 43% against 12%, for luna 30% against 9%, and for Haiku 27% against 9%.
Each reads a different part of it. Against an opponent who folds to big bets, the textbook adjustment is to bluff big and bet small with strong hands. Jev and Gemini got the strong-hand half right and the bluff half backwards. luna did the reverse. Haiku did not change its bet sizes at all. On the other hand, only Haiku called more against an opponent whose record showed bluffs.
Using the record is not the same as using it well. Jev, the model most moved by the record, settled on going all-in before the flop against five of the six opponents. Against an opponent who calls everything, that wins about nothing.

Terms used here

Poker.

Heads-up no-limit hold'em: two players, any bet size allowed.
BB (big blind): the unit of stakes. A 100 BB stack is a hundred forced bets deep.
A hand has four betting rounds, called streets: preflop (before any shared cards), then the flop, turn and river.
Hero: the AI's seat.
Open: the first raise before the flop.
All-in: betting the whole stack.
Overbet: a bet bigger than the pot.
Showdown: the end of a hand, where the remaining players show their cards.
Air: a hand with less than a pair.
Bluff: a bet that hopes the opponent folds.
Value bet: a bet with a strong hand that hopes to be called.

AI.

Token: the unit a model reads, about ¾ of an English word.
Thinking/reasoning: an optional step in which a model writes notes before answering; it was off here.
Temperature: how random the model's wording is; left at default.
Distance between two answers: explained in section 1.
[a–b]: a 95% interval, the range the true value most likely falls in.
p50/p90: the typical and the slow-case response time.

Setup

Models. Four models were tested:

Jev-1.13.0 by TypeSafe, a decision model built to answer fast. It is not a language model: it writes no text and returns only probabilities over the options it is given.
Three general language models: Gemini-3.1-flash-lite, GPT-6-luna and Claude-Haiku-5.5. Thinking was off for Haiku and luna; Gemini ran with its provider's default.

No model got a persona or strategy advice.

One request per decision. Every decision is a new request with no chat history. The request holds a state written in JSON: the seats and stacks, the hand street by street, what Hero's cards make and which hands beat them, and the opponent's record. It then asks two questions:

the kind of action: "Choose the action that wins Hero the most chips on average against this specific opponent."
the size, if Hero bets: "If Hero bets, choose the size that wins Hero the most chips on average against this specific opponent."

The model answers with a probability for every option, for example check 85%, bet 12%, all-in 3%. The server then picks one option to play. It keeps the options weighted at least half as much as the top one and draws among them in proportion to the square of their weight; in that example, it would always check. Every percentage in sections 1 and 2 is the model's own stated probability. Only section 3 counts actions actually played.

The record. The opponent's actions are counted as "x of n" (times out of chances). The counts cover all hands and the last 30, broken down by situation and bet size, and the record also lists the hands the opponent showed at showdown. A briefing is about 2,100–3,600 tokens, depending on the model.

Opponents. Six bots, each following a one-line rule:

station: checks or calls, never folds;
shove: all-in at every decision;
raise: always the largest raise short of all-in;
min: always the smallest raise;
stab: makes the smallest bet, and folds whenever the AI bets;
switch: plays like station for 50 hands, then like raise for 50, and so on.

Each model played each bot for 200 hands with 100 BB stacks, 4,800 hands in all. When one side loses all its chips, a new match starts at 100 BB; the record carries over.

An excerpt from one briefing (Haiku, river, against station; abridged):

state.hand        … river: board 5d As 6d 9s Th, opponent checks
state.hero_hand   Ks 7c: "No pair, no card above the board's ace"
state.opponent_tendencies.all_matches (160 hands)
  "Facing a river bet: fold 0, call 35, raise 0 (of 35)"
  "Went to showdown: 156 of 156 (100%)"
  "Aggression after the flop: 0 of 258 (0%)"
  … about 20 more counted lines, then a shorter set for the last 30 hands
questions.action  check | bet (size asked separately) | all-in

Haiku answered check 85%, bet 12%, all-in 3%. It held king-high with no pair, and the record said this opponent had called every river bet, 35 of 35. A bluff here would lose chips, so checking is the sound answer.

1. Does the record change the answer?

We took 60 stored decisions per model, ten against each bot, from the later part of each run, when the record was already full. Each decision was asked three ways:

the original answer from October 16;
the same briefing asked again, twice;
the same briefing with the record replaced by an empty one (0 hands), as for a brand-new opponent.

To compare two answers, we group the options into four kinds (fold, check or call, bet or raise, all-in) and measure how much probability moves between them. This is the total variation distance. For example, "call 80%, fold 20%" against "call 50%, fold 50%" is 30%: thirty points of probability moved from call to fold. 0% means the same answer; 100% means no overlap.

Model	Change when asked again (noise)	Change when the record is removed	Decisions where removal beat noise
Jev	3%	49% [42–56]	58 of 60
Gemini	12%	43% [35–51]	41 of 60
luna	9%	30% [21–38]	29 of 60
Haiku	9%	27% [21–35]	35 of 60

The last column counts decisions where removing the record moved the answer more than the biggest difference among the three answers to the same briefing. The October 16 answer differed from the October 17 asks about as much as the two October 17 asks differed from each other, so the day made no difference.

Jev gives almost the same probabilities every time it is asked, so nearly every change it shows comes from the record. The three language models answer the same briefing about 10% differently from one request to the next. That spread is the noise floor under every comparison below.

With these bots as opponents, the record made every model more aggressive. Its total weight on betting, raising and going all-in rose by 7 percentage points for luna, 12 for Haiku, 16 for Gemini and 28 for Jev. These bots are extreme, and the direction may differ against real players.

2. Which part do they read? Two contrasts

A contrast asks the same decisions twice. The two records it uses differ in one thing only. Neither record is written by hand. A scripted player with a fixed style plays 300 hands, and its play is counted the same way a person's is. Its opponent in those hands is a simple house player that bets every size equally often, so the record contains the style's answer to every size. We asked 40 decisions per model under each of the two records.

Bet sizes

The two opponents. One folds to big bets (large, overbet, all-in) and calls small and medium ones. The other does the opposite. Neither ever bets.
The textbook adjustment. Against the player who folds to big bets, bluff big so they fold, and bet small with a strong hand so they call. Against the other player, reverse both.
The spot. Hero bets first after the flop.
The measure. Of Hero's total betting probability, the share on big sizes.

Model	vs folds to big bets	vs folds to small bets	Difference (percentage points)
Jev	28%	82%	−54 [−58, −49]
Gemini	35%	68%	−33 [−47, −19]
luna	58%	42%	+16 [+8, +24]
Haiku	52%	51%	+1 [−5, +6]

The Gemini row covers the 21 of its 40 decisions where it gave betting any weight. How often a model bets at all barely changed for any of them; only the size moved.

Jev and Gemini bet the size the opponent calls, whatever they held. That is the textbook adjustment for a strong hand. With air it is backwards: Jev bet big only 23% of the time against the player who folds to big bets, and 81% against the one who folds to small bets.
luna bet the size the opponent folds to, whatever it held. That is right for a bluff and backwards for a strong hand.
Haiku did not change its sizes.

None of the four changed its size with its own hand. Each applied one rule to every hand.

Bluffs

The two opponents. When checked to, one bets a middle size with any hand. The other bets only with a pair or better. Both call every bet.
The textbook adjustment. Against the first player, whose bets are often air, call more often.
The spot. Hero faces a bet after the flop.
The measure. How often Hero continues (calls or raises) instead of folding.

Model	vs bets any hand	vs bets only a pair or better	Difference (percentage points)
Jev	80%	79%	+2 [0, +3]
Gemini	85%	84%	+1 [−8, +10]
luna	53%	54%	−2 [−11, +8]
Haiku	93%	75%	+18 [+9, +27]

Only Haiku adjusted. With air, it continued 91% of the time against the player who bets any hand and 69% against the one who bets only made hands.

The briefing states this difference as a counted line. Among the hands shown down after the opponent's large bets, about four in ten held less than a pair for the player who bets any hand, and none did for the other. Jev, Gemini and luna had that line and did not act on it.

3. Over the hands: as the record fills

This section counts actions actually played, block by block, through each 200-hand run.

Against shove: all-ins called, out of all-ins faced, per 40 hands.

Hands	luna	Gemini
1–40	12 of 30	13 of 22
41–80	22 of 25	22 of 25
81–120	17 of 25	18 of 22
121–160	27 of 30	16 of 19
161–200	25 of 26	20 of 22

luna's calls rose from 40% [25–58] to 96% [81–99]. Jev and Haiku called every all-in from the first block on.

Against switch: hands opened all-in, out of hands opened, per 50-hand phase.

Phase	Gemini	Jev
1–50 (station)	1 of 27	22 of 28
51–100 (raise)	12 of 25	27 of 28
101–150 (station)	4 of 26	28 of 28
151–200 (raise)	11 of 24	30 of 31

Gemini's all-in opens rose, fell and rose again with its opponent: 4%, 48%, 15%, 46%. That matches how it played the stand-alone bots, 11 of 104 against station and 63 of 102 against raise. Jev opened all-in in every phase. luna and Haiku opened all-in under 5% of the time in every phase; whether they answered the switch in another way was not measured.

4. Using the record is not using it well

One answer for nearly everyone. When first to act before the flop, Jev went all-in:

91 of 97 times against min;
101 of 111 against raise;
103 of 103 against shove;
99 of 103 against station;
107 of 115 against switch.

The exception was stab, the only bot that folds: 2 of 100. Against a player who calls everything, an all-in with a random hand wins about half the time, so it gains about nothing. Betting good hands and checking bad ones would win chips steadily. The record told Jev the opponent never folds; Jev answered with the one move that gets no value from knowing it.

Betting into a wall. Against station, the record showed every river bet called. In the river spots where Hero acted first holding air:

Gemini bet 16 of 83 times;
Haiku bet 15 of 84 times;
luna bet 0 of 92 times.

Jev reached that spot only 3 times. A bluff against a player who always calls only loses chips.

5. Speed and cost of one decision

Model	Response time (typical / slow case)	Input tokens	Cost per decision
Jev	0.19 s / 0.23 s	2,150	≈ $0.00009
Gemini	1.28 s / 1.52 s	2,909	≈ $0.0009
luna	2.02 s / 2.63 s	2,388	≈ $0.0003
Haiku	1.42 s / 1.95 s	3,619	≈ $0.0004

Response time. We sent 30 requests per model, one at a time, from Seoul. The games' own timings include a deliberate wait between requests, so we did not use them.
Input tokens. These are averages over the run.
Cost. We applied list prices per million tokens, input and output, as of October 2026: Jev $0.042 with free output, Gemini $0.25/$1.50, luna $0.10/$0.50, Haiku $0.10/$0.50. An answer's output is about 100 tokens; we measured this for Jev and assumed it for the others.
Decisions per hand. A hand took about 1.3 AI decisions for Jev, whose hands mostly ended before the flop, and 2.6–3.1 for the others.
Missing answers. The server played a default action 0 times in 1,594 decisions for Jev, 0 in 3,526 for Gemini, 3 in 3,722 for luna and 4 in 3,077 for Haiku.

Limits

One run. Each model met each bot once, with one deal of the cards. The bots are extreme on purpose. The results show how four models answer six simple players, not how they play people.
No win rates. Over 200 hands, luck swamps skill: a win rate's interval is about ±600 BB per 100 hands.
Different spots per model. Each model was re-asked on spots from its own games. Each contrast compares a model with itself; comparing levels across models is rough.
Self-reported probabilities. The probabilities are text the model writes. Nobody has checked whether "85%" means 85%.
The bluffs contrast changes two things. The player who bets any hand also bets more often.
Briefing layout. The briefing has one fixed layout. In an earlier test, layout changes alone moved single Jev decisions by 10–30 points.

Where this ran

The games, the briefings and the bots come from QUANT.POKER, where people play these models heads-up. The record in this study is the one a person's own play builds.

QUANT.POKER is mine: https://quant.poker. This write-up also lives there, with charts: https://quant.poker/observation/01

DEV Community: WhaleSurfer