DEV Community

khalid ali
khalid ali

Posted on

Training a reinforcement-learning agent to play Baloot (and shipping it to Android)

Baloot is a 32-card trick-taking game played in Saudi Arabia by four players in two teams: two contracts (Sun and Hokm), two bidding rounds, declarable melds called projects, doubling up to four times, and a scoring system (qaid) with rounding rules that catch even experienced players out. Building a bot that plays it well turned out to be a much bigger project than the card game itself. This is a write-up of the approach used in Arb3a Baloot (Unity client, Node.js server), which shipped on Android this week.

Why the rule-based bot failed

The first opponent was a hand-written heuristic: count trump strength, bid Hokm with jack and nine, lead high in Sun. It played legal Baloot but lost to humans, mostly because bidding depends on your partner's likely hand and the dealer position, not only on your own cards. Every hand-coded fix created a new exploitable habit.

State encoding

The encoder produces a 571-dimensional vector per decision:

  • the 32-card hand as a bitmap
  • trick history as ordered plays with seat indices
  • suits each opponent is known to be void in (inferred from failed follows)
  • confirmed projects
  • bidding context: who bid, what, in which round, and the doubling level
  • the running match score, because the right play at 140-60 is not the right play at 60-140

Training loop

PPO with self-play on a simulator that reuses the server's engine code, so the bot trains on exactly the rules it faces in production. Bidding and card play are two policy heads on a shared trunk. Rewards are shaped per round from the qaid delta rather than only at match end, which roughly halved wall-clock time to a competitive policy. A distillation stage compresses the best checkpoint into a small network that runs on-device for offline play.

Evaluation

A candidate is promoted only when it beats the current champion over a fixed set of 960 dealt games with seats rotated, so both sides see the same cards. A search-based player (lookahead over sampled hidden hands) beats the champion network by a few points but is too slow to run for every seat on the server, so it stays a benchmark.

Things that bit us

  • The model loader logged "model loaded" before verifying the file, so one deploy ran a random policy in production for days. It now prints the checksum and the evaluation win-rate at startup.
  • Turn timers lived only in memory, so a server restart during a doubled hand stranded the table. Timer flags are now persisted with the game state.

Shipping

The Android build is a Unity IL2CPP app bundle (arm64 + armeabi-v7a). The game is free with no ads.

Happy to answer questions about the encoder or the training setup.

Top comments (0)