Game playing is one way to learn machine learning strategies. A most game playing bots involve some searching mechanism. It’s how the bot can “see” which move can result in a favorable outcome down the line.
Lets learn about minimax, a useful technique to build an AI to compete on simple games.
Single Player Tree Searching
Lets play a little game. The object of the game is to end up with the highest number.
For a single turn, one player game the choice is simple.
You go right, end of with a score of 10! That’s the highest possible score in our little game.
Lets make it a little more complicated. Our game now has two turns, what do you choose?
On your first turn, if you make the same decision (pick 10), that will leave you with bad options on your second turn (2 or 3).
To achieve the highest score, you need to pick 1 on your first turn in order to be able to pick 20 on your second turn.
One way to model this, is to start at the bottom and choose the max at each node. If we do that across our entire tree, we determine our max score (20) and which decisions to make (left, left).
Two Player Tree Searching (Minimax)
What if we introduced another player into this game? Now two players are competing to against each other. With a small tweak in the rules.
- Player 1 wins if a positive number is chosen.
- Player 2 wins if a negative number is chosen.
- A tie occurs if 0 is chosen.
- Players alternate turns with Player 1 going first.
Lets take the following game.
Player 1 (P1) makes the first choice, then Player 2 (P2) has a turn, and finally Player 1 makes the final pick.
Player 1 wants positive numbers, so they’re looking to maximize their score.
Player 2 wants negative numbers, so they’re looking to minimize their score.
That’s where the term minimax comes from, from players wanting to either minimize or maximize their score.
How could this game play out? If each player choose the best available option to them at the time, the optimal game outcome is 7, which means Player 1 wins!
Lets model this game as well.
Again, Player 1 attempts to maximize their score, Player 2 attempts to minimize their score.
Player 1 goes left, Player 2 goes left, and finally Player 1 goes right landing on a score of 7.
Programming Minimax
Lets implement a minimax search in python!
We first need a data structure to hold our values.
| class Node(): | |
| def __init__(self, value): | |
| self.value = value | |
| self.left = None | |
| self.right = None |
We create a Node class, it can hold a single value and links to a left and right Node.
Then we’ll create a Choice class that represents the players choice.
| class Choice(): | |
| def __init__(self, move, value): | |
| self.move = move | |
| self.value = value | |
| def __str__(self): | |
| return self.move + ": " + str(self.value) |
Next we’ll initialize a tree with the values from the two player game we modeled above.
| # 4th layer | |
| leaf1 = Node(-8) | |
| leaf2 = Node(7) | |
| leaf3 = Node(15) | |
| leaf4 = Node(14) | |
| leaf5 = Node(8) | |
| leaf6 = Node(-5) | |
| leaf7 = Node(1) | |
| leaf8 = Node(0) | |
| # 3rd layer | |
| body3_1 = Node(3) | |
| body3_1.left = leaf1 | |
| body3_1.right = leaf2 | |
| body3_2 = Node(20) | |
| body3_2.left = leaf3 | |
| body3_2.right = leaf4 | |
| body3_3 = Node(-8) | |
| body3_3.left = leaf5 | |
| body3_3.right = leaf6 | |
| body3_4 = Node(-1) | |
| body3_4.left = leaf7 | |
| body3_4.right = leaf8 | |
| # 2nd layer | |
| body2_1 = Node(-2) | |
| body2_1.left = body3_1 | |
| body2_1.right = body3_2 | |
| body2_2 = Node(10) | |
| body2_2.left = body3_3 | |
| body2_2.right = body3_4 | |
| # root | |
| root = Node(None) | |
| root.left = body2_1 | |
| root.right = body2_2 | |
| print_tree(root, 4) |
You’ll notice on line 41, I’m printing the tree. Which outputs the following:
The
print_tree function is available on the github repo.
We’ll solve this by coding a recursive depth first search algorithm. The main concerns we want to keep in mind are:
- We need to keep track of which players turn it is, I’m using the is_max to track that.
- We need to know when to stop our search, in recursion that means we need some base cases.
- We want to explore our whole tree. We need some comparison logic to determine which sub tree we will choose to explore.
Our base case will be when our node no longer has any children, we just return the value of the node.
| def minimax(node, is_max): | |
| # base case, if no sub nodes, just return the value | |
| if (node.left is None and node.right is None): | |
| return Choice("end", node.value) |
In order to explore our whole tree, we’ll call minimax on each sub tree in our node.
| def minimax(node, is_max): | |
| # base case, if no sub nodes, return the value | |
| if (node.left is None and node.right is None): | |
| return Choice("end", node.value) | |
| # if child nodes exist, run minimax on each child nodes | |
| l_choice = minimax(node.left, not is_max) | |
| r_choice = minimax(node.right, not is_max) |
Making sure to reverse is_max because the players alternate turns.
Finally, we compare the results. If we’re maximizing we take the max results, then if we’re minimizing we take the minimum results.
| def minimax(node, is_max): | |
| # base case, if no sub nodes, return the value | |
| if (node.left is None and node.right is None): | |
| return Choice("end", node.value) | |
| # if child nodes exist, run minimax on each child nodes | |
| l_choice = minimax(node.left, not is_max) | |
| r_choice = minimax(node.right, not is_max) | |
| # compare results | |
| if (is_max): | |
| if (l_choice.value > r_choice.value): | |
| return Choice("left", l_choice.value) | |
| else: | |
| return Choice("right", r_choice.value) | |
| else: | |
| if (l_choice.value < r_choice.value): | |
| return Choice("left", l_choice.value) | |
| else: | |
| return Choice("right", r_choice.value) |
Lets create a little simulator to play our game for us.
| # initialize our state | |
| current_node = root | |
| is_max = True | |
| while (True): | |
| # run minimax on current node | |
| choice = minimax(current_node, is_max) | |
| # make choice based on minimax search | |
| if (choice.move == "left"): | |
| print ("Moving left to node with value " + str(current_node.left.value)) | |
| current_node = current_node.left | |
| elif (choice.move == "right"): | |
| print ("Moving right to node with value " + str(current_node.right.value)) | |
| current_node = current_node.right | |
| elif (choice.move == "end"): | |
| print ("Game ends with a score of " + str(choice.value)) | |
| break | |
| # flip players turn | |
| is_max = not is_max |
Running this outputs the following:
- Moving left to node with value -2
- Moving left to node with value 3
- Moving right to node with value 7
- Game ends with a score of 7
Congrats! We’ve correctly demonstrated how to use minimax to optimally solve a simple two player game.
Parting Thoughts
This game is about as simple as it gets. But searching is an integral part to most automated game playing bots. It’s how bots can “see” into the future.
For a challenge, see if you can use minimax to build a tic tac toe playing bot.
As you can imagine, with complicated games (Chess and Go) searching has its limitations due time and memory constraints. Further strategies such as alpha-beta prune and Monte Carlo simulation can help in those cases.
You might have noticed that my base case will only work with nodes that have two or 0 child nodes. The minimax in the git repo will contain code to handle that condition.
Originally published at thesharperdev.com.
Top comments (0)