This post covers the design and development of an AI Non-Player Character that responds to sound signals generated by the player. The post covers gameplay AI, not LLM AI models like Chat GPT or others, so you can stop reading if that was what you were looking for in this post.
Tools of the Trade: AI State Tree
At the time of this writing there are two ways to develop gameplay AI in Unreal Engine: Behavior Trees and State Trees. Behavior Trees have been part of the Unreal Engine for a long time now, while State Trees are a relatively new addition to Unreal Engine, having been added in the last 3 - 4 years. The main difference between these two models is how they operate. As its name implies, a Behavior Tree uses a tree structure with a root, branches, and leaves which the AI traverses in order to determine what behavior to carry out. In contrast, a State Tree uses a state machine based on the State programming pattern, and the AI evaluates only the state it is in and conditions for transitions to other states while the game is running.
When determining which of these models to use when developing gameplay AI, the primary consideration was which of these two models would most likely be in use in the foreseeable future. Given that Behavior Trees have been a part of Unreal Engine for around ten years while State Trees could still be considered a new development, it seemed likely that learning and using the State Tree model would be the most advantageous to an aspiring professional game developer. Thus it was decided that the gameplay AI would be developed using the AI State Tree model.
Designing AI Character Behaviors
One part of the design for this game from the beginning was to include AI characters that respond to sounds generated by the player as part of the puzzle solving process. The three primary behaviors planned for these NPCs were following the player, moving to a specified position, and attacking a given target. Initially, the plan was to develop each command as a toggleable behavior, with the player able to stop an NPC's current behavior and return it to an idle state by re-entering the previous command. This was eventually changed to instead having a cancel command that could stop the NPC's current behavior. The reasons for this change were two-fold: First, it did not feel like good design to require a player to re-enter the last full command given in order to return an NPC to its idle state. Second, it was much more difficult to program the toggling of behavior in the AI's State Tree using the same command. The result of this decision was that the player would have four commands they could issue to NPCs: Follow, Move To, Attack, and Cancel. As NPCs do not react to the Activate command presently, that command will be excluded from this section.
The Wild West of the AI State Tree System
Before explaining how the AI State Tree was built, a major limitation needs to be addressed. Epic Games is notorious for having little to no documentation for much of Unreal Engine's systems and capabilities, and AI State Trees are no different. In addition, finding authoritative sources for how to work with the AI State Tree system was difficult given that the system has only been part of the engine for a few years. This meant that there were no standard, recommended, or best practices from which to start the development of the gameplay AI State Tree in this project. It was only through the kindness of strangers in the Unreal Source Discord server and the Unreal Engine website forums that this State Tree came to be, and it is still unknown at this time how 'correct' the configuration of states, tasks, evaluators, and blueprint logic in the State Tree are in comparison to the work a professional developer with more experience might produce.
Anatomy of an AI Non-Player Character
AN AI Non-Player Character (NPC) in Unreal Engine is made up of several different parts, each contributing to the the NPC's function.
The Character
The character is the representation of the AI NPC in the game. It is made up of the character mesh, colliders, and any other visual components that the player sees when interacting with NPC.
The AI Controller
The AI Controller is a component of the character. It controls the character's movements and actions and provides the character senses such as sight and hearing.
The AI State Tree
The AI State Tree is the state machine housing the logic that the AI Controller uses when making decisions about how to control the character. The State Tree is made up of multiple States, each of which may contain one or more Tasks.
States
States are the building blocks of the State Tree. Each state holds a collection of tasks making up a specific behavior. Examples include Idle, Attack, Pursue, Patrol, etc.
Tasks
Tasks are the actions carried out in a specific state. Each task has a blueprint associated with it that makes up the logic for that task. Examples of this could be moving to a location or attacking a target.
Evaluators
Evaluators are very similar to tasks in that they are actions carried out while the State Tree is running. The difference between tasks and evaluators is that tasks only run while their associated state is running, whereas evaluators run constantly whenever the State Tree is running, regardless of the current state. The primary purpose of evaluators is to get information that the State Tree can use in its various states and tasks.
Breaking Down the State Tree
The image below shows the State Tree used by AI NPCs in the game:
Figure 1: An image of the State Tree used by AI NPCs in the game
States
As shown in the right side of the image above, the State Tree has a Root state and four child states: Idle, Follow, Move To, and Attack. Underneath each state's name are the tasks associated with that state. For example, the Idle state has two tasks, which are setting the behavior state variable in the character blueprint and displaying a debug text on screen in the game.
Transitions
Just to the right of each state's name, there is a question mark (?) followed by an arrow pointing to other state names. The Move To and Attack states also have an arrow with no question mark pointing to Idle. These arrows indicate how the State Tree transitions from one state to another. Arrows with a question mark (?) indicate transitions that occur in response to a state tree event. State tree events are notifications to the state tree with a specific tag to facilitate transitions between states. The image below shows an example of how an event is sent to the State Tree from the character blueprint.
Figure 2: An example of a State Tree event being sent from the NPC character blueprint
The arrows with no question marks to the right of the state names are transitions that occur when a state has been marked as completed. This can occur a number of different ways, but in general a state is marked as completed once its component tasks have been completed. The image below shows all the transitions the Move To state uses to transition to other states in the State Tree.
Figure 3: An image of the transitions used by the Move To state in the State Tree
Schema, Context, and Evaluators
Going back to Figure 1, on the right side of the image in the Asset Details panel is where the schema, context, and evaluators are listed.
Schema and Context
The Schema and Context are two sides of the same coin in that together they supply the State Tree with the default classes that the State Tree uses and provides references to those classes to be used in tasks and evaluators.
Evaluators
As previously mentioned, evaluators can be thought of as tasks running continuously as long as the State Tree is running. They are listed on the right of Figure 1 along with the Schema and Context. At present, the State Tree only has one evaluator, which is used to supply the State Tree with a reference to the player and the distance between the NPC and the player.
Tasks
Each state has tasks that are carried out over the duration of the state. These tasks are all being carried out simultaneously for as long as the State Tree is in the state associated with those tasks. Since the State Tree is a state machine, there are specific events for entering and exiting a state, completing a state, and tick. Tasks can be set to run on each of these events. Much of the task logic runs on tick, because the intention of the design for the NPC was that it would carry out a given behavior until it completed the command given to it, and many tasks require the NPC to continuously respond to changing circumstances. For example, when an NPC is given a command to follow the player, it must continuously check where the player is in relation to itself and then move until it reaches an acceptable following distance. An interesting revelation discovered during development of the AI State Tree was that the State Tree does not have a built-in method for sharing its current state with other classes or actors. This necessitated the creation of a BehaviorState enumeration and a task to set the value of that enumeration on the character blueprint so that the character has the ability to perform different actions based on the current state of its AI State Tree. That task runs whenever the AI State Tree enters a new state. Other tasks used in the State Tree involve the movement or attack actions the NPC will take. The image below shows the details of the tasks associated with the Attack state as an example.
Figure 4: An image of the tasks associated with the Attack state of the AI State Tree
How Does it Work? The AI Character and State Tree in Action
Now that an explanation has been given for the workings of the State Tree and its component parts, an explanation can be given for how the AI NPC behaves in the game.
NPCs are added to the Puzzle Actors array along with reflective surfaces and other actors that are meant to respond to sounds generated by the player. They are also added to a second array called NPC actors for reasons that will be explained later.
When an NPC is hit by a line trace, it goes through several steps to process the received signal.
First, the NPC checks to see if the signal it received was meant for it. This is done by checking the subject portion of the received signal's transmission string against its own ID number. If there is no match, the NPC ignores the signal.
Assuming the signal is addressed to the NPC, the next step the it takes is to check the GUID of the received signal against an array of previously processed signals. This is based on the same logic the reflective surfaces use to limit extra line traces when generating reflected signals and is explained in detail in Devlog #3.
If the incoming signal's GUID does not match any previously received signal, the NPC next looks at the command portion of the transmission string to determine how it should respond. Follow and Cancel commands do not require any additional logic, but Move To and Attack commands require must validate the target substring of the transmission string at this point to confirm that the target is valid. This is also where the previously mentioned NPC Actors array comes into play. Since an NPC could be moving to or attacking another NPC or a destination, it is currently necessary to iterate through arrays for both of those actor types to confirm which actor should be used for the command and whether or not it is valid.
The final step for the character blueprint is to send an event to its State Tree with the appropriate tag for the command it received. There are four tags: Idle, Follow, MoveTo, and Attack. Each corresponds to its respective state.
Once the State Tree has received the state tree event, it transitions to the appropriate state and begins carrying out the tasks associated with that state. It will continue to carry out those tasks until another event is sent to trigger a transition to a new state or the state is marked as complete by the State Tree.
What Does the AI Character Actually Do?
This is where the specific behavior in each state will finally be explained.
Idle
Idle is the default state. In the Idle state, there is only one meaningful task to complete, which is setting the value of the BehaviorState enumeration in the character blueprint, which the State Tree does when it enters this state. While in the Idle state in game, the character takes no visible actions. It is only waiting for a new command.
Follow
In the Follow state, the AI character will attempt to follow the player, stopping when it reaches a distance equalling the acceptance radius of the Move to Target task. That acceptance radius is currently set to 500 units, meaning that the NPC will stop once it gets reasonably close to the player. It will continuously check the distance between it and the player and resume moving if the player moves such that the distance between the player and NPC exceeds the acceptance radius. It will continue to do this until given another command.
Move To
When given a Move To command, will move to the location of the target associated with the command. This could be another NPC or it could be a specific location in the level. When moving to a location, the destination options are limited to locations associated with an ID number. A destination actor has been created to facilitate this behavior. Once the NPC has reached its destination it will automatically transition to the Idle state. While moving, it can be given a new command to either cancel its move or undertake a different behavior.
Attack
After receiving an Attack command, the NPC will move to the location of the specified target and begin attacking it, dealing damage to it if the target can take damage, and eventually destroying it. While it is attacking, an NPC can be given another command to cancel the attack or change its behavior. Once the target is destroyed it automatically transitions to the Idle state.
Final Thoughts
Developing the gameplay AI has been the most difficult and frustrating part of this project so far. The lack of readily available support resources made progress slow and the process of constant research and asking around for guidance was exhausting. However, it is good to see the final product working in the game.
Top comments (1)
One small terminology correction: an AIController is a separate AController actor that possesses the Pawn/Character, not a component of the Character. An AIPerceptionComponent is often placed on that controller, but perception is not inherent to the controller itself. That distinction becomes important when debugging possession, respawns, and controller lifecycle. For the Follow state, it is also worth ensuring the running task cancels its outstanding move request on ExitState/Stop so a later Cancel event cannot leave movement active.