Hide & Seek
Created using UE4 v4.9.2 and Blueprint
Overview
The purpose of this project was to expand my working knowledge of artificial intelligence. I have established how to develop an AI that uses both rule based and state based thinking, and this time I wanted to take it a little further. This AI uses what I consider to be reinforced learning to Hide from the player, and a Heat Map to Seek the player. This is the first time that I’ve used either of these concepts.
The Heat Map
The Heat Map was definitely an interesting problem in this project. Since I decided to use UE4, I didn’t really have access to the exact dimensions of the map. This made things complicated initially. I created a simple node that would be dispersed throughout the map at the beginning of the application. It checked for collision with the world and wouldn’t spawn if it was colliding. This kept the AI from trying to check nodes that he couldn’t reach.
The nodes started with the typical Heat Map functionality. Each node started at 0 plus or minus a very small variance to make things interesting. Each tick, the node would add to its current Heat and, once it reached 1, it would start to add to a timer. This timer tells the AI how long the node has been at maximum heat (which is 1). This timer is how the AI decides where he is going to go next. Once he chooses a node, he goes to that node and stops there for about 1.5 seconds. This is a little bit of an aesthetic choice to make him look like he’s actually searching for the player. Once the AI has reached its current node choice, then it will choose the next node with the highest time at maximum heat. However, any nodes that are currently visible to the AI will be set back to 0 heat. This keeps him from running to nodes that he should be able to see and breaking the believability of his seeking. |
Click an Image to zoom in
|
It was when I started developing the Hiding algorithm for the AI that I realized I could also use the Heat Map nodes to help him choose a position to hide. I decided to add a different functionality to the nodes specifically for this situation. If the AI doesn’t have any knowledge on what makes a good hiding place, then he will choose a spot from one of the two types of Nodes: open or closed. A Node determines if it is open or closed based on how many of the surrounding Nodes can see it. If less than 10% of the total number of Nodes in the map can see a Node, then it is considered to be a closed Node, or a Node in a closed off area; otherwise, it would be considered open, or in an open area.
The A.I.
The AI is able to be either the hider or the seeker. Depending on which of the roles the AI has, he interacts with the Heat Map in different ways. When he is seeking, the Heat Map performs like your typical Heat Map. However, when he is hiding, he uses the Heat Map to find a spot to hide. The AI determines his hiding position based on the information he has from the player’s previous hiding spots, as well as his own. If he doesn’t have any previous hiding spot knowledge then he will randomly choose one of the two types of hiding spots: open or closed. Once found, he will choose the type of spot that he didn’t choose before. This will help him get a general idea of how good each spot is.
So, what is it that determines a good spot? I decided that what makes a good spot is the time that it takes for the seeker to find the hider. In the case of the AI being the seeker, every time he begins seeking (whenever the player indicates that they’ve found a hiding spot and are ready) he sets a variable called seeking start time. As soon as the AI finds the player, he sets a new variable on one of the Nodes near the player that holds how long it took him to find the player (which is found by subtracting the seeking start time from the total game time). This Node is then added to an array of previous player hiding spots for later use. Once the AI has some hiding spots under his belt, then he will take the average of each type of Node (open and closed) and compare the two. If the total average of all of the closed hiding spots he’s hidden in is greater than the average time of all of the open hiding spots he’s hidden in then he will choose a closed Node over an Open Node and vice versa. If the AI has some of the player’s previous hiding spots in his array, then the hiding times for each of those hiding spots will be added into the averages for their respective Node types. This allows the AI to learn how to hide from the player based on where the player was hiding and how long it took the AI to find them. |
Click an Image to zoom in
|
|
|