I made a 'benchmark' where LLMs write code controlling units in a 1v1 RTS game.

#ai #javascript #gamedev

Link to the results and additional details: https://yare.io/ai-arena

The game is fairly simple. 9 vs. 9 units battling each other on a basic map. The only actions the units can do are move() and pew(). All of the complexity emerges from having to reason about where to move, and whom to pew.

Testing method

Every LLM first creates their 'baseline' bot by playing 10 rounds against a human-coded bot of decent strength. A round consists of:

write code based on the game's documentation
play a game (models are allowed to add console.log() for whatever they think is important to track
get a review of the finished game (ASCII representation of the game state at key moments + the logs they themselves coded in.

Once their baseline bot is ready, they play a 10-games round-robin tournament with each other with the same iterative loop (improving their bot every game).