First Person Shooter Bots Via Reinforcement Learning

Gregor Webster


Supervised by Frank C Langbein; Moderated by Yulia Cherdantseva

This project's focus was the building of better FPS bots through the use of reinforcement learning, and specifically the area of sample biassing as a means of achieving that improvement. To find out if sample biassing was a useful improvement, data needed to be gathered on both unbiased and biassed programs. To that end, a simple reinforcement learning algorithm was adopted and adapted to generate the data needed. After six runs, three unbiased and three with varying levels of bias, sufficient data had been gathered to establish the basic effects of biassing. The goal of the project was to establish if reward biassing was a useful tool for FPS bots, either in terms of providing an overall improvement, or simply by improving some desirable aspect of the bot, even if it was detrimental to other areas. Thus, for the project to be a success the baissed data would have to show that biassing could provide at least one useful benefit that unbiased sources could not provide. After analysing the data, the biassing proved to have a mix of positive and negative effects on the resulting bot, notably it had adopted a preference for high-risk high- reward strategies which lead to the program having a greater number of high scores, but a lower average score. This could be adapted into a tool which would suit certain problems better than an unbiased one. Overall, the project was deemed a success, but more data is still required to help quantify the effects at differing levels of bias.

Initial Plan (07/02/2022) [Zip Archive]

Final Report (27/05/2022) [Zip Archive]

Publication Form