Creating Othello AI Players Combining Heuristic Evaluation And Reinforcement Learning

Lida Wen


Supervised by Bailin Deng; Moderated by Hantao Liu

This project aims to explore possible ways of creating AI players that are capable to make sensible decisions in the board game Othello. The complexity and interactivity make this topic particularly interesting. To achieve the aim, the project starts with analysing the background, rulesets and advanced strategies used in the game. A GUI has then been designed with Pygame to visualise the game.

Four types of intelligent agents are created, which are Heuristics, Minimax, MCTS and DQN. Another three test agents have been implemented as benchmarks for the experiments. Performance is evaluated in terms of win rates against test agents, computational efficacy, and variables of training opponents. In conclusion, the adversarial heuristics models had superior results in the experiments because of the particularity of certain strategies in Othello, and the performance is sensitively affected by the choice of heuristics. RL agent would perform better if I had a higher quality training model and hardware. The purpose of this project is not to create the strongest Othello AI but to demonstrate the implementation and comparison of multiple approaches in order to contribute to the further improvement of the effectiveness of Othello AI player development.

Initial Plan (07/02/2022) [Zip Archive]

Final Report (27/05/2022) [Zip Archive]

Publication Form