Reinforcement learning (RL) enables agents to learn by interacting with the environment. The agent collects experience from trail-and-error and optimises its action rules from the environment feedback. In recent years, deep reinforcement learning, such as DQN [1], has achieved superhuman performance in playing chess, Atari games, etc. However, interpreting the behaviour of a deep RL agent is challenging, due to both the 'black-box' nature with deep learning and the randomness and long-time learning process with reinforcement learning. Some attempts [1,2] have been made to develop visual analytics to help understand agents behaviours. According to these works, experts are interested to understand the agent's memory, statistics from the learning process, and the relations between the agent's observation and action. This project aims to explore various visualization techniques to help experts gain better understandings in one or more of these aspects in deep reinforcement learning.

Good programming skills (preferably java) and interest in reinforcement learning are required for this project. Suitable for multiple students working on visualizing different aspects.

[1] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al. "Human-level control through deep reinforcement learning". Nature, 518(7540):529, 2015. [2] J. Wang, L. Gou, H.-W. Shen, and H. Yang. "DQNViz: A Visual Analytics Approach to Understand DeepQ-Networks". IEEE TVCG. 25(1): 288-298, 2019 [3] T. Jaunet, R. Vuillemot, C. Wolf. "DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning". Computer Graphics Forum, 2020