Automated Cyber Defence by Deep Reinforcement Learning

Melanie Meijer


Supervised by Tingting Li; Moderated by Ian M Cooper

Defending cyber security is a significantly unfair game between defenders and attackers, as defenders need to be cautious all the time to detect and react to every single attack, whilst attackers only need to strike once at any time. Rapid development in Artificial Intelligence (AI) provides the potential for distributed, adaptive defensive measures at machine speed and scale. It is possible now the defender can be trained as an intelligent agent to develop strategies to effectively respond to an attacker across an entire network automatically.

The project will use the experimental platform based on the OpenAI Gym interface provided by CAGE challenge (https://github.com/cage-challenge/cage-challenge-1), together with a cyber security scenario to investigate the techniques that can be employed to train the defensive agents, e.g. Reinforcement Learning and Game Theory. The defensive agent can select a set of pre-defined high-level actions including the analysis of hosts, removal of malicious code, restoring of systems from backup, employing deception on hosts and the creation of decoy services. Each action has costs (in terms of effort and system downtime) that must be balanced against the need to protect systems with various criticality from an attack. Some red (attack) agents are also provided in the experimental platform to train and test against the defensive agents.

Reference: [1] TTCP CAGE Challenge 1: https://github.com/cage-challenge/cage-challenge-1 [2] Standen, M., Lucas, M., Bowman, D., Richer, T. J., Kim, J., & Marriott, D. (2021). Cyborg: A gym for the development of autonomous cyber agents. arXiv preprint arXiv:2108.09118. [3] Foley, M., Hicks, C., Highnam, K., & Mavroudis, V. (2022, May). Autonomous Network Defence using Reinforcement Learning. In Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security (pp. 1252-1254).

Initial Plan (03/02/2023) [Zip Archive]

Final Report (11/05/2023) [Zip Archive]

Publication Form