Recent years have seen rapid advances in the field of artificial intelligence (AI), mainly due to breakthroughs in machine learning (ML) using deep neural networks. These approaches learn powerful classification and prediction models from large volumes of data, but have a limitation that the resulting models are usually "black boxes" - it's hard to determine how they work, and even harder to generate explanations ("why?") for a user!

Many AI / ML applications involve information fusion: the generation of classifications/predictions based on multimodal data from different kinds of sensor; for example, recognising activities in a video with a soundtrack (camera and microphone sensors), or exercise tracking using a combination of heart rate and steps walked (calculated via accelerometer or GPS data). Again, when deep neural networks are used for these applications, explanations are hard to generate, especially as these must involve the multiple data modalities in combination.

The aim of this project is to examine this problem of multimodal explanations in information fusion applications, ideally using (1) off-the-shelf deep neural network models trained on existing datasets and (2) application domains of interest to the Crime and Security Research Institute, such as CCTV video and social media data.

The project doesn’t require specific background knowledge but you’ll need to have a keen interest in AI and machine learning and to be willing to take on a research-focussed rather than software development project. The project will be supported by PhD students in the Crime & Security Research Institute DAIS group: https://www.csri-dais.org