Interactive sign language learning with real-time hand sign recognition

Raedah Almarzooq


Supervised by Dr Daniel J. Finnegan; Moderated by Paul L Rosin

British sign language (BSL) is a non-verbal visual language that is considered the primary mode of communication for deaf individuals in the United Kingdom (Jabb,2021). There are several resources for learning sign language from printed materials to online learning websites. Despite the usefulness of these resources as they offer different activities for learners, they all have the same limitation which is the lack of feedback. Receiving feedback and comments is crucial for the advancement and progress of the signing language skill . To address this limitation, an interactive web application that integrates hand sign recognition technology has been developed. This project aims to enhance the learning experience for BSL learners by creating an interactive and engaging web application that recognizes the hand sign and return feedback to the users.The study aims to investigate the effectiveness of incorporating the hand sign recognition feature in web application in enhancing the learning experience for the learners. The development of the web application involved utilising HTML, CSS, and Javascript to create an interactive and responsive application. Additionally, the Python programming language is used to develop the hand sign recognition model. The evaluation of the system is done through user testing and thematic analysis. The data analysis demonstrates the positive impact of the hand sign recognition feature in enhancing the learning experience.

Initial Plan (04/02/2023) [Zip Archive]

Final Report (19/05/2023) [Zip Archive]

Publication Form