Whenever someone receives a survey, there are normally two types of questions: (1) ones where the respondant selects an option or provides a score; and (2) ones where the respondant can write anything they'd like, such us when providing comments or suggestions. This project aims to develop tools for obtaining useful information from the latter type of responses.
FreeTxt (https://freetxt.app/ is a tool that "supports the analysis and visualisation of free-text data in both English and Welsh (e.g. data from surveys, questionnaires, feedback forms)" [1]. It includes functionalities such as sentiment analysis, word clouds and basic summarisation. In this project, we aim to implement apect-based sentiment analysis and summarisation.
Aspect-based sentiment analysis is the task of extracting key words in a sentence (the aspects) and performing sentiment analysis on those words, in the context of the sentence. Summarisation, in this context, is the act of generating summaries using large language models (LLMs) given the context of the aspects and their sentiments.
Implementing this system requires integration with the FreeTxt platform, using the existing FreeTxt application programming interface (API) and user interface itself to allow this system to be interacted with and display its data.
This project also aims for the system to be modular, with little new infrastructure required for changing artificial intelligence (AI) models or adding functions - providing the FreeTxt developers with a simple way of altering the system if needed.
The system developed in this project implements multiple AI models for the purposes of: - Aspect Extraction - Sentiment Analysis - Aspect Categorisation - Summarisation
The system developed meets the aims and provides a modular workflow for aspect-based sentiment analysis and summarisation, which can be started and viewed from the FreeTxt platform. Each model used has been tested based on runtime duration and accuracy in each of these tasks. The results find that LLMs struggle with aspect extraction and categorisation tasks, but succeed at generating summaries. The inaccuracies of LLMs lower the system's final output accuracy, however the system is fully functional and does not break the other tools and functions of the FreeTxt platform.