[PDF]

Online detection, grouping and tracking of human subjects for robotics application


Harry Wyatt

14/05/2026

Supervised by Victor Romero Cano; Moderated by Hantao Liu

Future social robots must exhibit social awareness to safely and successfully coexist in human-shared spaces, such as being able to act based on human presence and actions. In order to do this, they must be able to perceive humans and their social structures. This dissertation proposes and implements a two-tiered pipeline for human perception and tracking, and also for group detection. The solution utilises SAM 3 for human detection, ByteTrack for tracking and an LMM to perform socially-aware determination of human groups. Evaluations in both simulated and real scenarios showed that human detection was robust to partial occlusions and group detection was generally successful at detecting basic groups. However, the MOT was subject to ID swaps when targets had overlapping trajectories. Through careful and deliberate optimisations, the human detection and tracking pipeline can successfully run with a median latency of 13ms on an NVIDIA RTX 4090, with the group detection running at a lower frequency and executing with a median latency of 318.5ms. Moreover, this pipeline demonstrates promising results on edge devices such as the NVIDIA Jetson AGX Orin Developer Kit, achieving a median latency of 120ms with the group detection pipeline disabled.


Initial Plan (02/02/2026) [Zip Archive]

Final Report (14/05/2026) [Zip Archive]

Publication Form