Passive identification of IoT devices from network traffic is a valuable capability for asset visibility, segmentation, and security monitoring, but many machine-learning classifiers are evaluated only within a single dataset or collection environment. This project investigates whether flow-based IoT device-identification models remain reliable when tested under progressively more realistic forms of dataset and environment shift.

The project develops a leakage-aware NFStream pipeline to extract and prepare flow-level features from three public IoT traffic datasets: UNSW-DI, UNSW-AD, and YourThings. Evaluation is organised around a three-tier framework: within-dataset testing on UNSW-DI, cross-environment transfer from UNSW-DI to overlapping benign UNSW-AD devices, and cross-dataset transfer from UNSW-DI to YourThings at both device and category level.

Six classical classifiers and one 1D-CNN neural baseline are compared in the within-dataset setting, with the stronger classical models then tested under transfer. Additional temporal-decay, profile-sensitivity, permutation-importance, and feature-family ablation analyses are used to examine how performance changes and which features appear to support or undermine generalisation.

The final results show a clear generalisation gap: models that perform well within UNSW-DI degrade under UNSW-DI to UNSW-AD transfer and largely collapse under UNSW-DI to YourThings transfer. The project therefore provides an empirical account of the limits of flow-based supervised IoT device identification and highlights the need for transfer-aware evaluation and more environment-agnostic representations.