Phishing attacks are increasingly sophisticated due to the widespread availability of generative artificial intelligence (AI), which enables attackers to produce highly personalised phishing emails that are more difficult to detect using traditional rule-based and stylistic detection methods. This creates significant challenges for automated detection systems, particularly in maintaining robustness across shifts in time periods and domains. Prior work has explored both the use of stylometric feature sets to classify AI-generated emails and the analysis of stylistic differences between AI-generated and human-written phishing, often within small-scale and highly controlled experimental settings.

This research project addresses these issues by evaluating a layered machine learning (ML) classification system using stylistic-focused and URL-focused feature sets to assess temporal robustness and cross-domain generalisability. Feature analysis suggests that several stylometric indicators (including readability metrics and word-level features) remain significant for older AI-generated models but lose statistical significance when applied to newer models, indicating temporal drift in feature effectiveness. Multiple supervised learning models, including Random Forest and Gradient Boosting, were evaluated, showing strong in-distribution performance (93-94% accuracy), but feature importance analysis revealed inconsistencies and signs of overfitting.

Cross-dataset evaluation demonstrates a significant performance drop under domain shift, with accuracy ranging from 52-69% depending on the dataset pairing, indicating limited generalisability of static feature sets. A multi-layered classification approach incorporating a URL-based secondary layer improves performance by recovering 60-63% of false negatives, increasing overall accuracy to values as high as 76% in some configurations. However, results remain highly dependent on dataset composition, particularly the presence of URL structures.

Overall, the findings demonstrate that stylometric and static phishing feature spaces are insufficiently robust under temporal and domain shift conditions. While multi-layered classification provides partial mitigation, effective real-world deployment may require complementary human-in-the-loop decision support to address remaining misclassifications and model uncertainty.