All papers / No. 07 / AI Security Evaluation
Machine-learning phishing detectors routinely report accuracy between 95 and 99 percent, a range that suggests phishing detection is close to solved. Shortcut learning research shows why that reading is unsafe: a model can reach a high score by relying on correlations that happen to hold in training data instead of learning the detection task. The phishing literature leaves two questions unanswered. How much of reported accuracy reflects genuine phishing signal, and how much rests on shortcuts that an attacker can remove at little cost? Four standard classifiers, logistic regression, random forest, LightGBM, and histogram gradient boosting, were trained on the UCI Phishing Websites benchmark of 11,055 sites and 30 features, and shortcut reliance was measured through feature-family ablation, a shortcut flip test, and a mitigation retraining experiment.
© 2026 Prabin Shrestha. Licensed under CC BY-ND 4.0: share unchanged with attribution; no derivatives. The archival PDF is the Zenodo deposit linked above.
Keep reading