ML-Based Intrusion Detection with Feature Analysis on Unbalanced UNSW-NB15 Dataset

Intrusion detection in modern networks, encompassing the Internet of Things (IoT), software-defined networking (SDN), and cloud environments, represents a pressing research challenge for network security researchers and practitioners. Our research paper focused on utilizing the UNSW-NB15 intrusion dataset and applied a diverse set of machine learning(ML) models to evaluate their performance in this context. However, the dataset presented unique challenges, being highly imbalanced and featuring nine distinct types of attacks. Consequently, many conventional ML models struggled to accurately identify these attack types with high precision. To address this challenge, we have introduced a novel probabilistic-based method to select class-specific instances and conducted feature analysis to pinpoint the most informative attributes for training ML models. The objective was to equip these models with the capability to provide high-precision detection. The outcome of this endeavour was highly promising: our proposed instance selection method consistently delivered accuracy rates exceeding 99% and 98% across a range of tested ML models, supporting both binary and multi-class classification tasks, respectively. These findings underscore the potential of our approach in enhancing the accuracy and effectiveness of intrusion detection in modern network environments, offering a valuable contribution to the field of network security research.