Handling Novel Mobile Malware Attacks with Optimised Machine Learning Based Detection and Classification Models

Malicious behaviour analysis is one of the biggest and most prevalent challenges in cybersecurity. With the dominance of the Android ecosystem, a significant number of frameworks were proposed to address the huge number of malicious attacks targeting the consumer base of this platform. Although still developing, the application of machine learning techniques for malware detection has recently experienced a growing interest due to their potential to achieve better results compared to traditional techniques. However, the effectiveness of detection by learning varies according to the used features and models. Moreover, its application to mobile malware detection is even more challenging given the deployment constraints. In this paper, mobile malware detection is cast as a classification problem, and four main and relevant questions are considered: (1) Which set of features is more relevant for effective detection using ML models. (2) which models are best performing for this type of tasks (3) which solution can be the most lightweight and most effective for real-time detection (4) And finally, how can these models be optimized to address the risk of a zero-day attack. This paper describes a comprehensive investigation of the potential of traditional and advanced ML models to address the aforementioned issues. As a result of this in-depth study, a testbed has been prepared using 168 different models and three recent datasets. Furthermore, the main contribution of this work lies in the development of novel models that outperformed the state-of-the-art proposed approaches. One of which, combined early integration with Extra Trees Classifier which achieved a detection rate of 99.94% and an AUC score of 99.91%. Further experimentations were conducted on the deployability aspect of these models, where results have shown that Boosted algorithms offered the best balance of detection rates and resource utilisation for a lightweight and robust malware detection solution. Furthermore, this comprehensive analysis helped in one hand, gain more insight into the role of features in the learning task, which led to the identification of a set of characteristics that we believe should be considered to develop an effective dataset to counter novel malware attacks. On the other hand, it helped in highlighting future developments and the missing components in this field, where we ultimately proposed a framework that builds on this analysis to provide a better approach for future studies.