Intrusion Detection Using Disagreement-Based Semi-supervised Learning: Detection Enhancement and False Alarm Reduction

With the development of intrusion detection systems (IDSs), a number of machine learning approaches have been applied to intrusion detection. For a traditional supervised learning algorithm, training examples with ground-truth labels should be given in advance. However, in real applications, the number of labeled examples is limited whereas a lot of unlabeled data is widely available, because labeling data requires a large amount of human efforts and is thus very expensive. To mitigate this issue, several semi-supervised learning algorithms, which aim to label data automatically without human intervention, have been proposed to utilize unlabeled data in improving the performance of IDSs. In this paper, we attempt to apply disagreement-based semi-supervised learning algorithm to anomaly detection. Based on our previous work, we further apply this approach to constructing a false alarm filter and investigate its performance of alarm reduction in a network environment. The experimental results show that the disagreement-based scheme is very effective in detecting intrusions and reducing false alarms by automatically labeling unlabeled data, and that its performance can further be improved by co-working with active learning.