Performance comparison of intrusion detection systems and application of machine learning to Snort system
Introduction
Today many businesses rely on computer networks. These networks fulfil the needs of business, enterprises and government agencies to build knowledgeable, complicated information networks which integrate various technologies such as distributed data storage systems, encryption techniques, voice over IP (VoIP), remote or wireless access and web services. These computer networks have become more important as business partners access the information through extranets, customers communicate using networks through e-commerce transactions or Customer Relation Management (CRM) systems and employees connect with enterprise networks through virtual private networks (VPNs). These well-travelled paths make computer networks more vulnerable than ever before because today’s attackers are well organised as they have time, expertise and resources to launch the attacks that can avoid detection by even secure networks.
The attackers act like normal users, generate data and hide their malicious activities under terabytes of data. They know that many security mechanisms cannot protect the networks due to the large amount of data stored, scalability issues or due to the lack of detection capabilities. The enterprises and government agencies need to monitor their network traffic to detect malicious activities and perform analysis to differentiate the malicious and legitimate user activities to protect their networks. Detecting malicious activities require intrusion detection systems (IDS) and in today’s secure ICT infrastructure, the IDSs are part of most networks. However, the IDSs are only good if they have elite detection capabilities. It is critical that an IDS detection mechanism is accurate enough to differentiate between legitimate and malicious traffic that enter and leave the network. The possible results of using an IDS are as follows: detected malicious traffic (real alarms), undetected malicious traffic, legitimate traffic that IDS detect as malicious (false alarms) and legitimate traffic that IDS detect as good.
The elite IDSs detect as much malicious traffic as possible and reduce the number of false alarms. There are a number of commercial IDSs available in the market such as Juniper, McAfee, Cisco, Symantec etc. [1]. The commercial IDS generally do not provide the ideal performance as advertised and could compromise computer network security. Like the commercial IDSs, there are a number of open source IDSs available such as Snort, Suricata and Bro.
Snort and Suricata [2] were chosen for our study as we felt they have comparable functions, detection rule sets and syntax. They are both under GNU GPL licence. They both support intrusion prevention system (IPS) feature and support medium to high-speed network, though Suricata is more scalable with its multi-threaded architecture. Both support IPv6 traffic and their installation and deployment are easy. In contrast, Bro is a flexible script based IDS and its policy scripts or rules are written in its own Bro scripting language that does not rely on traditional signature detection [2]. It is under BSD licence and does not support IPv6 traffic. Installation of Bro can be difficult. Unlike Snort or Suricata, Bro does not offer inline intrusion prevention features. Both Snort and Suricata have similar features such as a module to capture the network packets, a module to decode and classify the network packets and a module to detect accurately the malicious or legitimate packets based on a rule set defined by both IDSs. Snort and Suricata inspect network packets for possible malicious traffic through the rule set and trigger alarms when the packet payload matches with one of the rules [3].
The Snort IDS has been in development since 1998 by Sourcefire and has become the de-facto standard for IDSs over the last decade. It is extensively deployed in networks and researched into. Snort has a single threaded architecture as shown in Fig. 1 which uses the TCP/IP stack to capture and inspect network packets payload [4]. Snort has added a multi-instance feature to its 2.9 release to address the limitation of single-thread and has hinted that version 3.0 will be multithreaded by default.
The Suricata IDS was developed in 2010 by the Open Information Security Foundation (OISF). Suricata is publicised as a future next generation IDS integrating new ideas such as multithreading as shown in Fig. 2. Based on the previous research it has improved on Snort because it uses multi-thread architecture to quickly capture and decode network packets [5].
Snort or Suricata can be installed on a computer host. The performance of both IDSs is directly proportional to the computer host performance in terms of CPU, memory utilisation and the packets drop rate by the network interface card.
The continued increase in network speed and malicious traffic causes significant problems for Snort and Suricata. Both IDSs have to process higher traffic volumes and inspect each incoming network packet to detect malicious traffic. In order for Snort and Suricata to have superior performance, both IDSs must process a vast volume of network traffic that can reach speeds up to 10 Gbps. If both IDSs fail to execute packet inspection at the required rate, they will drop the packets and allow malicious packets to enter the computer network undetected. Therefore, Snort and Suricata should be efficient enough to process network traffic up to 10 Gbps network speed.
Recently, numerous researchers have studied the support of machine learning algorithms for IDSs. Machine learning is a field of computer science that trains the computer to think like humans and take actions where required. In simple processing, a computer processes the information based on statements from primary memory. Machine learning techniques try to copy thinking processes such as logical reasoning, intuition, learning from the past, trial and error and generalisations [6].
Snort and Suricata use rules to detect the known malicious traffic. If malicious traffic matches with the rule set, then they will trigger the alarms. But both IDSs will not take any action against unknown malicious traffic. This is because Snort and Suricata do not use the machine learning techniques and hence cannot stop unknown malicious traffic. Machine learning techniques can help IDSs by accurately detecting the malicious traffic and therefore reduce the false positive alarms by proactively reacting against unknown malicious traffic. There are various machine learning algorithms that can be used for IDSs like Support Vector Machines, Decision Trees, Fuzzy Logic, BayesNet and NaiveBayes.
Snort and Suricata use pre-defined rules to detect malicious network traffic. If malicious traffic patterns match with the rule set then both IDSs trigger alarms, and these can be false positive, false negative or true positive alarms. Snort and Suricata alongside all the other IDSs have a common problem which is triggering false positive alarms [7]. For example, legitimate network traffic consists of DNS or web requests can lead the IDS to trigger a false positive alarm. This is because both IDSs have an identical rule set to match patterns for DNS or Web attacks. Therefore, false positive alarms affect the performance of Snort and Suricata and utilise a high computing resource for classifying the network traffic.
The aim of this paper is to do a performance comparison of Snort and Suricata and to implement machine learning algorithms on it to improve the detection accuracy. The aim led to the following objectives: (1) To critically review Snort and Suricata by measuring the performance and detection accuracy of both IDSs. (2) To evaluate the machine learning algorithms using three different datasets and to improve the selected IDS performance by implementing collective and optimised machine learning techniques through reducing the false positive alarms. The main contribution of this work is the performance comparison of Snort and Suricata at 10 Gbps and the application of the hybrid and optimised machine learning (ML) algorithms to Snort.
Section snippets
Related works on performance comparison
The idea of a performance comparison between Snort and Suricata is not new. Both perform well, but are not perfect and have limitations as shown in our experiments. Snort has a single-threaded architecture, and Suricata has a multi-threaded architecture which makes both IDS distinct from each other, but the rule set is the common feature of both IDS. Classifying the network traffic and accuracy of the rule set are the key elements of both IDS’s performance. Furthermore, computer host
Description of research methodology
Experiment scenarios were designed to make observations and to take measurements. This study demonstrated rigorous, repeatable, quantitative performance comparisons of both IDSs and evaluated the machine learning algorithms.
The experiments consisted of a test bed which compared Snort and Suricata’s detection accuracy in 10 Gbps network speed and with seven different types of malicious traffic. The seven types of malicious traffic were chosen because the rules could be applied consistently to
Experiment scenarios
The experiment scenarios were planned and set up to compare the performance of Snort and Suricata on identical VMs using identical rule set and under the same test conditions.
Snort IDS using machine learning techniques
We will consider Snort to implement machine learning [32]. Snort is dependent on the rule set to detect malicious traffic that gives an accurate description of the known malicious traffic. To reduce FPR, many researchers have used machine learning algorithms to classify normal and malicious traffic. The following machine learning algorithms have been extensively studied like Support Vector Machine, Decision Trees, Fuzzy Logic, BayesNet and NaiveBayes.
Also, various other machine learning
Evaluation of machine learning algorithms (MLAs)
Before using machine learning techniques in conjunction with Snort, it is important to know about high performing machine learning algorithms. Three publicly available datasets were used to conduct performance experiments on MLAs. Initially, the experiment evaluation environment was built which consisted of evaluation setup and data preprocessing using data mining software [33]. Secondly, a literature search was conducted, and a comprehensive set of five high-performing machine learning
Proposed Snort adaptive plug-in
As shown in the Table 8, Table 9, Snort has successfully detected six types of malicious traffic out of seven, but it triggered high false positive alarms (55.2% average). The proposed Snort adaptive plug-in architecture for Snort is to reduce the false positive alarms using machine learning algorithms. The proposed new architecture of Snort IDS is shown in Fig. 6.
The proposed Snort adaptive plug-in operates in parallel with the Snort’s rule set. The rationale for integrating the Snort adaptive
Conclusion
This study compared the performance of two open source IDSs, namely Snort and Suricata. They were both proved to be efficient and high performing IDS, though each had its own strengths and weaknesses. The analysis of the experiment results showed that Snort utilised less computational resources to process 10 Gbps network traffic whereas Suricata’s utilisation was higher. Suricata processed a higher number of packets per second as compared to Snort, and both IDSs had a high rate of false
Syed Ali Razah Shah has finished M.Sc. in the School of Computing, Teesside University, UK and he has research interest in computer networks, network security and AI based applications.
References (74)
- et al.
Data mining and machine learning — towards reducing false positives in intrusion detection
Inf. Secur. Techn. Rep.
(2005) - et al.
Practical real-time intrusion detection using machine learning approaches
Comput. Commun.
(2011) - et al.
Comparison of the ARMA, ARIMA, and the autoregressive artificial neural network models in forecasting the monthly inflow of Dez dam reservoir
J. Hydrol.
(2013) - et al.
Selecting the best model to estimate potential evapotranspiration with respect to climate change and magnitudes of extreme events
Agric. Water Manag.
(2017) - et al.
Temporal variations of reference evapotranspiration and its sensitivity to meteorological factors in Heihe River Basin, China
Water Sci. Eng.
(2015) - et al.
Fuzzy logic-driven and SVM-driven hybrid computational intelligence models applied to oil and gas reservoir characterization
J. Nat. Gas Sci. Eng.
(2011) - NIST, E. Aroms, NIST Special Publication 800-94 Guide to Intrusion Detection and Prevention Systems (Idps),...
A comparative analysis of open source intrusion detection systems
(2012)Open Source IDS High Performance Shootout
(2015)- Snort, 2014. Available at: https://www.snort.org/ (Accessed: 10 June...
Statistical analysis of false positives and false negatives from real traffic with intrusion detection/prevention systems
IEEE Commun. Mag.
Administrative evaluation of intrusion detection system
A comparative analysis of the Snort and Suricata intrusion-detection systems
A machine learning based approach for intrusion prevention using honeypot interaction patterns as training data
IDS — analysis and containment of false positive alerts
Int. J. Comput. Appl.
Quantitative analysis of IDS: Snort and Suricata
Proc. SPIE
An information-theoretic measure of IDS capability
Scalable high-performance parallel design for network IDS on many-core processors
Improving the accuracy of network IDS under load using selective packet discarding
Analyzing offensive and defensive networking tools in a laboratory environment
Metasploit Penetration Testing Cookbook
Network Intrusion Detection
Cited by (0)
Syed Ali Razah Shah has finished M.Sc. in the School of Computing, Teesside University, UK and he has research interest in computer networks, network security and AI based applications.
Biju Issac is a senior lecturer in the School of Computing, Teesside University, UK. He has done Bachelor of Engineering in Electronics and Communication Engineering (ECE), Master of Computer Applications (MCA) with honours and Ph.D. in Networking and Mobile Communications, by research. He is a Charted Engineer (CEng), and Senior Member of IEEE.