Performance comparison of intrusion detection systems and application of machine learning to Snort system

https://doi.org/10.1016/j.future.2017.10.016Get rights and content

Highlights

  • Two open source intrusion detection systems namely Snort and Suricata were compared.

  • Snort showed better detection accuracy but with false positive alarms.

  • Improving rule-based Snort’s accuracy with machine learning was attempted.

  • A Snort plug-in with SVM and Fuzzy logic produced good detection accuracy.

  • But the best result was achieved using an optimised SVM with firefly algorithm.

Abstract

This study investigates the performance of two open source intrusion detection systems (IDSs) namely Snort and Suricata for accurately detecting the malicious traffic on computer networks. Snort and Suricata were installed on two different but identical computers and the performance was evaluated at 10 Gbps network speed. It was noted that Suricata could process a higher speed of network traffic than Snort with lower packet drop rate but it consumed higher computational resources. Snort had higher detection accuracy and was thus selected for further experiments. It was observed that the Snort triggered a high rate of false positive alarms. To solve this problem a Snort adaptive plug-in was developed. To select the best performing algorithm for Snort adaptive plug-in, an empirical study was carried out with different learning algorithms and Support Vector Machine (SVM) was selected. A hybrid version of SVM and Fuzzy logic produced a better detection accuracy. But the best result was achieved using an optimised SVM with firefly algorithm with FPR (false positive rate) as 8.6% and FNR (false negative rate) as 2.2%, which is a good result. The novelty of this work is the performance comparison of two IDSs at 10 Gbps and the application of hybrid and optimised machine learning algorithms to Snort.

Introduction

Today many businesses rely on computer networks. These networks fulfil the needs of business, enterprises and government agencies to build knowledgeable, complicated information networks which integrate various technologies such as distributed data storage systems, encryption techniques, voice over IP (VoIP), remote or wireless access and web services. These computer networks have become more important as business partners access the information through extranets, customers communicate using networks through e-commerce transactions or Customer Relation Management (CRM) systems and employees connect with enterprise networks through virtual private networks (VPNs). These well-travelled paths make computer networks more vulnerable than ever before because today’s attackers are well organised as they have time, expertise and resources to launch the attacks that can avoid detection by even secure networks.

The attackers act like normal users, generate data and hide their malicious activities under terabytes of data. They know that many security mechanisms cannot protect the networks due to the large amount of data stored, scalability issues or due to the lack of detection capabilities. The enterprises and government agencies need to monitor their network traffic to detect malicious activities and perform analysis to differentiate the malicious and legitimate user activities to protect their networks. Detecting malicious activities require intrusion detection systems (IDS) and in today’s secure ICT infrastructure, the IDSs are part of most networks. However, the IDSs are only good if they have elite detection capabilities. It is critical that an IDS detection mechanism is accurate enough to differentiate between legitimate and malicious traffic that enter and leave the network. The possible results of using an IDS are as follows: detected malicious traffic (real alarms), undetected malicious traffic, legitimate traffic that IDS detect as malicious (false alarms) and legitimate traffic that IDS detect as good.

The elite IDSs detect as much malicious traffic as possible and reduce the number of false alarms. There are a number of commercial IDSs available in the market such as Juniper, McAfee, Cisco, Symantec etc. [1]. The commercial IDS generally do not provide the ideal performance as advertised and could compromise computer network security. Like the commercial IDSs, there are a number of open source IDSs available such as Snort, Suricata and Bro.

Snort and Suricata [2] were chosen for our study as we felt they have comparable functions, detection rule sets and syntax. They are both under GNU GPL licence. They both support intrusion prevention system (IPS) feature and support medium to high-speed network, though Suricata is more scalable with its multi-threaded architecture. Both support IPv6 traffic and their installation and deployment are easy. In contrast, Bro is a flexible script based IDS and its policy scripts or rules are written in its own Bro scripting language that does not rely on traditional signature detection [2]. It is under BSD licence and does not support IPv6 traffic. Installation of Bro can be difficult. Unlike Snort or Suricata, Bro does not offer inline intrusion prevention features. Both Snort and Suricata have similar features such as a module to capture the network packets, a module to decode and classify the network packets and a module to detect accurately the malicious or legitimate packets based on a rule set defined by both IDSs. Snort and Suricata inspect network packets for possible malicious traffic through the rule set and trigger alarms when the packet payload matches with one of the rules [3].

The Snort IDS has been in development since 1998 by Sourcefire and has become the de-facto standard for IDSs over the last decade. It is extensively deployed in networks and researched into. Snort has a single threaded architecture as shown in Fig. 1 which uses the TCP/IP stack to capture and inspect network packets payload [4]. Snort has added a multi-instance feature to its 2.9 release to address the limitation of single-thread and has hinted that version 3.0 will be multithreaded by default.

The Suricata IDS was developed in 2010 by the Open Information Security Foundation (OISF). Suricata is publicised as a future next generation IDS integrating new ideas such as multithreading as shown in Fig. 2. Based on the previous research it has improved on Snort because it uses multi-thread architecture to quickly capture and decode network packets [5].

Snort or Suricata can be installed on a computer host. The performance of both IDSs is directly proportional to the computer host performance in terms of CPU, memory utilisation and the packets drop rate by the network interface card.

The continued increase in network speed and malicious traffic causes significant problems for Snort and Suricata. Both IDSs have to process higher traffic volumes and inspect each incoming network packet to detect malicious traffic. In order for Snort and Suricata to have superior performance, both IDSs must process a vast volume of network traffic that can reach speeds up to 10 Gbps. If both IDSs fail to execute packet inspection at the required rate, they will drop the packets and allow malicious packets to enter the computer network undetected. Therefore, Snort and Suricata should be efficient enough to process network traffic up to 10 Gbps network speed.

Recently, numerous researchers have studied the support of machine learning algorithms for IDSs. Machine learning is a field of computer science that trains the computer to think like humans and take actions where required. In simple processing, a computer processes the information based on statements from primary memory. Machine learning techniques try to copy thinking processes such as logical reasoning, intuition, learning from the past, trial and error and generalisations [6].

Snort and Suricata use rules to detect the known malicious traffic. If malicious traffic matches with the rule set, then they will trigger the alarms. But both IDSs will not take any action against unknown malicious traffic. This is because Snort and Suricata do not use the machine learning techniques and hence cannot stop unknown malicious traffic. Machine learning techniques can help IDSs by accurately detecting the malicious traffic and therefore reduce the false positive alarms by proactively reacting against unknown malicious traffic. There are various machine learning algorithms that can be used for IDSs like Support Vector Machines, Decision Trees, Fuzzy Logic, BayesNet and NaiveBayes.

Snort and Suricata use pre-defined rules to detect malicious network traffic. If malicious traffic patterns match with the rule set then both IDSs trigger alarms, and these can be false positive, false negative or true positive alarms. Snort and Suricata alongside all the other IDSs have a common problem which is triggering false positive alarms [7]. For example, legitimate network traffic consists of DNS or web requests can lead the IDS to trigger a false positive alarm. This is because both IDSs have an identical rule set to match patterns for DNS or Web attacks. Therefore, false positive alarms affect the performance of Snort and Suricata and utilise a high computing resource for classifying the network traffic.

The aim of this paper is to do a performance comparison of Snort and Suricata and to implement machine learning algorithms on it to improve the detection accuracy. The aim led to the following objectives: (1) To critically review Snort and Suricata by measuring the performance and detection accuracy of both IDSs. (2) To evaluate the machine learning algorithms using three different datasets and to improve the selected IDS performance by implementing collective and optimised machine learning techniques through reducing the false positive alarms. The main contribution of this work is the performance comparison of Snort and Suricata at 10 Gbps and the application of the hybrid and optimised machine learning (ML) algorithms to Snort.

Section snippets

Related works on performance comparison

The idea of a performance comparison between Snort and Suricata is not new. Both perform well, but are not perfect and have limitations as shown in our experiments. Snort has a single-threaded architecture, and Suricata has a multi-threaded architecture which makes both IDS distinct from each other, but the rule set is the common feature of both IDS. Classifying the network traffic and accuracy of the rule set are the key elements of both IDS’s performance. Furthermore, computer host

Description of research methodology

Experiment scenarios were designed to make observations and to take measurements. This study demonstrated rigorous, repeatable, quantitative performance comparisons of both IDSs and evaluated the machine learning algorithms.

The experiments consisted of a test bed which compared Snort and Suricata’s detection accuracy in 10 Gbps network speed and with seven different types of malicious traffic. The seven types of malicious traffic were chosen because the rules could be applied consistently to

Experiment scenarios

The experiment scenarios were planned and set up to compare the performance of Snort and Suricata on identical VMs using identical rule set and under the same test conditions.

Snort IDS using machine learning techniques

We will consider Snort to implement machine learning [32]. Snort is dependent on the rule set to detect malicious traffic that gives an accurate description of the known malicious traffic. To reduce FPR, many researchers have used machine learning algorithms to classify normal and malicious traffic. The following machine learning algorithms have been extensively studied like Support Vector Machine, Decision Trees, Fuzzy Logic, BayesNet and NaiveBayes.

Also, various other machine learning

Evaluation of machine learning algorithms (MLAs)

Before using machine learning techniques in conjunction with Snort, it is important to know about high performing machine learning algorithms. Three publicly available datasets were used to conduct performance experiments on MLAs. Initially, the experiment evaluation environment was built which consisted of evaluation setup and data preprocessing using data mining software [33]. Secondly, a literature search was conducted, and a comprehensive set of five high-performing machine learning

Proposed Snort adaptive plug-in

As shown in the Table 8, Table 9, Snort has successfully detected six types of malicious traffic out of seven, but it triggered high false positive alarms (55.2% average). The proposed Snort adaptive plug-in architecture for Snort is to reduce the false positive alarms using machine learning algorithms. The proposed new architecture of Snort IDS is shown in Fig. 6.

The proposed Snort adaptive plug-in operates in parallel with the Snort’s rule set. The rationale for integrating the Snort adaptive

Conclusion

This study compared the performance of two open source IDSs, namely Snort and Suricata. They were both proved to be efficient and high performing IDS, though each had its own strengths and weaknesses. The analysis of the experiment results showed that Snort utilised less computational resources to process 10 Gbps network traffic whereas Suricata’s utilisation was higher. Suricata processed a higher number of packets per second as compared to Snort, and both IDSs had a high rate of false

Syed Ali Razah Shah has finished M.Sc. in the School of Computing, Teesside University, UK and he has research interest in computer networks, network security and AI based applications.

References (74)

  • Suricata, 2014. Available at: https://suricata-ids.org/ (Accessed: 10 June...
  • J. Frank, Artificial intelligence and intrusion detection: Current and future directions, in: Proceedings of the 17th...
  • HoCheng-Yuan et al.

    Statistical analysis of false positives and false negatives from real traffic with intrusion detection/prevention systems

    IEEE Commun. Mag.

    (2012)
  • E. Albin, N.C. Rowe, A realistic experimental comparison of the suricata and snort intrusion-detection systems, in:...
  • WangX. et al.

    Administrative evaluation of intrusion detection system

  • W. Bulajoul, A. James, M. Pannu, Network intrusion detection systems in high-speed traffic in computer networks, in:...
  • Z. Trabelsi, S. Zeidan, IDS performance enhancement technique based on dynamic traffic awareness histograms, in: IEEE...
  • A. Saboor, M. Akhlaq, B. Aslam, Experimental evaluation of Snort against DDoS attacks under different hardware...
  • M.B. Shahbaz, X. Wang, A. Behnad, J. Samarabandu, On efficiency enhancement of the correlation-based feature selection...
  • A. Alhomoud, R. Munir, J.P. Disso, I. Awan, A. Al-Dhelaan, Performance evaluation study of intrusion detection systems,...
  • AlbinE.

    A comparative analysis of the Snort and Suricata intrusion-detection systems

    (2011)
  • ZammitD.

    A machine learning based approach for intrusion prevention using honeypot interaction patterns as training data

    (2016)
  • C. Huang, J. Xiongand, Z. Peng, Applied research on Snort intrusion detection model in the campus network, in: 2012...
  • VictorJ. et al.

    IDS — analysis and containment of false positive alerts

    Int. J. Comput. Appl.

    (2010)
  • WhiteJ.S. et al.

    Quantitative analysis of IDS: Snort and Suricata

    Proc. SPIE

    (2013)
  • GuGuofei et al.

    An information-theoretic measure of IDS capability

  • K. Hafeez, M. Masood, O. Malik, Z. Anwar, LASSP: A logic analyzer for tweaking snort security and performance, in: 2010...
  • JiangH. et al.

    Scalable high-performance parallel design for network IDS on many-core processors

  • A. Munoz, S. Sezer, D. Burns, G. Douglas, An approach for unifying rule based deep packet inspection, in: 2011 IEEE...
  • R. Padmashani, S. Sathyadevan, D. Dath, BSnort IPS better snort intrusion detection/prevention system, in: 2012 12th...
  • PapadogiannakisA. et al.

    Improving the accuracy of network IDS under load using selective packet discarding

  • K. Salah, A. Qahtan, Boosting throughput of Snort NIDS under Linux, in: 2008 International Conference on Innovations in...
  • K. Thongkanchorn, S. Ngamsuriyaroj, V. Visoottiviseth, Evaluation studies of three IDS under various attacks and rule...
  • HeikuraN.

    Analyzing offensive and defensive networking tools in a laboratory environment

    (2015)
  • AgarwalM. et al.

    Metasploit Penetration Testing Cookbook

    (2013)
  • NorthcuttStephen et al.

    Network Intrusion Detection

    (2003)
  • Armitage, Fast and easy hacking, 2014. Available at: http://www.fastandeasyhacking.com/manual  (Accessed: 15 December...
  • Cited by (0)

    Syed Ali Razah Shah has finished M.Sc. in the School of Computing, Teesside University, UK and he has research interest in computer networks, network security and AI based applications.

    Biju Issac is a senior lecturer in the School of Computing, Teesside University, UK. He has done Bachelor of Engineering in Electronics and Communication Engineering (ECE), Master of Computer Applications (MCA) with honours and Ph.D. in Networking and Mobile Communications, by research. He is a Charted Engineer (CEng), and Senior Member of IEEE.

    View full text