Skip to main content
SpringerLink
Log in

Requirements for Training and Evaluation Dataset of Network and Host Intrusion Detection System

  • Conference paper
  • First Online:
New Knowledge in Information Systems and Technologies (WorldCIST'19 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 931))

Included in the following conference series:

Abstract

In the cyber domain, situational awareness of the critical assets is extremely important. For achieving comprehensive situational awareness, accurate sensor information is required. An important branch of sensors are Intrusion Detection Systems (IDS), especially anomaly based intrusion detection systems applying artificial intelligence or machine learning for anomaly detection. This millennium has seen the transformation of industries due to the developments in data based modelling methods. The most crucial bottleneck for modelling the IDS is the absence of publicly available datasets compliant to modern equipment, system design standards and cyber threat landscape. The predominant dataset, the KDD Cup 1999, is still actively used in IDS modelling research despite the expressed criticism. Other, more recent datasets, tend to record data only either from the perimeters of the testbed environment’s network traffic or from the effects that malware has on a single host machine. Our study focuses on forming a set of requirements for a holistic Network and Host Intrusion Detection System (NHIDS) dataset by reviewing existing and studied datasets within the field of IDS modelling. As a result, the requirements for state-of-the-art NHIDS dataset are presented to be utilised for research and development of NHIDS applying machine learning and artificial intelligence.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abubakar, A.I., Chiroma, H., Muaz, S.A., Ila, L.B.: A review of the advances in cyber security benchmark datasets for evaluating data-driven based intrusion detection systems. Procedia Comput. Sci. 62, 221–227 (2015). https://doi.org/10.1016/j.procs.2015.08.443

    Article  Google Scholar 

  2. Alejandre, F.V., Cortés, N.C., Anaya, E.A.: Feature selection to detect botnets using machine learning algorithms. In: 2017 International Conference on Electronics, Communications and Computers, CONIELECOMP 2017, pp. 1–7. IEEE (2017). https://doi.org/10.1109/CONIELECOMP.2017.7891834

  3. Aviv, A.J., Haeberlen, A.: Challenges in experimenting with botnet detection systems. In: Proceedings of the 4th Conference on Cyber Security Experimentation and Test, CSET 2011, p. 6. USENIX Association, Berkeley (2011). http://dl.acm.org/citation.cfm?id=2027999.2028005

  4. Bodström, T., Hämäläinen, T.: State of the art literature review on network anomaly detection with deep learning. In: Galinina, O., Andreev, S., Balandin, S., Koucheryavy, Y. (eds.) Internet of Things, Smart Spaces, and Next Generation Networks and Systems, pp. 64–76. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01168-0_7

    Google Scholar 

  5. Buczak, A., Guven, E.: A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Commun. Surv. Tutor. 18(2), 1153– (2015). https://doi.org/10.1109/COMST.2015.2494502

    Article  Google Scholar 

  6. Chattopadhyay, M., Sen, R., Gupta, S.: A comprehensive review and meta-analysis on applications of machine learning techniques in intrusion detection. Australas. J. Inf. Syst. 22, 1–27 (2018). https://doi.org/10.3127/ajis.v22i0.1667

  7. Chio, C., Freeman, D.: Machine Learning and Security. O’Reilly Media Inc., Sebastopol (2018)

    Google Scholar 

  8. Creech, G.: Developing a high-accuracy cross platform host-based intrusion detection system capable of reliably detecting zero-day attacks. Ph.D. thesis (2013). http://handle.unsw.edu.au/1959.4/53218

  9. Creech, G., Hu, J.: Generation of a new IDS test dataset: time to retire the KDD collection. In: IEEE Wireless Communications and Networking Conference, WCNC, pp. 4487–4492. IEEE (2013). https://doi.org/10.1109/WCNC.2013.6555301

  10. Creech, G., Hu, J.: A semantic approach to host-based intrusion detection systems using contiguousand discontiguous system call patterns. IEEE Trans. Comput. 63(4), 807–819 (2014). https://doi.org/10.1109/TC.2013.13

    Article  MathSciNet  MATH  Google Scholar 

  11. Ferguson, B., Tall, A., Olsen, D.: National cyber range overview. In: 2014 IEEE Military Communications Conference, pp. 123–128 (2014). https://doi.org/10.1109/MILCOM.2014.27

  12. García, S., Grill, M., Stiborek, J., Zunino, A.: An empirical comparison of botnet detection methods. Comput. Secur. 45, 100–123 (2014). https://doi.org/10.1016/j.cose.2014.05.011

    Article  Google Scholar 

  13. Hofmeyr, S.A., Forrest, S., Somayaji, A.: Intrusion detection using sequences of system calls. J. Comput. Secur. 6(3), 151–180 (1998). https://doi.org/10.3233/JCS-980109

    Article  Google Scholar 

  14. Husak, M., Komarkova, J., Bou-Harb, E., Celeda, P.: Survey of attack projection, prediction, and forecasting in cyber security. IEEE Commun. Surv. Tutor. (2018). https://doi.org/10.1109/COMST.2018.2871866

    Article  Google Scholar 

  15. JAMK University of Applied Sciences, Institute of Information Technology, JYVSECTEC: RGCE Cyber Range. http://www.jyvsectec.fi/rgce/. Accessed 23 Nov 2018

  16. Kokkonen, T., Puuska, S.: Blue team communication and reporting for enhancing situational awareness from white team perspective in cyber security exercises. In: Galinina, O., Andreev, S., Balandin, S., Koucheryavy, Y. (eds.) Internet of Things, Smart Spaces, and Next Generation Networks and Systems, pp. 277–288. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01168-0_26

    Google Scholar 

  17. Mathur, L., Raheja, M., Ahlawat, P.: Botnet detection via mining of network traffic flow. Procedia Comput. Sci. 132, 1668–1677 (2018). https://doi.org/10.1016/j.procs.2018.05.137

    Article  Google Scholar 

  18. Mishra, P., Pilli, E.S., Varadharajan, V., Tupakula, U.: Intrusion detection techniques in cloud environment: a survey. J. Netw. Comput. Appl. 77, 18–47 (2017). https://doi.org/10.1016/j.jnca.2016.10.015

    Article  Google Scholar 

  19. Mishra, P., Varadharajan, V., Tupakula, U., Pilli, E.S.: A detailed investigation and analysis of using machine learning techniques for intrusion detection. IEEE Commun. Surv. Tutor. (2018). https://doi.org/10.1109/COMST.2018.2847722

    Article  Google Scholar 

  20. Moustafa, N., Slay, J.: UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In: 2015 Military Communications and Information Systems Conference (MilCIS), pp. 1–6 (2015). https://doi.org/10.1109/MilCIS.2015.7348942

  21. Moustafa, N., Slay, J.: The evaluation of network anomaly detection systems: statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 data set. Inf. Secur. J. 25(1–3), 18–31 (2016). https://doi.org/10.1080/19393555.2015.1125974

    Article  Google Scholar 

  22. National Institute of Standards and Technology NIST: Cyber Ranges. https://www.nist.gov/sites/default/files/documents/2018/02/13/cyber_ranges.pdf. Accessed 23 Nov 2018

  23. Puuska, S., Kokkonen, T., Alatalo, J., Heilimo, E.: Anomaly-based network intrusion detection using wavelets and adversarial autoencoders. In: Lanet, J.-L., Toma, C. (eds.) Innovative Security Solutions for Information Technology and Communications, pp. 234–246. Springer International Publishing (2019). https://doi.org/10.1007/978-3-030-12942-2_18

    Chapter  Google Scholar 

  24. Saad, S., Traore, I., Ghorbani, A., Sayed, B., Zhao, D., Lu, W., Felix, J., Hakimian, P.: Detecting P2P botnets through network behavior analysis and machine learning. In: 2011 Ninth Annual International Conference on Privacy, Security and Trust, pp. 174–180 (2011). https://doi.org/10.1109/PST.2011.5971980

  25. Shiravi, A., Shiravi, H., Tavallaee, M., Ghorbani, A.A.: Toward developing a systematic approach to generate benchmark datasets for intrusion detection. Comput. Secur. 31(3), 357–374 (2012). https://doi.org/10.1016/j.cose.2011.12.012

    Article  Google Scholar 

  26. SimpleWiki: Labeled Dataset for Intrusion Detection. https://www.simpleweb.org/wiki/index.php/Labeled_Dataset_for_Intrusion_Detection. Accessed 19 November 2018

  27. Sperotto, A., Sadre, R., Van Vliet, F., Pras, A.: A labeled data set for flow-based intrusion detection. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5843, pp. 39–50. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04968-2_4

    Chapter  Google Scholar 

  28. Sperotto, A., Schaffrath, G., Sadre, R., Morariu, C., Pras, A., Stiller, B.: An overview of IP flow-based intrusion detection. IEEE Commun. Surv. Tutor. 12(3), 343–356 (2010). https://doi.org/10.1109/SURV.2010.032210.00054

    Article  Google Scholar 

  29. Stolfo, S.J., Fan, W., Lee, W., Prodromidis, A., Chan, P.K.: Cost-based modeling for fraud and intrusion detection: results from the JAM project. In: Proceedings DARPA Information Survivability Conference and Exposition, DISCEX 2000, vol. 2, pp. 130–144 (2000). https://doi.org/10.1109/DISCEX.2000.821515

  30. Tavallaee, M., Bagheri, E., Lu, W., Ghorbani, A.A.: A detailed analysis of the KDD CUP 99 data set. In: 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, pp. 1–6. IEEE (2009). https://doi.org/10.1109/CISDA.2009.5356528

  31. Tavallaee, M., Stakhanova, N., Ghorbani, A.A.: Toward credible evaluation of anomaly-based intrusion-detection methods. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 40(5), 516–524 (2010). https://doi.org/10.1109/TSMCC.2010.2048428

    Article  Google Scholar 

  32. Umer, M.F., Sher, M., Bi, Y.: Flow-based intrusion detection: techniques and challenges. Comput. Secur. 70, 238–254 (2017). https://doi.org/10.1016/j.cose.2017.05.009

    Article  Google Scholar 

  33. KDD Cup 1999 Data. University of California, Irvine. http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html. Accessed 23 Nov 2018

  34. University of New South Wales: The UNSW-NB15 Dataset Description. https://www.unsw.adfa.edu.au/unsw-canberra-cyber/cybersecurity/ADFA-NB15-Datasets/. Accessed 19 Nov 2018

  35. University of Victoria, ISOT Research Lab: Datasets. https://www.uvic.ca/engineering/ece/isot/datasets/. Accessed 23 Nov 2018

  36. Xin, Y., Kong, L., Liu, Z., Chen, Y., Li, Y., Zhu, H., Gao, M., Hou, H., Wang, C.: Machine learning and deep learning methods for cybersecurity. IEEE Access 6, 35365–35381 (2018). https://doi.org/10.1109/ACCESS.2018.2836950

    Article  Google Scholar 

Download references

Acknowledgment

This research is partially funded by the Regional Council of Central Finland/Council of Tampere Region and European Regional Development Fund as part of the New Business Innovations from Data-analytics project of JAMK University of Applied Sciences Institute of Information Technology.

Author information

Authors and Affiliations

  1. Institute of Information Technology, JAMK University of Applied Sciences, Jyväskylä, Finland

    Petteri Nevavuori & Tero Kokkonen

Authors
  1. Petteri Nevavuori
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tero Kokkonen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tero Kokkonen .

Editor information

Editors and Affiliations

  1. Departamento de Engenharia Informática, Universidade de Coimbra, Coimbra, Portugal

    Álvaro Rocha

  2. The Ohio State University, Columbus, OH, USA

    Hojjat Adeli

  3. Faculdade de Engenharia/LIACC, Universidade do Porto, Porto, Portugal

    Luís Paulo Reis

  4. DIMES, Università della Calabria, Arcavacata di Rende, Italy

    Sandra Costanzo

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nevavuori, P., Kokkonen, T. (2019). Requirements for Training and Evaluation Dataset of Network and Host Intrusion Detection System. In: Rocha, Á., Adeli, H., Reis, L., Costanzo, S. (eds) New Knowledge in Information Systems and Technologies. WorldCIST'19 2019. Advances in Intelligent Systems and Computing, vol 931. Springer, Cham. https://doi.org/10.1007/978-3-030-16184-2_51

Download citation

Publish with us

Policies and ethics

Search

Navigation