Skip to main content
Top
Published in: International Journal of Information Security 5/2020

16-11-2019 | Regular Contribution

Cyberattack triage using incremental clustering for intrusion detection systems

Authors: Sona Taheri, Adil M. Bagirov, Iqbal Gondal, Simon Brown

Published in: International Journal of Information Security | Issue 5/2020

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Intrusion detection systems (IDSs) are devices or software applications that monitor networks or systems for malicious activities and signals alerts/alarms when such activity is discovered. However, an IDS may generate many false alerts which affect its accuracy. In this paper, we develop a cyberattack triage algorithm to detect these alerts (so-called outliers). The proposed algorithm is designed using the clustering, optimization and distance-based approaches. An optimization-based incremental clustering algorithm is proposed to find clusters of different types of cyberattacks. Using a special procedure, a set of clusters is divided into two subsets: normal and stable clusters. Then, outliers are found among stable clusters using an average distance between centroids of normal clusters. The proposed algorithm is evaluated using the well-known IDS data sets—Knowledge Discovery and Data mining Cup 1999 and UNSW-NB15—and compared with some other existing algorithms. Results show that the proposed algorithm has a high detection accuracy and its false negative rate is very low.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Chen, P.T., Laih, C.S.: Idsic: an intrusion detection system with identification capability. Int. J. Inf. Secur. 7(3), 185–197 (2008) Chen, P.T., Laih, C.S.: Idsic: an intrusion detection system with identification capability. Int. J. Inf. Secur. 7(3), 185–197 (2008)
2.
go back to reference Liao, H.J., Lin, Y.C., Lin, C.H.R., Tung, K.Y.: Review: intrusion detection system: a comprehensive review. J. Netw. Comput. Appl. 36(1), 16–24 (2013) Liao, H.J., Lin, Y.C., Lin, C.H.R., Tung, K.Y.: Review: intrusion detection system: a comprehensive review. J. Netw. Comput. Appl. 36(1), 16–24 (2013)
3.
go back to reference McHugh, J.: Intrusion and intrusion detection. Int. J. Inf. Secur. 1(1), 14–35 (2001)MATH McHugh, J.: Intrusion and intrusion detection. Int. J. Inf. Secur. 1(1), 14–35 (2001)MATH
5.
go back to reference Sommer, R., Paxson, V.: Outside the closed world: On using machine learning for network intrusion detection. In: Security and Privacy, pp. 305–316 (2010) Sommer, R., Paxson, V.: Outside the closed world: On using machine learning for network intrusion detection. In: Security and Privacy, pp. 305–316 (2010)
6.
go back to reference Umer, M.F., Sher, M., Bi, X.: A two-stage flow-based intrusion detection model for next-generation networks. PloS One 13, e0180945 (2018) Umer, M.F., Sher, M., Bi, X.: A two-stage flow-based intrusion detection model for next-generation networks. PloS One 13, e0180945 (2018)
7.
go back to reference Archana, D.W., Chatur, P.N.: Comparison of firewall and intrusion detection system. Int. J. Comput. Sci. Inf. Technol. 5(1), 674–678 (2014) Archana, D.W., Chatur, P.N.: Comparison of firewall and intrusion detection system. Int. J. Comput. Sci. Inf. Technol. 5(1), 674–678 (2014)
8.
go back to reference Kanika, U.: Security of network using Ids and firewall. Int. J. Sci. Res. Publ. 3(6), 1–4 (2013) Kanika, U.: Security of network using Ids and firewall. Int. J. Sci. Res. Publ. 3(6), 1–4 (2013)
9.
go back to reference Chakir, E.M., Codjovi, C., Khamlichi, Y.I., Moughit, M., First Settat, H.: False positives reduction in intrusion detection systems using alert correlation and data mining techniques. Int. J. Adv. Res. Comput. Sci. Softw. Eng. IJARCSSE 5, 77–85 (2015) Chakir, E.M., Codjovi, C., Khamlichi, Y.I., Moughit, M., First Settat, H.: False positives reduction in intrusion detection systems using alert correlation and data mining techniques. Int. J. Adv. Res. Comput. Sci. Softw. Eng. IJARCSSE 5, 77–85 (2015)
10.
go back to reference Gupta, N., Srivastava, K., Sharma, A.: Reducing false positive in intrusion detection system: a survey. Int. J. Comput. Sci. Inf. Technol. 7, 1600–1603 (2016) Gupta, N., Srivastava, K., Sharma, A.: Reducing false positive in intrusion detection system: a survey. Int. J. Comput. Sci. Inf. Technol. 7, 1600–1603 (2016)
11.
go back to reference Lippmann, R.P., Fried, D.J., Graf, I., Haines, J.W., Kendall, K.R., Mcclung, D., et al.: Evaluating intrusion detection systems: the 1998 darpa off-line intrusion detection evaluation. DARPA Inf. Surviv. Conf. Expos. 2, 12–26 (2000) Lippmann, R.P., Fried, D.J., Graf, I., Haines, J.W., Kendall, K.R., Mcclung, D., et al.: Evaluating intrusion detection systems: the 1998 darpa off-line intrusion detection evaluation. DARPA Inf. Surviv. Conf. Expos. 2, 12–26 (2000)
12.
go back to reference Spathoulas, G.P., Katsikas, S.K.: Reducing false positives in intrusion detection systems. Comput. Secur. 29(1), 35–44 (2010) Spathoulas, G.P., Katsikas, S.K.: Reducing false positives in intrusion detection systems. Comput. Secur. 29(1), 35–44 (2010)
13.
go back to reference Spathoulas, G.P., Katsikas, S.K.: Reducing false positives in intrusion detection systems. Comput. Secur. 29(1), 35–44 (2010) Spathoulas, G.P., Katsikas, S.K.: Reducing false positives in intrusion detection systems. Comput. Secur. 29(1), 35–44 (2010)
14.
go back to reference Kadam, P.U., Deshmukh, M.: Various approaches for intrusion detection system: an overview. Int. J. Innov. Res. Comput. Commun. Eng. 2(11), 6894–6902 (2014) Kadam, P.U., Deshmukh, M.: Various approaches for intrusion detection system: an overview. Int. J. Innov. Res. Comput. Commun. Eng. 2(11), 6894–6902 (2014)
15.
go back to reference Pareek, V., Mishra, A., Sharma, A., Chauhan, R., Bansal, S.: A deviation based outlier intrusion detection system. In: Chaki, N., Nagamalai, D., Meghanathan, N., Boumerdassi, S. (eds.) Recent Trends in Network Security and Applications, pp. 395–401. Springer, Berlin (2010) Pareek, V., Mishra, A., Sharma, A., Chauhan, R., Bansal, S.: A deviation based outlier intrusion detection system. In: Chaki, N., Nagamalai, D., Meghanathan, N., Boumerdassi, S. (eds.) Recent Trends in Network Security and Applications, pp. 395–401. Springer, Berlin (2010)
16.
go back to reference Mujumdar, A., Masiwal, G.,Dr. Meshram, B.B.: Analysis of signature-based and behavior-based anti-malware approaches. Int. J. Adv. Res. Comput. Eng. Tech. (IJARCET). 2(6), 2037–2039 (2013) Mujumdar, A., Masiwal, G.,Dr. Meshram, B.B.: Analysis of signature-based and behavior-based anti-malware approaches. Int. J. Adv. Res. Comput. Eng. Tech. (IJARCET). 2(6), 2037–2039 (2013)
17.
go back to reference Julisch, K.: Clustering intrusion detection alarms to support root cause analysis. ACM Trans. Inf. Syst. Secur. 6(4), 443–471 (2003) Julisch, K.: Clustering intrusion detection alarms to support root cause analysis. ACM Trans. Inf. Syst. Secur. 6(4), 443–471 (2003)
18.
go back to reference Rubin, S., Jha, S., Miller, B.P.: Automatic generation and analysis of NIDS attacks. In: Proceedings of the 20th Annual Computer Security Applications Conference, pp. 28–38. IEEE Computer Society, Washington, DC (2004) Rubin, S., Jha, S., Miller, B.P.: Automatic generation and analysis of NIDS attacks. In: Proceedings of the 20th Annual Computer Security Applications Conference, pp. 28–38. IEEE Computer Society, Washington, DC (2004)
19.
go back to reference Mishra, P., Varadharajan, V., Tupakula, U., Pilli, E.S.: A detailed investigation and analysis of using machine learning techniques for intrusion detection. IEEE Commun. Surv. Tutor. 21(1), 686–728 (2019) Mishra, P., Varadharajan, V., Tupakula, U., Pilli, E.S.: A detailed investigation and analysis of using machine learning techniques for intrusion detection. IEEE Commun. Surv. Tutor. 21(1), 686–728 (2019)
20.
go back to reference Li, Z., Das, A., Zhou, J.: Usaid: unifying signature-based and anomaly-based intrusion detection. In: Ho, T.B., Cheung, D., Liu, H. (eds.) Advances in Knowledge Discovery and Data Mining, pp. 702–712. Springer, Berlin (2005) Li, Z., Das, A., Zhou, J.: Usaid: unifying signature-based and anomaly-based intrusion detection. In: Ho, T.B., Cheung, D., Liu, H. (eds.) Advances in Knowledge Discovery and Data Mining, pp. 702–712. Springer, Berlin (2005)
21.
go back to reference Wagner, D., Soto, P.: Mimicry attacks on host-based intrusion detection systems. In: Proceedings of the 9th ACM Conference on Computer and Communications Security, pp. 255–264. ACM, New York (2002) Wagner, D., Soto, P.: Mimicry attacks on host-based intrusion detection systems. In: Proceedings of the 9th ACM Conference on Computer and Communications Security, pp. 255–264. ACM, New York (2002)
22.
go back to reference Breunig, M., Kriegel, H., Ng, R., Sander, J.: Lof: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp. 93–104 (2000) Breunig, M., Kriegel, H., Ng, R., Sander, J.: Lof: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp. 93–104 (2000)
23.
go back to reference Schubert, E., Zimek, A., Kriegel, H.P.: Generalized Outlier Detection with Flexible Kernel Density Estimates, pp. 542–550 (2014) Schubert, E., Zimek, A., Kriegel, H.P.: Generalized Outlier Detection with Flexible Kernel Density Estimates, pp. 542–550 (2014)
24.
go back to reference Zhang, K., Hutter, M., Jin, H.: A new local distance-based outlier detection approach for scattered real-world data. In: Theeramunkong, T., Kijsirikul, B., Cercone, N., Ho, T.B. (eds.) Advances in Knowledge Discovery and Data Mining, pp. 813–822. Springer, Berlin (2009) Zhang, K., Hutter, M., Jin, H.: A new local distance-based outlier detection approach for scattered real-world data. In: Theeramunkong, T., Kijsirikul, B., Cercone, N., Ho, T.B. (eds.) Advances in Knowledge Discovery and Data Mining, pp. 813–822. Springer, Berlin (2009)
25.
go back to reference Zhu, Q., Feng, J., Huang, J.: Natural neighbor: a self-adaptive neighborhood method without parameter k. Pattern Recognit. Lett. 80, 30–36 (2016) Zhu, Q., Feng, J., Huang, J.: Natural neighbor: a self-adaptive neighborhood method without parameter k. Pattern Recognit. Lett. 80, 30–36 (2016)
26.
go back to reference Jiang, M.F., Tseng, S.S., Su, C.M.: Two-phase clustering process for outliers detection. Pattern Recognit. Lett. 22(6), 691–700 (2001)MATH Jiang, M.F., Tseng, S.S., Su, C.M.: Two-phase clustering process for outliers detection. Pattern Recognit. Lett. 22(6), 691–700 (2001)MATH
27.
go back to reference Wang, C.H.: Outlier identification and market segmentation using kernel based clustering techniques. Expert Syst. Appl. 36(2), 3744–3750 (2009) Wang, C.H.: Outlier identification and market segmentation using kernel based clustering techniques. Expert Syst. Appl. 36(2), 3744–3750 (2009)
28.
go back to reference Lian, D., Xu, L., Liu, Y., Lee, J.: Cluster-based outlier detection. Ann. Oper. Res. 168(1), 151–168 (2009)MathSciNetMATH Lian, D., Xu, L., Liu, Y., Lee, J.: Cluster-based outlier detection. Ann. Oper. Res. 168(1), 151–168 (2009)MathSciNetMATH
29.
go back to reference Hachmi, F., Boujenfa, K., Limam, M.: An optimization process to identify outliers generated by intrusion detection systems. Secur. Commun. Netw. 8(18), 3469–3480 (2015) Hachmi, F., Boujenfa, K., Limam, M.: An optimization process to identify outliers generated by intrusion detection systems. Secur. Commun. Netw. 8(18), 3469–3480 (2015)
30.
go back to reference Pachgade, S.D., Dhande, S.S.: A heuristic algorithm for solving the minimum sum-of-squares clustering problems. Int. J. Adv. Res. Comput. Sci. Softw. Eng. 2(6), 12–16 (2012) Pachgade, S.D., Dhande, S.S.: A heuristic algorithm for solving the minimum sum-of-squares clustering problems. Int. J. Adv. Res. Comput. Sci. Softw. Eng. 2(6), 12–16 (2012)
31.
go back to reference Rizk, H., ElGokhy, M., Sarhan, A.: A hybrid outlier detection algorithm based on partitioning clustering and density measures. In: 2015 Tenth International Conference on Computer Engineering and Systems (ICCES), pp. 175–181 (2015) Rizk, H., ElGokhy, M., Sarhan, A.: A hybrid outlier detection algorithm based on partitioning clustering and density measures. In: 2015 Tenth International Conference on Computer Engineering and Systems (ICCES), pp. 175–181 (2015)
32.
go back to reference Dickson, A., Thomas, C.: Optimizing false alerts using multi-objective particle swarm optimization method. In: IEEE International Conference on Signal Processing, Informatics, Communication and Energy Systems (2015) Dickson, A., Thomas, C.: Optimizing false alerts using multi-objective particle swarm optimization method. In: IEEE International Conference on Signal Processing, Informatics, Communication and Energy Systems (2015)
33.
go back to reference Olsson, C., Eriksson, A., Hartley, R.: Outlier removal using duality. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2010) Olsson, C., Eriksson, A., Hartley, R.: Outlier removal using duality. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2010)
34.
go back to reference Seo, Y., Lee, H., Lee, S.: Outlier removal by convex optimization for l-infinity approaches. In: Toshikazu, W., Fay, H., Stephen, L. (eds.) Advances in Image and Video Technology, pp. 203–214. Springer, Heidelberg (2009) Seo, Y., Lee, H., Lee, S.: Outlier removal by convex optimization for l-infinity approaches. In: Toshikazu, W., Fay, H., Stephen, L. (eds.) Advances in Image and Video Technology, pp. 203–214. Springer, Heidelberg (2009)
35.
go back to reference Cannady, J., Harrell, J.: A comparative analysis of current intrusion detection technologies. In: Proc. of the Fourth Technology for Information Security Conference’96 (TISC’96) (2000) Cannady, J., Harrell, J.: A comparative analysis of current intrusion detection technologies. In: Proc. of the Fourth Technology for Information Security Conference’96 (TISC’96) (2000)
36.
go back to reference Bagirov, A.M., Ordin, B., Ozturk, G., Xavier, A.E.: An incremental clustering algorithm based on hyperbolic smoothing. Comput. Optim. Appl. 61(1), 219–241 (2015)MathSciNetMATH Bagirov, A.M., Ordin, B., Ozturk, G., Xavier, A.E.: An incremental clustering algorithm based on hyperbolic smoothing. Comput. Optim. Appl. 61(1), 219–241 (2015)MathSciNetMATH
37.
go back to reference Bagirov, A.M., Taheri, S., Ugon, J.: Nonsmooth DC programming approach to the minimum sum-of-squares clustering problems. Pattern Recognit. 53, 12–24 (2016)MATH Bagirov, A.M., Taheri, S., Ugon, J.: Nonsmooth DC programming approach to the minimum sum-of-squares clustering problems. Pattern Recognit. 53, 12–24 (2016)MATH
38.
go back to reference Ordin, B., Bagirov, A.M.: A heuristic algorithm for solving the minimum sum-of-squares clustering problems. J. Global Optim. 61(2), 341–361 (2015)MathSciNetMATH Ordin, B., Bagirov, A.M.: A heuristic algorithm for solving the minimum sum-of-squares clustering problems. J. Global Optim. 61(2), 341–361 (2015)MathSciNetMATH
39.
go back to reference Bagirov, A.M.: Modified global k-means algorithm for minimum sum-of squares clustering problems. Pattern Recognit. 41(10), 3192–3199 (2008)MATH Bagirov, A.M.: Modified global k-means algorithm for minimum sum-of squares clustering problems. Pattern Recognit. 41(10), 3192–3199 (2008)MATH
43.
go back to reference Tan, P.N., Steinbach, M., Kumar, V.: Introduction to Data Mining. Addison-Wesley Longman Publishing Co., Inc., Boston (2005) Tan, P.N., Steinbach, M., Kumar, V.: Introduction to Data Mining. Addison-Wesley Longman Publishing Co., Inc., Boston (2005)
44.
go back to reference Nour, M., Jill, S.: Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In: Military Communications and Information Systems Conference, IEEE, pp. 1–6 (2015) Nour, M., Jill, S.: Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In: Military Communications and Information Systems Conference, IEEE, pp. 1–6 (2015)
Metadata
Title
Cyberattack triage using incremental clustering for intrusion detection systems
Authors
Sona Taheri
Adil M. Bagirov
Iqbal Gondal
Simon Brown
Publication date
16-11-2019
Publisher
Springer Berlin Heidelberg
Published in
International Journal of Information Security / Issue 5/2020
Print ISSN: 1615-5262
Electronic ISSN: 1615-5270
DOI
https://doi.org/10.1007/s10207-019-00478-3

Other articles of this Issue 5/2020

International Journal of Information Security 5/2020 Go to the issue

Premium Partner