Skip to main content
Top

2022 | OriginalPaper | Chapter

DAEMON: Dynamic Auto-encoders for Contextualised Anomaly Detection Applied to Security MONitoring

Authors : Alexandre Dey, Eric Totel, Benjamin Costé

Published in: ICT Systems Security and Privacy Protection

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The slow adoption rate of machine learning-based methods for novel attack detection by Security Operation Centers (SOC) analysts can be partly explained by their lack of data science expertise and the insufficient explainability of the results provided by these approaches. In this paper, we present an anomaly-based detection method that fuses events coming from heterogeneous sources into sets describing the same phenomenons and relies on a deep auto-encoder model to highlight anomalies and their context. To implicate security analysts and benefit from their expertise, we focus on limiting the need of data science knowledge during the configuration phase. Results on a lab environment, monitored using off-the-shelf tools, show good detection performances on several attack scenarios (F1 score \({\approx }0.9\)), and eases the investigation of anomalies by quickly finding similar anomalies through clustering.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Footnotes
1
Timestamp excepted.
 
Literature
1.
go back to reference Bertero, C., Roy, M., Sauvanaud, C., Trédan, G.: Experience report: log mining using natural language processing and application to anomaly detection. In: 2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE), pp. 351–360. IEEE (2017) Bertero, C., Roy, M., Sauvanaud, C., Trédan, G.: Experience report: log mining using natural language processing and application to anomaly detection. In: 2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE), pp. 351–360. IEEE (2017)
2.
go back to reference Brogi, G., Tong, V.V.T.: TerminAPTor: highlighting advanced persistent threats through information flow tracking. In: 2016 8th IFIP International Conference on New Technologies, Mobility and Security (NTMS), pp. 1–5. IEEE (2016) Brogi, G., Tong, V.V.T.: TerminAPTor: highlighting advanced persistent threats through information flow tracking. In: 2016 8th IFIP International Conference on New Technologies, Mobility and Security (NTMS), pp. 1–5. IEEE (2016)
3.
go back to reference de Masson d’Autume, C., Ruder, S., Kong, L., Yogatama, D.: Episodic memory in lifelong language learning. CoRR abs/1906.01076 (2019) de Masson d’Autume, C., Ruder, S., Kong, L., Yogatama, D.: Episodic memory in lifelong language learning. CoRR abs/1906.01076 (2019)
4.
go back to reference Debnath, B., et al.: LogLens: a real-time log analysis system. In: 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), pp. 1052–1062. IEEE (2018) Debnath, B., et al.: LogLens: a real-time log analysis system. In: 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), pp. 1052–1062. IEEE (2018)
6.
go back to reference Ding, Z., Fei, M.: An anomaly detection approach based on isolation forest algorithm for streaming data using sliding window. IFAC Proceedings Volumes 46, 12–17 (2013)CrossRef Ding, Z., Fei, M.: An anomaly detection approach based on isolation forest algorithm for streaming data using sliding window. IFAC Proceedings Volumes 46, 12–17 (2013)CrossRef
7.
go back to reference Du, M., Li, F., Zheng, G., Srikumar, V.: DeepLog: anomaly detection and diagnosis from system logs through deep learning. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 1285–1298. ACM (2017) Du, M., Li, F., Zheng, G., Srikumar, V.: DeepLog: anomaly detection and diagnosis from system logs through deep learning. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 1285–1298. ACM (2017)
9.
go back to reference Ester, M., Kriegel, H.-P., Sander, J., Xu, X., et al.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: KDD, vol. 96, pp. 226–231 (1996) Ester, M., Kriegel, H.-P., Sander, J., Xu, X., et al.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: KDD, vol. 96, pp. 226–231 (1996)
10.
go back to reference Hassan, W.U., Bates, A., Marino, D.: Tactical provenance analysis for endpoint detection and response systems. In: 2020 IEEE Symposium on Security and Privacy (SP), pp. 1172–1189. IEEE (2020) Hassan, W.U., Bates, A., Marino, D.: Tactical provenance analysis for endpoint detection and response systems. In: 2020 IEEE Symposium on Security and Privacy (SP), pp. 1172–1189. IEEE (2020)
11.
go back to reference He, S., Zhu, J., He, P., Lyu, M.R.: Experience report: system log analysis for anomaly detection. In: 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE), pp. 207–218. IEEE (2016) He, S., Zhu, J., He, P., Lyu, M.R.: Experience report: system log analysis for anomaly detection. In: 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE), pp. 207–218. IEEE (2016)
12.
go back to reference Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. Proc. Natl. Acad. Sci. 114, 3521–3526 (2017)MathSciNetCrossRef Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. Proc. Natl. Acad. Sci. 114, 3521–3526 (2017)MathSciNetCrossRef
13.
go back to reference Leichtnam, L., Totel, E., Prigent, N., Mé, L.: Forensic analysis of network attacks: restructuring security events as graphs and identifying strongly connected sub-graphs. In: 2020 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW), pp. 565–573. IEEE (2020) Leichtnam, L., Totel, E., Prigent, N., Mé, L.: Forensic analysis of network attacks: restructuring security events as graphs and identifying strongly connected sub-graphs. In: 2020 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW), pp. 565–573. IEEE (2020)
14.
go back to reference Liu, F., Wen, Y., Zhang, D., Jiang, X., Xing, X., Meng, D.: Log2vec: a heterogeneous graph embedding based approach for detecting cyber threats within enterprise. In: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pp. 1777–1794 (2019) Liu, F., Wen, Y., Zhang, D., Jiang, X., Xing, X., Meng, D.: Log2vec: a heterogeneous graph embedding based approach for detecting cyber threats within enterprise. In: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pp. 1777–1794 (2019)
15.
go back to reference Liu, F.T., Ting, K.M., Zhou, Z.-H.: Isolation forest. In: 2008 Eighth IEEE International Conference on Data Mining, pp. 413–422. IEEE (2008) Liu, F.T., Ting, K.M., Zhou, Z.-H.: Isolation forest. In: 2008 Eighth IEEE International Conference on Data Mining, pp. 413–422. IEEE (2008)
16.
go back to reference Milajerdi, S.M., Gjomemo, R., Eshete, B., Sekar, R., Venkatakrishnan, V.: HOLMES: real-time apt detection through correlation of suspicious information flows. In: 2019 IEEE Symposium on Security and Privacy (SP), pp. 1137–1152. IEEE (2019) Milajerdi, S.M., Gjomemo, R., Eshete, B., Sekar, R., Venkatakrishnan, V.: HOLMES: real-time apt detection through correlation of suspicious information flows. In: 2019 IEEE Symposium on Security and Privacy (SP), pp. 1137–1152. IEEE (2019)
17.
go back to reference Mirsky, Y., Doitshman, T., Elovici, Y., Shabtai, A.: Kitsune: an ensemble of autoencoders for online network intrusion detection. arXiv preprint arXiv:1802.09089 (2018) Mirsky, Y., Doitshman, T., Elovici, Y., Shabtai, A.: Kitsune: an ensemble of autoencoders for online network intrusion detection. arXiv preprint arXiv:​1802.​09089 (2018)
19.
go back to reference Pascoal, C., De Oliveira, M.R., Valadas, R., Filzmoser, P., Salvador, P., Pacheco, A.: Robust feature selection and robust PCA for internet traffic anomaly detection. In: 2012 Proceedings IEEE INFOCOM, pp. 1755–1763. IEEE (2012) Pascoal, C., De Oliveira, M.R., Valadas, R., Filzmoser, P., Salvador, P., Pacheco, A.: Robust feature selection and robust PCA for internet traffic anomaly detection. In: 2012 Proceedings IEEE INFOCOM, pp. 1755–1763. IEEE (2012)
20.
go back to reference Paxson, V.: Bro: a system for detecting network intruders in real-time. Comput. Netw. 31, 2435–2463 (1999)CrossRef Paxson, V.: Bro: a system for detecting network intruders in real-time. Comput. Netw. 31, 2435–2463 (1999)CrossRef
21.
go back to reference Pei, K., et al.: HERCULE: attack story reconstruction via community discovery on correlated log graph. In: Proceedings of the Annual Computer Security Applications Conference (ACSAC). ACM (2016) Pei, K., et al.: HERCULE: attack story reconstruction via community discovery on correlated log graph. In: Proceedings of the Annual Computer Security Applications Conference (ACSAC). ACM (2016)
22.
go back to reference Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T.P., Wayne, G.: Experience replay for continual learning. CoRR abs/1811.11682 (2018) Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T.P., Wayne, G.: Experience replay for continual learning. CoRR abs/1811.11682 (2018)
23.
go back to reference Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. Technical report, California Univ San Diego La Jolla Inst for Cognitive Science (1985) Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. Technical report, California Univ San Diego La Jolla Inst for Cognitive Science (1985)
24.
go back to reference Sadoddin, R., Ghorbani, A.A.: An incremental frequent structure mining framework for real-time alert correlation. Comput. Secur. 28, 153–173 (2009)CrossRef Sadoddin, R., Ghorbani, A.A.: An incremental frequent structure mining framework for real-time alert correlation. Comput. Secur. 28, 153–173 (2009)CrossRef
25.
go back to reference Sprechmann, P., et al.: Memory-based parameter adaptation (2018) Sprechmann, P., et al.: Memory-based parameter adaptation (2018)
26.
go back to reference Valeur, F.: Real-time intrusion detection alert correlation. Citeseer (2006) Valeur, F.: Real-time intrusion detection alert correlation. Citeseer (2006)
27.
go back to reference Veeramachaneni, K., Arnaldo, I., Korrapati, V., Bassias, C., Li, K.: Ai2: training a big data machine to defend. In: 2016 IEEE 2nd International Conference on Big Data Security on Cloud (BigDataSecurity), IEEE International Conference on High Performance and Smart Computing (HPSC), and IEEE International Conference on Intelligent Data and Security (IDS), pp. 49–54. IEEE (2016) Veeramachaneni, K., Arnaldo, I., Korrapati, V., Bassias, C., Li, K.: Ai2: training a big data machine to defend. In: 2016 IEEE 2nd International Conference on Big Data Security on Cloud (BigDataSecurity), IEEE International Conference on High Performance and Smart Computing (HPSC), and IEEE International Conference on Intelligent Data and Security (IDS), pp. 49–54. IEEE (2016)
28.
go back to reference Viinikka, J., Debar, H., Mé, L., Lehikoinen, A., Tarvainen, M.: Processing intrusion detection alert aggregates with time series modeling. Inf. Fusion 10, 312–324 (2009)CrossRef Viinikka, J., Debar, H., Mé, L., Lehikoinen, A., Tarvainen, M.: Processing intrusion detection alert aggregates with time series modeling. Inf. Fusion 10, 312–324 (2009)CrossRef
29.
go back to reference Xosanavongsa, C., Totel, E., Bettan, O.: Discovering correlations: a formal definition of causal dependency among heterogeneous events. In: 2019 IEEE European Symposium on Security and Privacy (EuroS P), pp. 340–355 (2019) Xosanavongsa, C., Totel, E., Bettan, O.: Discovering correlations: a formal definition of causal dependency among heterogeneous events. In: 2019 IEEE European Symposium on Security and Privacy (EuroS P), pp. 340–355 (2019)
30.
go back to reference Zhou, C., Paffenroth, R.C.: Anomaly detection with robust deep autoencoders. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 665–674. ACM (2017) Zhou, C., Paffenroth, R.C.: Anomaly detection with robust deep autoencoders. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 665–674. ACM (2017)
Metadata
Title
DAEMON: Dynamic Auto-encoders for Contextualised Anomaly Detection Applied to Security MONitoring
Authors
Alexandre Dey
Eric Totel
Benjamin Costé
Copyright Year
2022
DOI
https://doi.org/10.1007/978-3-031-06975-8_4

Premium Partner