Skip to main content

2018 | OriginalPaper | Buchkapitel

Feature Extraction in Security Analytics: Reducing Data Complexity with Apache Spark

verfasst von : Dimitrios Sisiaridis, Olivier Markowitch

Erschienen in: Security with Intelligent Computing and Big-data Services

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Feature extraction is the first task of pre-processing input logs in order to detect cybersecurity threats and attacks while utilizing machine learning. When it comes to the analysis of heterogeneous data derived from different sources, this task is found to be time-consuming and difficult to be managed efficiently. In this paper we present an approach for handling feature extraction for security analytics of heterogeneous data derived from different network sensors. The approach is implemented in Apache Spark, using its python API, named pyspark.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
The term flattening refers to data expressed in 2-D.
 
2
The kill chain model [2] is an intelligence-driven, threat-focused approach to study intrusions from the adversaries perspective. The fundamental element is the indicator which corresponds to any piece of information that can describe a threat or an attack. Indicators can be either atomic such as IP or email addresses, computed such as hash values or regular expressions, or behavioural which are collections of computed and atomic indicators such as statements.
 
Literatur
1.
Zurück zum Zitat Bird, S., Klein, E., Loper, E.: Natural Language Processing with Python. O’ Reilly Media Inc. (2009) Bird, S., Klein, E., Loper, E.: Natural Language Processing with Python. O’ Reilly Media Inc. (2009)
2.
Zurück zum Zitat Hutchins, E.M., Cloppert, M.J., Amin, R.M.: Intelligence-driven computer network defense informed by analysis of adversary campaigns and intrusion kill chains. In: Ryan, J. (ed.) Leading Issues in Information Warfare and Security Research, vol. 1, p. 80. Academic Publishing International Ltd., Reading (2011) Hutchins, E.M., Cloppert, M.J., Amin, R.M.: Intelligence-driven computer network defense informed by analysis of adversary campaigns and intrusion kill chains. In: Ryan, J. (ed.) Leading Issues in Information Warfare and Security Research, vol. 1, p. 80. Academic Publishing International Ltd., Reading (2011)
3.
Zurück zum Zitat Kalyan, V., Ignacio, A., Alfredo, C.-I., Vamsi, K., Costas, B., Ke, L.: AI2: Training a big data machine to defend. In: IEEE International Conference on Big Data Security, New York, NY, USA, June 2016 Kalyan, V., Ignacio, A., Alfredo, C.-I., Vamsi, K., Costas, B., Ke, L.: AI2: Training a big data machine to defend. In: IEEE International Conference on Big Data Security, New York, NY, USA, June 2016
4.
Zurück zum Zitat Shyu, M.-L., Huang, Z., Luo, H.: Efficient mining and detection of sequential intrusion patterns for network intrusion detection systems. In: Yu, P.S., Tsai, J.J.P. (eds.) Machine Learning in Cyber Trust, pp. 133–154. Springer, Boston (2009)CrossRef Shyu, M.-L., Huang, Z., Luo, H.: Efficient mining and detection of sequential intrusion patterns for network intrusion detection systems. In: Yu, P.S., Tsai, J.J.P. (eds.) Machine Learning in Cyber Trust, pp. 133–154. Springer, Boston (2009)CrossRef
5.
Zurück zum Zitat Sisiaridis, D., Carcillo, F., Markowitch, O.: A framework for threat detection in communication systems. In: Proceedings of the 20th Pan-Hellenic Conference on Informatics, pp. 68:1–68:6. ACM (2016) Sisiaridis, D., Carcillo, F., Markowitch, O.: A framework for threat detection in communication systems. In: Proceedings of the 20th Pan-Hellenic Conference on Informatics, pp. 68:1–68:6. ACM (2016)
6.
Zurück zum Zitat Sisiaridis, D., Kuchta, V., Markowitch, O.: A categorical approach in handling event-ordering in distributed systems. In: Parallel and Distributed Systems (ICPADS), pp. 1145–1150. IEEE (2016) Sisiaridis, D., Kuchta, V., Markowitch, O.: A categorical approach in handling event-ordering in distributed systems. In: Parallel and Distributed Systems (ICPADS), pp. 1145–1150. IEEE (2016)
Metadaten
Titel
Feature Extraction in Security Analytics: Reducing Data Complexity with Apache Spark
verfasst von
Dimitrios Sisiaridis
Olivier Markowitch
Copyright-Jahr
2018
DOI
https://doi.org/10.1007/978-3-319-76451-1_29

Premium Partner