Skip to main content
Top
Published in: International Journal of Information Security 5/2017

20-07-2016 | Regular Contribution

Detecting zero-day attacks using context-aware anomaly detection at the application-layer

Authors: Patrick Duessel, Christian Gehl, Ulrich Flegel, Sven Dietrich, Michael Meier

Published in: International Journal of Information Security | Issue 5/2017

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Anomaly detection allows for the identification of unknown and novel attacks in network traffic. However, current approaches for anomaly detection of network packet payloads are limited to the analysis of plain byte sequences. Experiments have shown that application-layer attacks become difficult to detect in the presence of attack obfuscation using payload customization. The ability to incorporate syntactic context into anomaly detection provides valuable information and increases detection accuracy. In this contribution, we address the issue of incorporating protocol context into payload-based anomaly detection. We present a new data representation, called \({c}_n\)-grams, that allows to integrate syntactic and sequential features of payloads in an unified feature space and provides the basis for context-aware detection of network intrusions. We conduct experiments on both text-based and binary application-layer protocols which demonstrate superior accuracy on the detection of various types of attacks over regular anomaly detection methods. Furthermore, we show how \({c}_n\)-grams can be used to interpret detected anomalies and thus, provide explainable decisions in practice.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Appendix
Available only for authorised users
Literature
1.
go back to reference Borisov, N., Brumley, D.J., Wang, H., Dunagan, J., Joshi, P., Guo, C.: Generic application-level protocol analyzer and its language. In: Proceedings of Network and Distributed System Security Symposium (NDSS) (2007) Borisov, N., Brumley, D.J., Wang, H., Dunagan, J., Joshi, P., Guo, C.: Generic application-level protocol analyzer and its language. In: Proceedings of Network and Distributed System Security Symposium (NDSS) (2007)
2.
go back to reference Cretu, G., Stavrou, A., Locasto, M., Stolfo, S.J., Keromytis, A.D.: Casting out demons: sanitizing training data for anomaly sensors. In: ieeesp (2008) Cretu, G., Stavrou, A., Locasto, M., Stolfo, S.J., Keromytis, A.D.: Casting out demons: sanitizing training data for anomaly sensors. In: ieeesp (2008)
3.
go back to reference Cui, W., Kannan, J., Wang. H.J.: Discoverer: automatic protocol reverse engineering from network traces. In: SS’07: Proceedings of 16th USENIX Security Symposium on USENIX Security Symposium, pp. 1–14 (2007) Cui, W., Kannan, J., Wang. H.J.: Discoverer: automatic protocol reverse engineering from network traces. In: SS’07: Proceedings of 16th USENIX Security Symposium on USENIX Security Symposium, pp. 1–14 (2007)
4.
go back to reference Detristan, T., Ulenspiegel, T., Malcom, Y., Underduk, M.: Polymorphic shellcode engine using spectrum analysis. Phrack. 11(61) (2003) Detristan, T., Ulenspiegel, T., Malcom, Y., Underduk, M.: Polymorphic shellcode engine using spectrum analysis. Phrack. 11(61) (2003)
5.
go back to reference Düssel, P., Gehl, C., Laskov, P., Rieck, K.: Incorporation of application layer protocol syntax into anomaly detection. In: ICISS, pp. 188–202 (2008) Düssel, P., Gehl, C., Laskov, P., Rieck, K.: Incorporation of application layer protocol syntax into anomaly detection. In: ICISS, pp. 188–202 (2008)
6.
go back to reference Folga, P., Sharif, M., Perdisci, R., Kolesnikov, O., Lee, W.: Polymorphic blending attacks. In: Proceedings of USENIX Security Symposium, pp. 241–256 (2006) Folga, P., Sharif, M., Perdisci, R., Kolesnikov, O., Lee, W.: Polymorphic blending attacks. In: Proceedings of USENIX Security Symposium, pp. 241–256 (2006)
7.
go back to reference Forrest, S., Hofmeyr, S.A., Somayaji, A., Longstaff, T.A.: A sense of self for unix processes. In: Proceedings of IEEE Symposium on Security and Privacy, pp. 120–128. Oakland (1996) Forrest, S., Hofmeyr, S.A., Somayaji, A., Longstaff, T.A.: A sense of self for unix processes. In: Proceedings of IEEE Symposium on Security and Privacy, pp. 120–128. Oakland (1996)
8.
go back to reference Gao, D., Reiter, M.K., Song, D.: Behavioral distance measurement using hidden markov models. In: Recent Adances in Intrusion Detection (RAID), pp. 19–40 (2006) Gao, D., Reiter, M.K., Song, D.: Behavioral distance measurement using hidden markov models. In: Recent Adances in Intrusion Detection (RAID), pp. 19–40 (2006)
9.
go back to reference Ingham, K.L., Somayaji, A., Burge, J., Forrest, S.: Learning DFA representations of HTTP for protecting web applications. Comput. Netw. 51(5), 1239–1255 (2007)CrossRefMATH Ingham, K.L., Somayaji, A., Burge, J., Forrest, S.: Learning DFA representations of HTTP for protecting web applications. Comput. Netw. 51(5), 1239–1255 (2007)CrossRefMATH
10.
go back to reference Kloft, M., Laskov, P.: Security analysis of online centroid anomaly detection. Technical report UCB/EECS-2010-22. EECS Department, University of California, Berkeley (2010) Kloft, M., Laskov, P.: Security analysis of online centroid anomaly detection. Technical report UCB/EECS-2010-22. EECS Department, University of California, Berkeley (2010)
11.
go back to reference Kolesnikov, O., Dagon, D., Lee, W.: Advanced polymorphic worms: evading IDS by blending with normal traffic. In: Proceedings of USENIX Security Symposium (2004) Kolesnikov, O., Dagon, D., Lee, W.: Advanced polymorphic worms: evading IDS by blending with normal traffic. In: Proceedings of USENIX Security Symposium (2004)
12.
go back to reference Kruegel, C., Vigna, G.: Anomaly detection of web-based attacks. In: Proceedings of 10th ACM Conference on Computer and Communications Security, pp. 251–261 (2003) Kruegel, C., Vigna, G.: Anomaly detection of web-based attacks. In: Proceedings of 10th ACM Conference on Computer and Communications Security, pp. 251–261 (2003)
13.
go back to reference Kruegel, C., Toth, T., Kirda, E.: Service specific anomaly detection for network intrusion detection. In: Proceedings of ACM Symposium on Applied, Computing, pp. 201–208 (2002) Kruegel, C., Toth, T., Kirda, E.: Service specific anomaly detection for network intrusion detection. In: Proceedings of ACM Symposium on Applied, Computing, pp. 201–208 (2002)
14.
go back to reference Kruegel, C., Vigna, G., Robertson, W.: A multi-model approach to the detection of web-based attacks. Comput. Netw. 48(5), 717–738 (2005)CrossRef Kruegel, C., Vigna, G., Robertson, W.: A multi-model approach to the detection of web-based attacks. Comput. Netw. 48(5), 717–738 (2005)CrossRef
15.
go back to reference Lee, W., Stolfo, S.J.: A framework for constructing features and models for intrusion detection systems. ACM Trans. Inf. Syst. Secur. 3, 227–261 (2000)CrossRef Lee, W., Stolfo, S.J.: A framework for constructing features and models for intrusion detection systems. ACM Trans. Inf. Syst. Secur. 3, 227–261 (2000)CrossRef
16.
go back to reference Lodhi, H., Saunders, C., Shawe-Taylor, J., Cristianini, N., Watkins, C.: Text classification using string kernels. J. Mach. Learn. Res. 2, 419–444 (2002)MATH Lodhi, H., Saunders, C., Shawe-Taylor, J., Cristianini, N., Watkins, C.: Text classification using string kernels. J. Mach. Learn. Res. 2, 419–444 (2002)MATH
17.
go back to reference Mahoney, M.V., Chan, P.K.: PHAD: packet header anomaly detection for identifying hostile network traffic. Technical Report CS-2001-2, Florida Institute of Technology (2001) Mahoney, M.V., Chan, P.K.: PHAD: packet header anomaly detection for identifying hostile network traffic. Technical Report CS-2001-2, Florida Institute of Technology (2001)
18.
go back to reference Mahoney, M.V., Chan, P.K.: Learning nonstationary models of normal network traffic for detecting novel attacks. In: Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pp. 376–385 (2002) Mahoney, M.V., Chan, P.K.: Learning nonstationary models of normal network traffic for detecting novel attacks. In: Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pp. 376–385 (2002)
19.
go back to reference Mahoney, M.V., Chan, P.K.: An analysis of the 1999 DARPA/Lincoln Laboratory evaluation data for network anomaly detection. In: Giovanni V, Kruegel, Christopher, Erland J (eds) Recent Advances in Intrusion Detection. Springer, Berlin, Heidelberg, pp. 220–237 (2003) Mahoney, M.V., Chan, P.K.: An analysis of the 1999 DARPA/Lincoln Laboratory evaluation data for network anomaly detection. In: Giovanni V, Kruegel, Christopher, Erland J (eds) Recent Advances in Intrusion Detection. Springer, Berlin, Heidelberg, pp. 220–237 (2003)
20.
go back to reference Müller, K.-R., Mika, S., Rätsch, G., Tsuda, K., Schölkopf, B.: An introduction to kernel-based learning algorithms. IEEE Neural Netw. 12(2), 181–201 (2001)CrossRef Müller, K.-R., Mika, S., Rätsch, G., Tsuda, K., Schölkopf, B.: An introduction to kernel-based learning algorithms. IEEE Neural Netw. 12(2), 181–201 (2001)CrossRef
21.
go back to reference Pang, R., Paxson, V., Sommer, R., Peterson, L.L.: binpac: A yacc for writing application protocol parsers. In: Proceedings of ACM Internet Measurement Conference, pp. 289–300 (2006) Pang, R., Paxson, V., Sommer, R., Peterson, L.L.: binpac: A yacc for writing application protocol parsers. In: Proceedings of ACM Internet Measurement Conference, pp. 289–300 (2006)
22.
go back to reference Paxson, V.: Bro: a system for detecting network intruders in real-time. In: Proceedings of USENIX Security Symposium, pp. 31–51 (1998) Paxson, V.: Bro: a system for detecting network intruders in real-time. In: Proceedings of USENIX Security Symposium, pp. 31–51 (1998)
23.
go back to reference Paxson, V.: The bro 0.8 user manual. Lawrence Berkeley National Laboratory and ICSI Center for Internet Research (2004) Paxson, V.: The bro 0.8 user manual. Lawrence Berkeley National Laboratory and ICSI Center for Internet Research (2004)
24.
go back to reference Perdisci, R., Ariu, D., Fogla, P., Giacinto, G., Lee, W.: McPAD: a multiple classifier system for accurate payload-based anomaly detection. Comput. Netw. 53(6), 864–881 (2009) Perdisci, R., Ariu, D., Fogla, P., Giacinto, G., Lee, W.: McPAD: a multiple classifier system for accurate payload-based anomaly detection. Comput. Netw. 53(6), 864–881 (2009)
25.
go back to reference Rieck, K., Laskov, P.: Detecting unknown network attacks using language models. In: Detection of Intrusions and Malware, and Vulnerability Assessment, Proceedings of 3rd DIMVA Conference, LNCS, pp. 74–90 (2006) Rieck, K., Laskov, P.: Detecting unknown network attacks using language models. In: Detection of Intrusions and Malware, and Vulnerability Assessment, Proceedings of 3rd DIMVA Conference, LNCS, pp. 74–90 (2006)
26.
go back to reference Rieck, K., Laskov, P.: Language models for detection of unknown attacks in network traffic. J. Comput. Virol. 2(4), 243–256 (2007) Rieck, K., Laskov, P.: Language models for detection of unknown attacks in network traffic. J. Comput. Virol. 2(4), 243–256 (2007)
27.
go back to reference Rieck, K., Laskov, P.: Visualization and explanation of payload-based anomaly detection. In: Proceedings of European Conference on Computer Network Defense (EC2ND) (2009) Rieck, K., Laskov, P.: Visualization and explanation of payload-based anomaly detection. In: Proceedings of European Conference on Computer Network Defense (EC2ND) (2009)
28.
go back to reference Roesch, M.: Snort: lightweight intrusion detection for networks. In: Proceedings of USENIX Large Installation System Administration Conference LISA, pp. 229–238 (1999) Roesch, M.: Snort: lightweight intrusion detection for networks. In: Proceedings of USENIX Large Installation System Administration Conference LISA, pp. 229–238 (1999)
29.
go back to reference Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)CrossRefMATH Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)CrossRefMATH
30.
go back to reference Song, Y., Keromytis, A.D., Stolfo, S.J.: Spectrogram: a mixture-of-markov-chains model for anomaly detection in web traffic. In: Proceedings of Network and Distributed System Security Symposium (NDSS) (2009) Song, Y., Keromytis, A.D., Stolfo, S.J.: Spectrogram: a mixture-of-markov-chains model for anomaly detection in web traffic. In: Proceedings of Network and Distributed System Security Symposium (NDSS) (2009)
31.
go back to reference Tax, D., Duin, R.: Data domain description by support vectors. In: Verleysen, M. (ed.) Proceedings ESANN, pp. 251–256. D. Facto Press, Brussels (1999) Tax, D., Duin, R.: Data domain description by support vectors. In: Verleysen, M. (ed.) Proceedings ESANN, pp. 251–256. D. Facto Press, Brussels (1999)
32.
go back to reference Vishwanathan, S.V.N., Smola, A.J.: Fast kernels for string and tree matching. In: Tsuda, K., Schölkopf, B., Vert, J.F. (eds.) Kernels and Bioinformatics, pp. 113–130. MIT Press, Cambridge (2004) Vishwanathan, S.V.N., Smola, A.J.: Fast kernels for string and tree matching. In: Tsuda, K., Schölkopf, B., Vert, J.F. (eds.) Kernels and Bioinformatics, pp. 113–130. MIT Press, Cambridge (2004)
33.
go back to reference Wang, K., Stolfo, S.J.: Anomalous payload-based network intrusion detection. In: Recent Adances in Intrusion Detection (RAID), pp. 203–222 (2004) Wang, K., Stolfo, S.J.: Anomalous payload-based network intrusion detection. In: Recent Adances in Intrusion Detection (RAID), pp. 203–222 (2004)
34.
go back to reference Wang, K., Parekh, J.J., Stolfo, S.J.: Anagram: a content anomaly detector resistant to mimicry attack. In: Recent Adances in Intrusion Detection (RAID), pp. 226–248 (2006) Wang, K., Parekh, J.J., Stolfo, S.J.: Anagram: a content anomaly detector resistant to mimicry attack. In: Recent Adances in Intrusion Detection (RAID), pp. 226–248 (2006)
36.
go back to reference Wondracek, G., Milani, C.P., Kruegel, C., Kirda, E.: Automatic network protocol analysis. In: 15th Symposium on Network and Distributed System Security (NDSS) (2008) Wondracek, G., Milani, C.P., Kruegel, C., Kirda, E.: Automatic network protocol analysis. In: 15th Symposium on Network and Distributed System Security (NDSS) (2008)
Metadata
Title
Detecting zero-day attacks using context-aware anomaly detection at the application-layer
Authors
Patrick Duessel
Christian Gehl
Ulrich Flegel
Sven Dietrich
Michael Meier
Publication date
20-07-2016
Publisher
Springer Berlin Heidelberg
Published in
International Journal of Information Security / Issue 5/2017
Print ISSN: 1615-5262
Electronic ISSN: 1615-5270
DOI
https://doi.org/10.1007/s10207-016-0344-y

Other articles of this Issue 5/2017

International Journal of Information Security 5/2017 Go to the issue

Premium Partner