Top

Published in:

2023 | OriginalPaper | Chapter

R2-AD2: Detecting Anomalies by Analysing the Raw Gradient

Authors : Jan-Philipp Schulze, Philip Sperl, Ana Răduțoiu, Carla Sagebiel, Konstantin Böttinger

Published in: Machine Learning and Knowledge Discovery in Databases

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Neural networks follow a gradient-based learning scheme, adapting their mapping parameters by back-propagating the output loss. Samples unlike the ones seen during training cause a different gradient distribution. Based on this intuition, we design a novel semi-supervised anomaly detection method called R2-AD2. By analysing the temporal distribution of the gradient over multiple training steps, we reliably detect point anomalies in strict semi-supervised settings. Instead of domain dependent features, we input the raw gradient caused by the sample under test to an end-to-end recurrent neural network architecture. R2-AD2 works in a purely data-driven way, thus is readily applicable in a variety of important use cases of anomaly detection.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter ARES: Locally Adaptive Reconstruction-Based Anomaly Scoring

next chapter Hop-Count Based Self-supervised Anomaly Detection on Attributed Networks

https://github.com/Fraunhofer-AISEC/R2-AD2.

Akcay, S., Atapour-Abarghouei, A., Breckon, T.P.: GANomaly: semi-supervised anomaly detection via adversarial training. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11363, pp. 622–637. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20893-6_39CrossRef

Beggel, L., Pfeiffer, M., Bischl, B.: Robust anomaly detection in images using adversarial autoencoders. In: Brefeld, U., Fromont, E., Hotho, A., Knobbe, A., Maathuis, M., Robardet, C. (eds.) ECML PKDD 2019. LNCS (LNAI), vol. 11906, pp. 206–222. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-46150-8_13CrossRef

Bergman, L., Hoshen, Y.: Classification-based anomaly detection for general data. In: International Conference on Learning Representations (2020). https://openreview.net/forum?id=H1lK_lBtvS

Blackard, J.A., Dean, D.J.: Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables. Comput. Electron. Agric. 24(3), 131–151 (1999). https://doi.org/10.1016/S0168-1699(99)00046-0CrossRef

Borghesi, A., Bartolini, A., Lombardi, M., Milano, M., Benini, L.: Anomaly detection using autoencoders in high performance computing systems. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 9428–9433 (2019). https://doi.org/10.1609/aaai.v33i01.33019428

Chalapathy, R., Menon, A.K., Chawla, S.: Anomaly detection using one-class neural networks. arXiv:1802.06360 [cs, stat] (2019). http://arxiv.org/abs/1802.06360, arXiv: 1802.06360

Dhaliwal, J., Shintre, S.: Gradient similarity: an explainable approach to detect adversarial attacks against deep learning. arXiv:1806.10707 [cs] (2018). http://arxiv.org/abs/1806.10707, arXiv: 1806.10707

Golan, I., El-Yaniv, R.: Deep anomaly detection using geometric transformations. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 31. Curran Associates, Inc. (2018), https://proceedings.neurips.cc/paper/2018/file/5e62d03aec0d17facfc5355dd90d441c-Paper.pdf

Goyal, S., Raghunathan, A., Jain, M., Simhadri, H.V., Jain, P.: DROCC: deep robust one-class classification. In: Proceedings of the 37th International Conference on Machine Learning, pp. 3711–3721. PMLR (2020).https://proceedings.mlr.press/v119/goyal20c.html, iSSN: 2640-3498

10.

Habibi Lashkari, A., Kaur, G., Rahali, A.: DIDarknet: a contemporary approach to detect and characterize the darknet traffic using deep image learning. In: 2020 the 10th International Conference on Communication and Network Security, ICCNS 2020, pp. 1–13. Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3442520.3442521

11.

Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. In: International Conference on Learning Representations (2017). https://openreview.net/forum?id=Hkg4TI9xl

12.

Hendrycks, D., Mazeika, M., Dietterich, T.: Deep anomaly detection with outlier exposure. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=HyxCxhRcY7

13.

Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735CrossRef

14.

Huang, C., Ye, F., Zhao, P., Zhang, Y., Wang, Y.F., Tian, Q.: ESAD: end-to-end deep semi-supervised anomaly detection. In: The 32nd British Machine Vision Conference (2021). https://www.bmvc2021-virtualconference.com/conference/papers/paper_0329.html

15.

Huang, R., Geng, A., Li, Y.: On the importance of gradients for detecting distributional shifts in the wild. arXiv:2110.00218 [cs] (2021). http://arxiv.org/abs/2110.00218, arXiv: 2110.00218

16.

Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on Machine Learning, pp. 448–456. PMLR (2015). https://proceedings.mlr.press/v37/ioffe15.html, iSSN: 1938-7228

17.

Kwon, G., Prabhushankar, M., Temel, D., AlRegib, G.: Backpropagated gradient representations for anomaly detection. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12366, pp. 206–226. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58589-1_13CrossRef

18.

Kwon, G., Prabhushankar, M., Temel, D., AlRegib, G.: Novelty detection through model-based characterization of neural networks. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 3179–3183 (2020). https://doi.org/10.1109/ICIP40778.2020.9190706, iSSN: 2381-8549

19.

Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998). https://doi.org/10.1109/5.726791CrossRef

20.

Lee, J., AlRegib, G.: Open-set recognition with gradient-based representations. In: 2021 IEEE International Conference on Image Processing (ICIP), pp. 469–473 (2021). https://doi.org/10.1109/ICIP42928.2021.9506430, iSSN: 2381-8549

21.

Lee, J., Prabhushankar, M., AlRegib, G.: Gradient-based adversarial and out-of-distribution detection. In: International Conference on Machine Learning (ICML) Workshop on New Frontiers in Adversarial Machine Learning (2022)

22.

Li, D., Chen, D., Jin, B., Shi, L., Goh, J., Ng, S.-K.: MAD-GAN: multivariate anomaly detection for time series data with generative adversarial networks. In: Tetko, I.V., Kůrková, V., Karpov, P., Theis, F. (eds.) ICANN 2019. LNCS, vol. 11730, pp. 703–716. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30490-4_56CrossRef

23.

Lust, J., Condurache, A.P.: GraN: an efficient gradient-norm based detector for adversarial and misclassified examples. In: ESANN 2020, p. 6 (2020)

24.

Mamun, M.S.I., Rathore, M.A., Lashkari, A.H., Stakhanova, N., Ghorbani, A.A.: Detecting malicious URLs using lexical analysis. In: Chen, J., Piuri, V., Su, C., Yung, M. (eds.) NSS 2016. LNCS, vol. 9955, pp. 467–482. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46298-1_30CrossRef

25.

MontazeriShatoori, M., Davidson, L., Kaur, G., Lashkari, A.H.: Detection of DoH tunnels using time-series classification of encrypted traffic. In: The 5th IEEE Cyber Science and Technology Congress, pp. 63–70 (2020). https://doi.org/10.1109/DASC-PICom-CBDCom-CyberSciTech49142.2020.00026

26.

Pang, G., Cao, L., Chen, L., Liu, H.: Learning Representations of ultrahigh-dimensional data for random distance-based outlier detection. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, pp. 2041–2050. Association for Computing Machinery, New York (2018). https://doi.org/10.1145/3219819.3220042

27.

Pang, G., Shen, C., Cao, L., Hengel, A.V.D.: Deep learning for anomaly detection: a review. ACM Comput. Surv. 54(2), 38:1–38:38 (2021). https://doi.org/10.1145/3439950

28.

Pang, G., Shen, C., van den Hengel, A.: Deep anomaly detection with deviation networks. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, pp. 353–362. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3292500.3330871

29.

Pozzolo, A.D., Caelen, O., Johnson, R.A., Bontempi, G.: Calibrating probability with undersampling for unbalanced classification. In: 2015 IEEE Symposium Series on Computational Intelligence, pp. 159–166 (2015). https://doi.org/10.1109/SSCI.2015.33

30.

Ruff, L., et al.: A unifying review of deep and shallow anomaly detection. In: Proceedings of the IEEE, pp. 1–40 (2021). https://doi.org/10.1109/JPROC.2021.3052449

31.

Ruff, L., et al.: Deep semi-supervised anomaly detection. In: International Conference on Learning Representations (2020). https://openreview.net/forum?id=HkgH0TEYwH

32.

Salehi, M., Mirzaei, H., Hendrycks, D., Li, Y., Rohban, M.H., Sabokrou, M.: A unified survey on anomaly, novelty, open-set, and out-of-distribution detection: solutions and future challenges. arXiv:2110.14051 [cs] (2021). http://arxiv.org/abs/2110.14051, arXiv: 2110.14051

33.

Schlegl, T., Seeböck, P., Waldstein, S.M., Langs, G., Schmidt-Erfurth, U.: f-AnoGAN: fast unsupervised anomaly detection with generative adversarial networks. Med. Image Anal. 54, 30–44 (2019). https://doi.org/10.1016/j.media.2019.01.010CrossRef

34.

Schulze, J.-P., Sperl, P., Böttinger, K.: DA3G: detecting adversarial attacks by analysing gradients. In: Bertino, E., Shulman, H., Waidner, M. (eds.) ESORICS 2021. LNCS, vol. 12972, pp. 563–583. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88418-5_27CrossRef

35.

Sharafaldin, I., Lashkari, A.H., Ghorbani, A.A.: Toward generating a new intrusion detection dataset and intrusion traffic characterization. In: ICISSP, pp. 108–116 (2018)

36.

Sohn, K., Li, C.L., Yoon, J., Jin, M., Pfister, T.: Learning and evaluating representations for deep one-class classification. In: International Conference on Learning Representations (2021). https://openreview.net/forum?id=HCSgyPUfeDj

37.

Sperl, P., Schulze, J.-P., Böttinger, K.: Activation anomaly analysis. In: Hutter, F., Kersting, K., Lijffijt, J., Valera, I. (eds.) ECML PKDD 2020. LNCS (LNAI), vol. 12458, pp. 69–84. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67661-2_5CrossRef

38.

Sun, J., et al.: Gradient-based novelty detection boosted by self-supervised binary classification. arXiv:2112.09815 [cs] (2021). http://arxiv.org/abs/2112.09815, arXiv: 2112.09815

39.

Tavallaee, M., Bagheri, E., Lu, W., Ghorbani, A.A.: A detailed analysis of the KDD CUP 99 data set. In: 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, pp. 1–6 (2009). https://doi.org/10.1109/CISDA.2009.5356528, iSSN: 2329-6275

40.

Vu, H.S., Ueta, D., Hashimoto, K., Maeno, K., Pranata, S., Shen, S.M.: Anomaly detection with adversarial dual autoencoders. arXiv:1902.06924 [cs] (2019). http://arxiv.org/abs/1902.06924, arXiv: 1902.06924

41.

Wilcoxon, F.: Individual comparisons by ranking methods. In: Kotz, S., Johnson, N.L. (eds.) Breakthroughs in Statistics: Methodology and Distribution, Springer Series in Statistics, pp. 196–202. Springer, New York (1992). https://doi.org/10.1007/978-1-4612-4380-9_16

42.

Woods, K.S.: Comparative evaluation of pattern recognition techniques for detection of microcalcifications in mammography. Int. J. Pattern Recogn. Artif. Intell. 07(06), 1417–1436 (1993). https://doi.org/10.1142/S0218001493000698CrossRef

43.

Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv:1708.07747 [cs, stat] (2017). http://arxiv.org/abs/1708.07747, arXiv: 1708.07747

44.

Yamanaka, Y., Iwata, T., Takahashi, H., Yamada, M., Kanai, S.: Autoencoding binary classifiers for supervised anomaly detection. In: Nayak, A.C., Sharma, A. (eds.) PRICAI 2019. LNCS (LNAI), vol. 11671, pp. 647–659. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-29911-8_50CrossRef

45.

Ye, Z., Chen, Y., Zheng, H.: Understanding the effect of bias in deep anomaly detection. In: Zhou, Z.H. (ed.) Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, International Joint Conferences on Artificial Intelligence Organization , IJCAI-2021, pp. 3314–3320 (2021). https://doi.org/10.24963/ijcai.2021/456

Title: R2-AD2: Detecting Anomalies by Analysing the Raw Gradient
Authors: Jan-Philipp Schulze
Philip Sperl
Ana Răduțoiu
Carla Sagebiel
Konstantin Böttinger
Publisher: Springer International Publishing
Book: Machine Learning and Knowledge Discovery in Databases
Print ISBN: 978-3-031-26386-6

Electronic ISBN: 978-3-031-26387-3

Copyright Year: 2023
DOI: https://doi.org/10.1007/978-3-031-26387-3_13

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner