Skip to main content
main-content
Top

Hint

Swipe to navigate through the chapters of this book

2021 | OriginalPaper | Chapter

Supervised Autoencoder Variants for End to End Anomaly Detection

Authors: Max Lübbering, Michael Gebauer, Rajkumar Ramamurthy, Rafet Sifa, Christian Bauckhage

Published in: Pattern Recognition. ICPR International Workshops and Challenges

Publisher: Springer International Publishing

share
SHARE

Abstract

Despite the success of deep learning in various domains such as natural language processing, speech recognition, and computer vision, learning from a limited amount of samples and generalizing to unseen data still pose challenges. Notably, in the tasks of outlier detection and imbalanced dataset classification, the label of interest is either scarce or its distribution is skewed, causing aggravated generalization problems. In this work, we pursue the direction of multi-task learning, specifically the idea of using supervised autoencoders (SAE), which allows us to combine unsupervised and supervised objectives in an end to end fashion. We extend this approach by introducing an adversarial supervised objective to enrich the representations which are learned for the classification task. We conduct thorough experiments on a broad range of tasks, including outlier detection, novelty detection, and imbalanced classification, and study the efficacy of our method against standard baselines using autoencoders. Our work empirically shows that the SAE methods outperform one class autoencoders, adversarially trained autoencoders and multi layer perceptrons in terms of AUPR score comparison. Additionally, our analysis of the obtained representations suggests that the adversarial reconstruction loss functions enforce the encodings to separate into class-specific clusters, which was not observed for non-adversarial reconstruction loss functions.

To get access to this content you need the following product:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 69.000 Bücher
  • über 500 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Testen Sie jetzt 15 Tage kostenlos.

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 50.000 Bücher
  • über 380 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




Testen Sie jetzt 15 Tage kostenlos.

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 58.000 Bücher
  • über 300 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Testen Sie jetzt 15 Tage kostenlos.

Literature
1.
4.
go back to reference Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends® Mach. Learn. 3(1) (2011) Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends® Mach. Learn. 3(1) (2011)
5.
go back to reference Buda, M., Maki, A., Mazurowski, M.A.: A systematic study of the class imbalance problem in CNNs. Neural Netw. 106, 249–259 (2018) CrossRef Buda, M., Maki, A., Mazurowski, M.A.: A systematic study of the class imbalance problem in CNNs. Neural Netw. 106, 249–259 (2018) CrossRef
6.
go back to reference Cardie, C., Howe, N.: Improving minority class prediction using case-specific feature weights (1997) Cardie, C., Howe, N.: Improving minority class prediction using case-specific feature weights (1997)
9.
go back to reference Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. 41(3), 1–58 (2009) CrossRef Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. 41(3), 1–58 (2009) CrossRef
10.
go back to reference Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321321–357 (2002) CrossRef Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321321–357 (2002) CrossRef
11.
go back to reference Chen, J., Sathe, S., Aggarwal, C., Turaga, D.: Outlier detection with autoencoder ensembles. In: Proceedings of the SIAM International Conference on Data Mining (2017) Chen, J., Sathe, S., Aggarwal, C., Turaga, D.: Outlier detection with autoencoder ensembles. In: Proceedings of the SIAM International Conference on Data Mining (2017)
12.
go back to reference Chong, Y.S., Tay, Y.H.: Abnormal event detection in videos using spatiotemporal autoencoder. In: Proceedings of the International Symposium on Neural Networks (2017) Chong, Y.S., Tay, Y.H.: Abnormal event detection in videos using spatiotemporal autoencoder. In: Proceedings of the International Symposium on Neural Networks (2017)
14.
go back to reference Davis, J., Goadrich, M.: The relationship between precision-recall and ROC curves. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 233–240 (2006) Davis, J., Goadrich, M.: The relationship between precision-recall and ROC curves. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 233–240 (2006)
15.
go back to reference Divekar, A., Parekh, M., Savla, V., Mishra, R., Shirole, M.: Benchmarking datasets for anomaly-based network intrusion detection: KDD CUP 99 alternatives. In: Proceedings of 3rd International Conference on Computing, Communication and Security (ICCCS) (2018) Divekar, A., Parekh, M., Savla, V., Mishra, R., Shirole, M.: Benchmarking datasets for anomaly-based network intrusion detection: KDD CUP 99 alternatives. In: Proceedings of 3rd International Conference on Computing, Communication and Security (ICCCS) (2018)
17.
go back to reference Gao, S., Zhang, Y., Jia, K., Lu, J., Zhang, Y.: Single sample face recognition via learning deep supervised autoencoders. IEEE Trans. Inf. Forensics Secur. 10(10), 2108–2118 (2015) CrossRef Gao, S., Zhang, Y., Jia, K., Lu, J., Zhang, Y.: Single sample face recognition via learning deep supervised autoencoders. IEEE Trans. Inf. Forensics Secur. 10(10), 2108–2118 (2015) CrossRef
18.
go back to reference Gogoi, P., Borah, B., Bhattacharyya, D., Kalita, J.: Outlier identification using symmetric neighborhoods. Procedia Technol. 6, 239–246 (2012) CrossRef Gogoi, P., Borah, B., Bhattacharyya, D., Kalita, J.: Outlier identification using symmetric neighborhoods. Procedia Technol. 6, 239–246 (2012) CrossRef
20.
go back to reference Hautamaki, V., Karkkainen, I., Franti, P.: Outlier detection using k-nearest neighbour graph. In: Proceedings of the 17th International Conference on Pattern Recognition, vol. 3 (2004) Hautamaki, V., Karkkainen, I., Franti, P.: Outlier detection using k-nearest neighbour graph. In: Proceedings of the 17th International Conference on Pattern Recognition, vol. 3 (2004)
23.
go back to reference Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. In: Proceedings of International Conference on Learning Representations (2017) Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. In: Proceedings of International Conference on Learning Representations (2017)
24.
go back to reference Huang, C., Li, Y., Change Loy, C., Tang, X.: Learning deep representation for imbalanced classification. In: Proceedings of Conference on Computer Vision and Pattern Recognition (2016) Huang, C., Li, Y., Change Loy, C., Tang, X.: Learning deep representation for imbalanced classification. In: Proceedings of Conference on Computer Vision and Pattern Recognition (2016)
25.
go back to reference Ishii, Y., Takanashi, M.: Low-cost unsupervised outlier detection by autoencoders with robust estimation. J. Inf. Process. 27, 335–339 (2019) Ishii, Y., Takanashi, M.: Low-cost unsupervised outlier detection by autoencoders with robust estimation. J. Inf. Process. 27, 335–339 (2019)
26.
go back to reference Japkowicz, N., Myers, C., Gluck, M., et al.: A novelty detection approach to classification. In: Proceedings of the International Joint Conference on Artificial Intelligence (1995) Japkowicz, N., Myers, C., Gluck, M., et al.: A novelty detection approach to classification. In: Proceedings of the International Joint Conference on Artificial Intelligence (1995)
28.
go back to reference Kannan, R., Woo, H., Aggarwal, C.C., Park, H.: Outlier detection for text data. In: Proceedings of the International Conference on Data Mining (2017) Kannan, R., Woo, H., Aggarwal, C.C., Park, H.: Outlier detection for text data. In: Proceedings of the International Conference on Data Mining (2017)
30.
go back to reference Kubat, M., Holte, R.C., Matwin, S.: Machine learning for the detection of oil spills in satellite radar images. Machine Learn. 30, 195–215 (1998) CrossRef Kubat, M., Holte, R.C., Matwin, S.: Machine learning for the detection of oil spills in satellite radar images. Machine Learn. 30, 195–215 (1998) CrossRef
31.
go back to reference Kukar, M., Kononenko, I., et al.: Cost-sensitive learning with neural networks. In: Proceedings of European Conference on Artificial Intelligence (1998) Kukar, M., Kononenko, I., et al.: Cost-sensitive learning with neural networks. In: Proceedings of European Conference on Artificial Intelligence (1998)
32.
go back to reference Lawrence, S., Burns, I., Back, A., Tsoi, A.C., Giles, C.L.: Neural network classification and prior class probabilities. In: Neural Networks: Tricks of the Trade (1998) Lawrence, S., Burns, I., Back, A., Tsoi, A.C., Giles, C.L.: Neural network classification and prior class probabilities. In: Neural Networks: Tricks of the Trade (1998)
33.
go back to reference Le, L., Patterson, A., White, M.: Supervised autoencoders: improving generalization performance with unsupervised regularizers. In: Proceedings of Neural Information Processing Systems Le, L., Patterson, A., White, M.: Supervised autoencoders: improving generalization performance with unsupervised regularizers. In: Proceedings of Neural Information Processing Systems
34.
go back to reference Le, L., Patterson, A., White, M.: Supervised autoencoders: improving generalization performance with unsupervised regularizers. In: Advances in Neural Information Processing Systems (2018) Le, L., Patterson, A., White, M.: Supervised autoencoders: improving generalization performance with unsupervised regularizers. In: Advances in Neural Information Processing Systems (2018)
36.
go back to reference Liu, T., Tao, D., Song, M., Maybank, S.J.: Algorithm-dependent generalization bounds for multi-task learning. IEEE Trans. Pattern Analy. Mach. Intell. 39(2), 227–241 (2016) CrossRef Liu, T., Tao, D., Song, M., Maybank, S.J.: Algorithm-dependent generalization bounds for multi-task learning. IEEE Trans. Pattern Analy. Mach. Intell. 39(2), 227–241 (2016) CrossRef
38.
go back to reference Mazurowski, M.A., Habas, P.A., Zurada, J.M., Lo, J.Y., Baker, J.A., Tourassi, G.D.: Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance. Neural Netw. 21, 427–436 (2008) CrossRef Mazurowski, M.A., Habas, P.A., Zurada, J.M., Lo, J.Y., Baker, J.A., Tourassi, G.D.: Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance. Neural Netw. 21, 427–436 (2008) CrossRef
39.
go back to reference McInnes, L., Healy, J., Melville, J.: UMAP: uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:​1802.​03426 (2018) McInnes, L., Healy, J., Melville, J.: UMAP: uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:​1802.​03426 (2018)
40.
go back to reference Neyshabur, B., Bhojanapalli, S., McAllester, D., Srebro, N.: Exploring generalization in deep learning. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 5947–5956 (2017) Neyshabur, B., Bhojanapalli, S., McAllester, D., Srebro, N.: Exploring generalization in deep learning. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 5947–5956 (2017)
41.
go back to reference Olszewski, D.: A probabilistic approach to fraud detection in telecommunications. Knowl.-Based Syst. 26, 246–258 (2012) CrossRef Olszewski, D.: A probabilistic approach to fraud detection in telecommunications. Knowl.-Based Syst. 26, 246–258 (2012) CrossRef
42.
go back to reference Panigrahi, S., Kundu, A., Sural, S., Majumdar, A.K.: Credit card fraud detection: a fusion approach using Dempster-Shafer theory and Bayesian learning. Inf. Fusion 10(4), 354–363 (2009) CrossRef Panigrahi, S., Kundu, A., Sural, S., Majumdar, A.K.: Credit card fraud detection: a fusion approach using Dempster-Shafer theory and Bayesian learning. Inf. Fusion 10(4), 354–363 (2009) CrossRef
43.
go back to reference Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP) (2014) Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP) (2014)
44.
go back to reference Perera, P., Nallapati, R., Xiang, B.: OCGAN: one-class novelty detection using GANs with constrained latent representations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2898–2906 (2019) Perera, P., Nallapati, R., Xiang, B.: OCGAN: one-class novelty detection using GANs with constrained latent representations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2898–2906 (2019)
45.
go back to reference Ranzato, M., Szummer, M.: Semi-supervised learning of compact document representations with deep networks. In: Proceedings of International Conference on Machine learning (2008) Ranzato, M., Szummer, M.: Semi-supervised learning of compact document representations with deep networks. In: Proceedings of International Conference on Machine learning (2008)
46.
go back to reference Sarvari, H., Domeniconi, C., Prenkaj, B., Stilo, G.: Unsupervised boosting-based autoencoder ensembles for outlier detection. arXiv preprint arXiv:​1910.​09754 (2019) Sarvari, H., Domeniconi, C., Prenkaj, B., Stilo, G.: Unsupervised boosting-based autoencoder ensembles for outlier detection. arXiv preprint arXiv:​1910.​09754 (2019)
48.
go back to reference Scholkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Optimization, and Beyond, Regularization (2001) Scholkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Optimization, and Beyond, Regularization (2001)
49.
go back to reference Tavallaee, M., Bagheri, E., Lu, W., Ghorbani, A.A.: A detailed Analysis of the KDD CUP 99 Data Set. In: Proceedings of IEEE Symposium on Computational Intelligence for Security and Defense Applications (2009) Tavallaee, M., Bagheri, E., Lu, W., Ghorbani, A.A.: A detailed Analysis of the KDD CUP 99 Data Set. In: Proceedings of IEEE Symposium on Computational Intelligence for Security and Defense Applications (2009)
50.
go back to reference Vu, H.S., Ueta, D., Hashimoto, K., Maeno, K., Pranata, S., Shen, S.M.: Anomaly detection with adversarial dual autoencoders. arXiv preprint arXiv:​1902.​06924 (2019) Vu, H.S., Ueta, D., Hashimoto, K., Maeno, K., Pranata, S., Shen, S.M.: Anomaly detection with adversarial dual autoencoders. arXiv preprint arXiv:​1902.​06924 (2019)
51.
go back to reference Wang, X., Du, Y., Lin, S., Cui, P., Yang, Y.: Self-adversarial variational autoencoder with gaussian anomaly prior distribution for anomaly detection. CoRR, abs/1903.00904 (2019) Wang, X., Du, Y., Lin, S., Cui, P., Yang, Y.: Self-adversarial variational autoencoder with gaussian anomaly prior distribution for anomaly detection. CoRR, abs/1903.00904 (2019)
53.
go back to reference Yang, Y., Liu, X.: A re-examination of text categorization methods. In: Proceedings of International Conference on Research and development in Information Retrieval (1999) Yang, Y., Liu, X.: A re-examination of text categorization methods. In: Proceedings of International Conference on Research and development in Information Retrieval (1999)
55.
56.
go back to reference Zhou, C., Paffenroth, R.C.: Anomaly detection with robust deep autoencoders. In: Proceedings of the International Conference on Knowledge Discovery and Data Mining (2017) Zhou, C., Paffenroth, R.C.: Anomaly detection with robust deep autoencoders. In: Proceedings of the International Conference on Knowledge Discovery and Data Mining (2017)
Metadata
Title
Supervised Autoencoder Variants for End to End Anomaly Detection
Authors
Max Lübbering
Michael Gebauer
Rajkumar Ramamurthy
Rafet Sifa
Christian Bauckhage
Copyright Year
2021
DOI
https://doi.org/10.1007/978-3-030-68790-8_44

Premium Partner