Abstract
Imbalanced datasets pose severe challenges in training well performing classifiers. This problem is also prevalent in the domain of outlier detection since outliers occur infrequently and are generally treated as minorities. One simple yet powerful approach is to use autoencoders which are trained on majority samples and then to classify samples based on the reconstruction loss. However, this approach fails to classify samples whenever reconstruction errors of minorities overlap with that of majorities. To overcome this limitation, we propose an adversarial loss function that maximizes the loss of minorities while minimizing the loss for majorities. This way, we obtain a well-separated reconstruction error distribution that facilitates classification. We show that this approach is robust in a wide variety of settings, such as imbalanced data classification or outlier- and novelty detection.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aggarwal, C.C.: Outlier Analysis. Data Mining, pp. 237–263. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-14142-8_8
An, J., Cho, S.: Variational autoencoder based anomaly detection using reconstruction probability. Special Lect. IE 2(1), 1–18 (2015)
Laura, B., Michael, P., Bernd, B.: Robust anomaly detection in images using adversarial autoencoders. In: Proceedings of Joint European Conference on Machine Learning and Knowledge Discovery in Databases (2019)
Buda, M., Maki, A., Mazurowski, M.A.: A systematic study of the class imbalance problem in convolutional neural networks. Neural Netw. 106, 249–259 (2018)
Cardie, C., Howe, N.: Improving minority class prediction using case-specific feature weights (1997)
Raghavendra, C., Aditya, K.M., Sanjay, C.: Anomaly Detection using One-Class Neural Networks. arXiv preprint arXiv:1802.06360 (2018)
Varun, C., Arindam, B., Vipin, K.: Anomaly detection: a survey. ACM Comput. Surv. 41(3), 1–58 (2009)
Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)
Jinghui, C., Saket, S., Charu, A., Deepak, T.: Outlier detection with autoencoder ensembles. In: Proceedings of the SIAM International Conference on Data Mining (2017)
Yong, S.C., Yong, H.T.: Abnormal event detection in videos using spatiotemporal autoencoder. In: Proceedings of International Symposium on Neural Network (2017)
Dau, H.A., Ciesielski, V., Song, A.: Anomaly detection using replicator neural networks trained on examples of one class. In: Dick, G., et al. (eds.) SEAL 2014. LNCS, vol. 8886, pp. 311–322. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-13563-2_27
Hoang, A.D., Vic, C., Andy, S.: Anomaly detection using replicator neural networks trained on examples of one class. In: Proceedings of the 10th International Conference on Simulated Evolution and Learning (2014)
Abhishek, D., Meet, P., Vaibhav, S., Rudra, M., Mahesh, S.: Benchmarking datasets for anomaly-based network intrusion detection: KDD CUP 99 alternatives. In: Proceedings of 3rd International Conference on Computing, Communication and Security (ICCCS) (2018)
Haimonti, D., Chris, G., Kirk, B., Hillol, K.: Distributed top-k outlier detection from astronomy catalogs using the demac system. In: Proceedings of the 2007 SIAM International Conference on Data Mining (2007)
Gogoi, P., Borah, B., Bhattacharyya, D.K., Kalita, J.K.: Outlier identification using symmetric neighborhoods. Procedia Technol. 6, 239–246 (2012)
Ian, J.G., Jonathon, S., Christian, S.: Explaining and harnessing Adversarial Examples. arXiv preprint arXiv:1412.6572 (2014)
Ville, H., Ismo, K., Pasi, F.: Outlier detection using k-nearest neighbour graph. In: Proceedings of the 17th International Conference on Pattern Recognition, vol. 3 (2004)
Douglas, M.H.: Identification of Outliers. Springer, Berlin (1980)
Simon, H., Hongxing, H., Graham, W., Rohan, B.: Outlier detection using replicator neural networks. In: Proceedings of International Conference on Data Warehousing and Knowledge Discovery (2002)
Chen, H., Yining, L., Chen, C.L., Xiaoou, T.: Learning deep representation for imbalanced classification. In: Proceedings of Conference on Computer Vision and Pattern Recognition (2016)
Ishii, Y., Takanashi, M.: Low-cost unsupervised outlier detection by autoencoders with robust estimation. J. Inf. Process. 27, 335–339 (2019)
Nathalie, J., Catherine, M., Mark, G., et al.: A novelty detection approach to classification. In: Proceedings of International Joint Conference on Artificial Intelligence (1995)
Japkowicz, N., Stephen, S.: The class imbalance problem: a systematic study. Intell. Data Anal. 6(5), 429–449 (2002)
Thorsten, J.: Text categorization with support vector machines: learning with many relevant features. In: Proceedings of European Conference on Machine Learning (1998)
Ramakrishnan, K., Hyenkyun, W., Charu, C.A., Haesun, P.: Outlier detection for text data. In: Proceedings of International Conference on Data Mining (2017)
Kubat, M., Holte, R.C., Matwin, S.: Machine learning for the detection of oil spills in satellite radar image. Mach. Learn. 30, 195–215 (1988)
Matjaz, K., Igor, K., et al.: Cost-sensitive learning with neural networks. In: Proceedings of European Conference on Artificial Intelligence (1998)
Longin, J.L., Aleksandar, L., Dragoljub, P.: Outlier detection with kernel density functions. In: Proceedings of International Workshop on Machine Learning and Data Mining in Pattern Recognition (2007)
Steve, L., Ian, B., Andrew, B., Ah Chung, T., Giles, C.L.: Neural network classification and prior class probabilities. In: Neural Networks: Tricks of the Trade (1998)
Lei, L., Andrew, P., Martha, W.: Supervised autoencoders: improving generalization performance with unsupervised regularizers. In: Proceedings of Neural Information Processing Systems (2018)
Hyoung-joo, L., Sungzoon, C.: The novelty detection approach for different degrees of class imbalance. In: Proceedings of International Conference on Neural Information processing (2006)
Alireza, M.,, Jonathon, S., Navdeep, J., Ian, G., Brendan, F.: Adversarial Autoencoders. arXiv preprint arXiv:1511.05644 (2015)
Mazurowski, M.A., Habas, P.A., Zurada, J.M., Lo, J.Y., Baker, J.A., Tourassi, G.D.: Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance. Neural Networks 21(2–3), 427–436 (2008)
Ninareh, M., Fred, M., Nripsuta, S., Kristina, L., Aram, G.: A survey on Bias and Fairness in Machine Learning. arXiv preprint arXiv:1908.09635 (2019)
Ng, W.W., Zeng, G., Zhang, J., Yeung, D.S., Pedrycz, W.: Dual autoencoders features for imbalance classification problem. Pattern Recogn. 60, 875–889 (2016)
Olszewski, D.: A probabilistic approach to fraud detection in telecommunications. Knowl.-Based Syst. 26, 246–258 (2012)
Panigrahi, S., Kundu, A., Sural, S., Majumdar, A.K.: Credit card fraud detection: A fusion approach using Dempster-theory and Bayesian learning. Information Fusion 10(4), 354–363 (2009)
Jeffrey, P., Richard, S., Christopher, D.M.: Glove: global vectors for word representation. In: Proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP), (2014)
Hamed, S., Carlotta, D., Bardh, P., Giovanni, S.: Unsupervised Boosting-based Autoencoder Ensembles for Outlier Detection. arXiv preprint arXiv:1910.09754, 2019
Scholkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines. Optimization, and Beyond, Regularization (2001)
Tavallaee, M., Bagheri, E., Lu, W., Ghorbani, A.A.: A detailed analysis of the KDD CUP 99 data set. In: 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications (2009)
Yang, Y., Liu, X.: A re-examination of text categorization methods. In: Proceedings of International Conference on Research and Development in Information, Retrieval (1999)
Matthew, D.Z.A.: An Adaptive Learning Rate Method. arXiv preprint arXiv:1212.5701 (2012)
Zhou, C., Paffenroth, R.C.: Anomaly detection with robust deep autoencoders. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2017)
Junyi, Z., Jinliang, Z., Ping, J.: Credit Card Fraud Detection Using Autoencoder Neural Network. arXiv preprint arXiv:1908.11553 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Lübbering, M., Ramamurthy, R., Gebauer, M., Bell, T., Sifa, R., Bauckhage, C. (2020). From Imbalanced Classification to Supervised Outlier Detection Problems: Adversarially Trained Auto Encoders. In: Farkaš, I., Masulli, P., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2020. ICANN 2020. Lecture Notes in Computer Science(), vol 12396. Springer, Cham. https://doi.org/10.1007/978-3-030-61609-0_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-61609-0_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-61608-3
Online ISBN: 978-3-030-61609-0
eBook Packages: Computer ScienceComputer Science (R0)