Top

Neural Computing and Applications

Published in:

18-01-2019 | IWINAC 2015

Improving deep learning performance with missing values via deletion and compensation

Authors: Adrián Sánchez-Morales, José-Luis Sancho-Gómez, Juan-Antonio Martínez-García, Aníbal R. Figueiras-Vidal

Published in: Neural Computing and Applications | Issue 17/2020

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Missing values in a dataset is one of the most common difficulties in real applications. Many different techniques based on machine learning have been proposed in the literature to face this problem. In this work, the great representation capability of the stacked denoising auto-encoders is used to obtain a new method of imputating missing values based on two ideas: deletion and compensation. This method improves imputation performance by artificially deleting values in the input features and using them as targets in the training process. Nevertheless, although the deletion of samples is demonstrated to be really efficient, it may cause an imbalance between the distributions of the training and the test sets. In order to solve this issue, a compensation mechanism is proposed based on a slight modification of the error function to be optimized. Experiments over several datasets show that the deletion and compensation not only involve improvements in imputation but also in classification in comparison with other classical techniques.

previous article Nonlinear predictability analysis of brain dynamics for automatic recognition of negative stress

next article Fixed-time synchronization of competitive neural networks with proportional delays and impulsive effect

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Sharpe PK, Solly RJ (1995) Dealing with missing values in neural network-based diagnostic systems. Neural Comput Appl 3(2):73–77. https://doi.org/10.1007/BF01421959CrossRef

Little R, Rubin D (2002) Statistical analysis with missing data, 2nd edn. Wiley, LondonCrossRef

García-Laencina PJ, Sancho-Gómez JL, Figueiras-Vidal AR (2010) Pattern classification with missing data: a review. Neural Comput Appl 19(2):263–282. https://doi.org/10.1007/s00521-009-0295-6CrossRef

Quinlan JR (1993) C4.5: programs for machine learning. Morgan-Kaufmann, Burlington

Lim CP, Leong JH, Kuan MM (2005) A hybrid neural network system for pattern classification tasks with missing features. IEEE Trans Pattern Anal Mach Intell 27:648–653. https://doi.org/10.1109/TPAMI.2005.64CrossRef

Del Castillo PR, Cardeosa J (2012) Fuzzy min–max neural networks for categorical data: application to missing data imputation. Neural Comput Appl 21(6):1349–1362. https://doi.org/10.1007/s00521-011-0574-xCrossRef

Delalleau O, Courville A, Bengio Y (2008) Gaussian mixtures with missing data: an efficient EM training algorithm. In: Proceeding of the computing research association conference, Snowbird, p 155

Ghahramani Z, Jordan MI (1994) Supervised learning from incomplete data via an EM approach. In: Cowan JD, Tesauro G, Alspector J (eds) Advances in neural information processing systems, vol 6. Morgan-Kaufmann, Burlington, pp 120–127

Zio MD, Guarnera U, Luzi O (2007) Imputation through finite Gaussian mixture models. Comput Stat Data Anal 51(11):5305–5316. https://doi.org/10.1016/j.csda.2006.10.002MathSciNetCrossRefMATH

10.

García-Laencina PJ, Sancho-Gómez JL, Figueiras-Vidal AR, Verleysen M (2009) K nearest neighbours with mutual information for simultaneous classification and missing data imputation. Neurocomputing 72(7–9):1483–1493. https://doi.org/10.1016/j.neucom.2008.11.026CrossRef

11.

Batista GE, Monard MC (2003) An analysis of four missing data treatment methods for supervised learning. Appl Artif Intell 17(5–6):519–533. https://doi.org/10.1080/713827181CrossRef

12.

Troyanskaya O, Cantor M, Sherlock G, Brown P, Hastie T, Tibshirani R, David Botstein D, Altman RB (2001) Missing value estimation methods for DNA microarrays. Bioinformatics 17(6):520–525CrossRef

13.

Fessant F, Midenet S (2002) Self-organising map for data imputation and correction in surveys. Neural Comput Appl 10(4):300–310. https://doi.org/10.1007/s005210200002CrossRefMATH

14.

Peng H, Zhu S (2007) Handling of incomplete data sets using ICA and SOM in data mining. Neural Comput Appl 16(2):167–172. https://doi.org/10.1007/s00521-006-0058-6CrossRef

15.

Latif BA, Mercier G (2010) Self-organizing maps. https://doi.org/10.5772/9178

16.

Gupta A, Lam MS (1996) Estimating missing values using neural networks. J Oper Res Soc 47:229–238. https://doi.org/10.2307/2584344CrossRefMATH

17.

Nishanth KJ, Ravi V, Ankaiaha N, Bose I (2012) Soft computing based imputation and hybrid data and text mining: the case of predicting the severity of phishing alerts. Expert Syst Appl 39(12):10583–10589. https://doi.org/10.1016/j.eswa.2012.02.138CrossRef

18.

Smola AJ, Vishwanathan SVN, Hofmann T (2005) Kernel methods for missing variables. In: Proceedings of the 10th international workshop on artificial intelligence and statistics, pp 325–332

19.

García-Laencina PJ, Sancho-Gómez JL, Figueiras-Vidal AR (2013) Classifying patterns with missing values using multi-task learning perceptrons. Expert Syst Appl 40(4):1333–1341. https://doi.org/10.1016/j.eswa.2012.08.057CrossRef

20.

Bengio Y, Lecun Y (2007) Scaling learning algorithms towards AI. MIT Press, Cambridge

21.

Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput Appl 18(7):1527–1554. https://doi.org/10.1162/neco.2006.18.7.1527MathSciNetCrossRefMATH

22.

Bengio Y (2009) Learning deep architectures for AI. Found Trends Mach Learn 2(1):1–127. https://doi.org/10.1561/2200000006CrossRefMATH

23.

Deng L, Yu D (2014) Deep learning: methods and applications. Found Trends Signal Process 7(3–4):197–387. https://doi.org/10.1561/2000000039MathSciNetCrossRefMATH

24.

Beaulieu-Jones BK, Moore JH (2017) Missing data imputation in the electronic health record using deeply learned autoencoders. World Scientific, Singapore, pp 207–218. https://doi.org/10.1142/97898132078130021CrossRef

25.

Gondara L, Wang K (2017) Multiple imputation using deep denoising autoencoders. arXiv:1705.02737v2

26.

Sánchez-Morales A, Sancho-Gómez JL, Figueiras-Vidal AR (2017) Values deletion to improve deep imputation processes. In: International work-conference on the interplay between natural and artificial computation, IWINAC 2017, Coruna, pp 240–246. https://doi.org/10.1007/978-3-319-59773-7-25

27.

Vincent P, Larochelle H, Bengio Y, Manzagol PA (2008) Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th international conference on machine learning, ICML’08. ACM, New York, pp 1096–1103. https://doi.org/10.1145/1390156.1390294

28.

Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA (2010) Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11:3371–3408MathSciNetMATH

29.

Alvear-Sandoval RF, Figueiras-Vidal AR (2018) On building ensembles of stacked denoising auto-encoding classifiers and their further improvement. Inf Fusion 39:41–52. https://doi.org/10.1016/j.inffus.2017.03.008CrossRef

30.

Little RJA, Rubin DB (1986) Statistical analysis with missing data. Wiley, LondonMATH

31.

Schafer JL (1997) Analysis of incomplete multivariate data. Chapman & Hall, LondonCrossRef

32.

Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml

33.

Delve: data for evaluating learning in valid experiments. https://www.cs.toronto.edu/~delve/data/datasets.html

34.

Schmitt P, Mandel J, Guedj M (2015) A comparison of six methods for missing data imputation. J Biomet Biostat 6:224. https://doi.org/10.4172/2155-6180.1000224CrossRef

35.

Azur MJ, Stuart EA, Frangakis C, Leaf PJ (2011) Multiple imputation by chained equations: what is it and how does it work? Int J Methods Psychiatr Res 20(1):40–49. https://doi.org/10.1002/mpr.329CrossRef

36.

Brahma PP, Wu D, She Y (2016) Why deep learning works: a manifold disentanglement perspective. IEEE Trans Neural Netw Learn Syst 27(10):1997–2008. https://doi.org/10.1109/tnnls.2015.2496947MathSciNetCrossRef

37.

Goodfellow I, McDaniel P, Papernot N (2018) Making machine learning robust against adversarial inputs. Commun ACM 61(6):56–66. https://doi.org/10.1145/3134599CrossRef

38.

Vorobeychik Y, Kantarcioglu M (2018) Adversarial machine learning. Synth Lect Artif Intell Mach Learn 12(3):1–169CrossRef

Title: Improving deep learning performance with missing values via deletion and compensation
Authors: Adrián Sánchez-Morales
José-Luis Sancho-Gómez
Juan-Antonio Martínez-García
Aníbal R. Figueiras-Vidal
Publication date: 18-01-2019
Publisher: Springer London
Published in: Neural Computing and Applications / Issue 17/2020
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI: https://doi.org/10.1007/s00521-019-04013-2

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 17/2020

Constructing interval-valued generalized partitioned Bonferroni mean operator with several extensions for MAGDM

Semi-supervised person re-identification by similarity-embedded cycle GANs

Multi-attribute group decision-making using double hierarchy hesitant fuzzy linguistic preference information

A multi-objective open set orienteering problem

Analysing wear behaviour of Al–CaCO3 composites using ANN and Sugeno-type fuzzy inference systems

Coping with opponents: multi-objective evolutionary neural networks for fighting games

Premium Partner