Top

Neural Computing and Applications

Published in:

16-03-2017 | New Trends in data pre-processing methods for signal and image classification

An improved dynamic sampling back-propagation algorithm based on mean square error to face the multi-class imbalance problem

Authors: R. Alejo, J. Monroy-de-Jesús, J. C. Ambriz-Polo, J. H. Pacheco-Sánchez

Published in: Neural Computing and Applications | Issue 10/2017

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

In this paper, we present an improved dynamic sampling approach (I-SDSA) for facing the multi-class imbalance problem. I-SDSA is a modification of the back-propagation algorithm, which is focused to make a better use of the training samples for improving the classification performance of the multilayer perceptron (MLP). I-SDSA uses the mean square error and a Gaussian function to identify the best samples to train the neural network. Results shown in this article stand out that I-SDSA makes better exploitation of the training dataset and improves the MLP classification performance. In others words, I-SDSA is a successful technique for dealing with the multi-class imbalance problem. In addition, results presented in this work indicate that the proposed method is very competitive in terms of classification performance with respect to classical over-sampling methods (also, combined with well-known features selection methods) and other dynamic sampling approaches, even in training time and size it is better than the over-sampling methods .

previous article Guest editorial: New trends in data pre-processing methods for signal and image classification

next article A cloud-based recommendation service using principle component analysis–scale-invariant feature transform algorithm

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

This MLP only has two neural network outputs (\(z_{0}^{q}\) and \(z_{1}^{q}\)), because it has been designed to work with datasets of two classes.

Abdi L, Hashemi S (2016) To combat multi-class imbalanced problems by means of over-sampling techniques. IEEE Trans Knowl Data Eng 28(1):1041–4347. doi:10.1109/TKDE.2015.2458858 CrossRef

Al-Shahib A, Breitling R, Gilbert D (2005) Feature selection and the class imbalance problem in predicting protein function from sequence. Appl Bioinform 4(3):195–203. doi:10.2165/00822942-200504030-00004 CrossRef

Alcalá-Fdez J, Fernandez A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2011) Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J Mult Valued Logic Soft Comput 17(2–3):255–287

Alejo R, García V, Pacheco-Sánchez JH (2015) An efficient over-sampling approach based on mean square error back-propagation for dealing with the multi-class imbalance problem. Neural Process Lett 42(3):603–617. doi:10.1007/s11063-014-9376-3 CrossRef

Alejo R, Monroy-de Jesús J, Pacheco-Sánchez JH, López-González E, Antonio-Velázquez JA (2016) A selective dynamic sampling back-propagation approach for handling the two-class imbalance problem. Appl Sci 6(7):200. doi:10.3390/app6070200. http://www.mdpi.com/2076-3417/6/7/200

Alejo R, Valdovinos R, García V, Pacheco-Sanchez JH (2013) A hybrid method to face class overlap and class imbalance on neural networks and multi-class scenarios. Pattern Recognit Lett 34(4):380–388CrossRef

Anand R, Mehrotra K, Mohan C, Ranka S (1993) An improved algorithm for neural network classification of imbalanced training sets. IEEE Trans Neural Netw 4:962–969CrossRef

Asuncion A, Newman D (2007) UCI machine learning repository. http://www.ics.uci.edu/~mlearn/

Batista G, Prati R, Monard M (2004) A study of the behavior of several methods for balancing machine learning training data. SIGKDD Explor Newsl 6:20–29CrossRef

10.

Baumgardner M, Biehl L, Landgrebe D (1992) 220 band aviris hyperspectral image data set: June 12, Indian pine test site 3 (2016). http://engineering.purdue.edu/biehl/MultiSpec/hyperspectral.html

11.

Bruzzone L, Serpico SB (1997) Classification of imbalanced remote-sensing data by neural networks. Pattern Recognit Lett 18:1323–1328CrossRef

12.

Bunkhumpornpat C, Sinapiromsaran K, Lursinsap C (2009) Safe-level-smote: safe-level-synthetic minority over-sampling technique for handling the class imbalanced problem. In: Pacific-Asia conference on knowledge discovery and data mining(PAKDD09). Lecture notes on computer science, vol 5476. Springer, pp 475–482. doi:10.1007/978-3-642-01307-2_43

13.

Chawla N, Cieslak D, Hall L, Ajay J (2008) Automatically countering imbalance and its empirical relationship to cost. Data Min Knowl Discov 17:225–252MathSciNetCrossRef

14.

Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) Smote: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357MATH

15.

Duda R, Hart P, Stork D (2001) Pattern classification and scene analysis, 2nd edn. Wiley, New YorkMATH

16.

Erguzel TT, Tas C, Cebi M (2015) A wrapper-based approach for feature selection and classification of major depressive disorder–bipolar disorders. Comput Biol Med 64:127–137. doi:10.1016/j.compbiomed.2015.06.021 CrossRef

17.

Fawcett T (2006) An introduction to roc analysis. Pattern Recognit Lett 27:861–874CrossRef

18.

Fernández A, López V, Galar M, del Jesus M, Herrera F (2013) Analysing the classification of imbalanced data-sets with multiple classes: binarization techniques and ad-hoc approaches. Knowl Based Syst 42:97–110CrossRef

19.

Fernández-Navarro F, Hervás-Martínez C, Gutiérrez PA (2011) A dynamic over-sampling procedure based on sensitivity for multi-class problems. Pattern Recognit 44(8):1821–1833CrossRefMATH

20.

Fernández-Navarro F, Hervás-Martínez C, García-Alonso C, Torres-Jiménez M (2011) Determination of relative agrarian technical efficiency by a dynamic over-sampling procedure guided by minimum sensitivity. Expert Syst Appl 38(10):12483–12490CrossRef

21.

Galar M, Fernández A, Tartas EB, Sola HB, Herrera F (2012) A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Trans Syst Man Cybern Part C 42(4):463–484. doi:10.1109/TSMCC.2011.2161285 CrossRef

22.

García S, Herrera F (2008) An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. J Mach Learn Res 9:2677–2694MATH

23.

Han H, Wang WY, Mao BH (2005) Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning. In: Huang DS, Zhang XP, Huang GB (eds) Advances in intelligent computing. ICIC 2005. Lecture notes in computer science, vol 3644. Springer, Berlin, pp 878–887

24.

Hart P (1968) The condensed nearest neighbour rule. IEEE Trans Inf Theory 14(5):515–516CrossRef

25.

Hatami N, Ebrahimpour R, Ghaderi R (2013) Ecoc-based training of neural networks for face recognition. CoRR abs/1312.3990. http://arxiv.org/abs/1312.3990

26.

He H, Bai Y, Garcia E, Li S (2008) Adasyn: adaptive synthetic sampling approach for imbalanced learning. In: IJCNN. pp 1322–1328

27.

He H, Garcia E (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284CrossRef

28.

Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6(2):65–70MathSciNetMATH

29.

Jing H, Wang B, Yang Y, Xu Y (2009) A general framework of feature selection for text categorization. Springer, Berlin, pp 647–662. doi:10.1007/978-3-642-03070-3_49

30.

Lecun Y, Bottou L, Orr GB, Müller KR (1998) Efficient backprop. In: Orr G, Müller K. (eds) Neural networks—tricks of the trade. Lecture notes in computer science, vol 1524. Springer, pp 5–50

31.

Lin M, Tang K, Yao X (2013) Dynamic sampling approach to training neural networks for multiclass imbalance classification. IEEE Trans Neural Netw Learn Syst 24(4):647–660CrossRef

32.

Liu H, Setiono R (1996) Feature selection and classification: a probabilistic wrapper approach. In: 9th International conference on industrial and engineering applications of artificial intelligence and expert systems(IEA-AIE96). pp 419–424

33.

López V, Fernández A, García S, Palade V, Herrera F (2013) An insight into classification with imbalanced data: empirical results and current trends on using data intrinsic characteristics. Inf Sci 250:113–141CrossRef

34.

Luengo J, García S, Herrera F (2009) A study on the use of statistical tests for experimentation with neural networks: analysis of parametric test conditions and non-parametric tests. Expert Syst Appl 36(4):7798–7808CrossRef

35.

Maldonado S, Weber R, Famili F (2014) Feature selection for high-dimensional class-imbalanced data sets using support vector machines. Inf Sci 286:228–246. doi:10.1016/j.ins.2014.07.015 CrossRef

36.

Mirza B, Lin Z, Liu N (2015) Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift. Neurocomputing 149:316–329. doi:10.1016/j.neucom.2014.03.075 CrossRef

37.

Nanni L, Fantozzi C, Lazzarini N (2015) Coupling different methods for overcoming the class imbalance problem. Neurocomputing 158(C):48–61CrossRef

38.

Prati R, Batista G, Monard M (2004) Class imbalances versus class overlapping: an analysis of a learning system behavior. In: MICAI. pp 312–321

39.

Prati RC, Batista GE, Monard MC (2009) Data mining with imbalanced class distributions: concepts and methods. In: Proceedings of the 4th Indian international conference on artificial intelligence, IICAI, Tumkur, Karnataka, India, 16-18 Dec 2009, pp 359–376

40.

Sánchez JS, García V, Mollineda RA (2011) Exploring synergetic effects of dimensionality reduction and resampling tools on hyperspectral imagery data classification. In: Proceedings of the 7th International conference on machine learning and data mining in pattern recognition, MLDM’11. Springer, Berlin, pp 511–523. http://dl.acm.org/citation.cfm?id=2033831.2033875

41.

Shaffer J (1986) Modified sequentially rejective multiple test procedures. J Am Stat Assoc 81(375): 826–831. http://www.jstor.org/stable/2289016

42.

Show-Jane Y, Yue-Shi L (2009) Cluster-based under-sampling approaches for imbalanced data distributions. Expert Syst Appl 36:5718–5727. doi:10.1016/j.eswa.2008.06.108 CrossRef

43.

Sun T, Jiao L, Feng J, Liu F, Zhang X (2015) Imbalanced hyperspectral image classification based on maximum margin. IEEE Geosci Remote Sens Lett 12(3):522–526CrossRef

44.

Tomek I (1976) Two modifications of CNN. IEEE Trans Syst Man Cybern 7(2):679–772MathSciNetMATH

45.

Wang J, Jean JSN (1993) Resolving multifont character confusion with neural networks. Pattern Recognit 26(1):175–187CrossRef

46.

Wang S, Yao X (2012) Multiclass imbalance problems: analysis and potential solutions. IEEE Trans Syst Man Cybern Part B 42(4):1119–1130CrossRef

47.

Xu-Ying L, Qian-Qian L, Zhi-Hua Z (2013) Learning imbalanced multi-class data with optimal dichotomy weights. In: 2013 IEEE 13th international conference on data mining, Dallas, TX, USA, 7–10 Dec 2013, pp 478–487. doi:10.1109/ICDM.2013.51

48.

Yang J, Honavar V (1998) Feature subset selection using a genetic algorithm. IEEE Intell Syst 13(2):44–49CrossRef

49.

Zheng Z, Wu X, Srihari R (2004) Feature selection for text categorization on imbalanced data. SIGKDD Explor Newsl 6(1):80–89. doi:10.1145/1007730.1007741 CrossRef

50.

Zhou ZH, Liu XY (2006) Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Trans Knowl Data Eng 18:63–77CrossRef

51.

Zhou ZH, Liu XY (2010) On multi-class cost-sensitive learning. Comput Intell 26(3):232–257MathSciNetCrossRef

Title: An improved dynamic sampling back-propagation algorithm based on mean square error to face the multi-class imbalance problem
Authors: R. Alejo
J. Monroy-de-Jesús
J. C. Ambriz-Polo
J. H. Pacheco-Sánchez
Publication date: 16-03-2017
Publisher: Springer London
Published in: Neural Computing and Applications / Issue 10/2017
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI: https://doi.org/10.1007/s00521-017-2938-3

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 10/2017

Investigation of different approaches for noise reduction in functional near-infrared spectroscopy signals for brain–computer interface applications

Automated screening of congestive heart failure using variational mode decomposition and texture features extracted from ultrasound images

Muscular synergy classification and myoelectric control using high-order cross-cumulants

A new approach to eliminating EOG artifacts from the sleep EEG signals for the automatic sleep stage classification

A novel image segmentation approach based on neutrosophic c-means clustering and indeterminacy filtering

Automatic detection of respiratory arrests in OSA patients using PPG and machine learning techniques

Premium Partner