Skip to main content
Top
Published in: Neural Processing Letters 1/2019

13-07-2019

An Ar2p Deep Learning Architecture for the Discovery and the Selection of Features

Authors: E. Puerto, J. Aguilar, R. Vargas, J. Reyes

Published in: Neural Processing Letters | Issue 1/2019

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In the context of pattern recognition processes with machine learning algorithms, either through supervised, semi-supervised or unsupervised methods, one of the most important elements to consider are the features that are used to represent the phenomenon to be studied. In this sense, this paper proposes a deep learning architecture for Ar2p, which is based on supervised and unsupervised mechanisms for the discovery and the selection of features for classification problems (called Ar2p-DL). Ar2p is an algorithm of pattern recognition based on the systematic functioning of the human brain. Ar2p-DL is composed of three phases: the first phase, called feature analysis, is supported by two feature-engineering approaches to discover or select atomic features/descriptors. The feature engineering approach used for the discovery, is based on a classical clustering technique, K-means; and the approach used for the selection, is based on a classification technique, Random Forest. The second phase, called aggregation, creates a feature hierarchy (merge of descriptors) from the atomic features/descriptors (it uses as aggregation strategy the DBSCAN algorithm). Finally, the classification phase carries out the classification of the inputs based on the feature hierarchy, using the classical Ar2p algorithm. The last phase of Ar2p-DL uses a supervised learning approach, while the first phases combine supervised and unsupervised learning approaches. To analyze the performance of Ar2p-DL, several tests have been made using different benchmarks (datasets) from the UCI Machine Learning Repository, in order to compare Ar2p-DL with other classification methods.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Guyon I, Gunn S, Nikravesh M, Zadeh LA (2006) Feature extraction foundations and applications. Springer, BerlinCrossRefMATH Guyon I, Gunn S, Nikravesh M, Zadeh LA (2006) Feature extraction foundations and applications. Springer, BerlinCrossRefMATH
2.
3.
go back to reference Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 6:85–117CrossRef Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 6:85–117CrossRef
4.
go back to reference LeCun Y, Bengio Y (1998) Convolutional networks for images, speech, and time series. In: Arbib MA (ed) The handbook of brain theory and neural networks. MIT Press, Cambridge, MA, pp 255–258 LeCun Y, Bengio Y (1998) Convolutional networks for images, speech, and time series. In: Arbib MA (ed) The handbook of brain theory and neural networks. MIT Press, Cambridge, MA, pp 255–258
5.
go back to reference Salakhutdinov R, Hinton G (2010) Efficient learning of deep Boltzmann machines. In: Proc. intl conference on artificial intelligence and statistics, pp 693–700 Salakhutdinov R, Hinton G (2010) Efficient learning of deep Boltzmann machines. In: Proc. intl conference on artificial intelligence and statistics, pp 693–700
6.
go back to reference Hua Y, Guo J, Hua Z (2015) Deep belief networks and deep learning. In: Proc. int. conf. intel. comput. internet things, pp 1–4 Hua Y, Guo J, Hua Z (2015) Deep belief networks and deep learning. In: Proc. int. conf. intel. comput. internet things, pp 1–4
7.
go back to reference Ester M, Kriegel H, Sander J, Xu X (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proc. 2nd intl conf on knowledge discovery and data mining, pp 226–231 Ester M, Kriegel H, Sander J, Xu X (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proc. 2nd intl conf on knowledge discovery and data mining, pp 226–231
8.
go back to reference Aguilar J (2001) Learning algorithm and retrieval process for the multiple classes random neural network model. Neural Process Lett 13(1):81–91CrossRefMATH Aguilar J (2001) Learning algorithm and retrieval process for the multiple classes random neural network model. Neural Process Lett 13(1):81–91CrossRefMATH
9.
go back to reference Aguilar J (1998) Definition of an energy function for the random neural to solve optimization problems. Neural Netw 11(4):731–738CrossRef Aguilar J (1998) Definition of an energy function for the random neural to solve optimization problems. Neural Netw 11(4):731–738CrossRef
10.
go back to reference Gelenbe E, Yin Y (2016) Deep learning with random neural networks. In: Proc. international joint conference on neural networks (IJCNN), pp 1633–1638 Gelenbe E, Yin Y (2016) Deep learning with random neural networks. In: Proc. international joint conference on neural networks (IJCNN), pp 1633–1638
11.
go back to reference Aguilar J (2004) A color pattern recognition problem based on the multiple classes random neural network model. Neurocomputing 61:71–83CrossRef Aguilar J (2004) A color pattern recognition problem based on the multiple classes random neural network model. Neurocomputing 61:71–83CrossRef
12.
go back to reference Zhang J, Yu J, Tao D (2018) Local deep-feature alignment for unsupervised dimension reduction. IEEE Trans Image Process 27(5):2420–2432MathSciNetCrossRefMATH Zhang J, Yu J, Tao D (2018) Local deep-feature alignment for unsupervised dimension reduction. IEEE Trans Image Process 27(5):2420–2432MathSciNetCrossRefMATH
15.
go back to reference Kumar G, Bhatia PK (2014) A detailed review of feature extraction in image processing systems. In: Proc. fourth international conference on advanced computing & communication technologies. IEEE, pp 5–12 Kumar G, Bhatia PK (2014) A detailed review of feature extraction in image processing systems. In: Proc. fourth international conference on advanced computing & communication technologies. IEEE, pp 5–12
16.
go back to reference Pacheco F, Exposito E, Gineste M, Budoin C, Aguilar J (2019) Towards the deployment of machine learning solutions in traffic network classification: a systematic survey. IEEE Commun Surv Tutor 21(2):1988–2014CrossRef Pacheco F, Exposito E, Gineste M, Budoin C, Aguilar J (2019) Towards the deployment of machine learning solutions in traffic network classification: a systematic survey. IEEE Commun Surv Tutor 21(2):1988–2014CrossRef
17.
go back to reference Chang M, Buš P, Schmitt G (2017) Feature extraction and K-means clustering approach to explore important features of urban identity. In: 16th IEEE international conference on machine learning and applications (ICMLA), pp 1139–1144 Chang M, Buš P, Schmitt G (2017) Feature extraction and K-means clustering approach to explore important features of urban identity. In: 16th IEEE international conference on machine learning and applications (ICMLA), pp 1139–1144
18.
go back to reference Puerto E, Aguilar J (2017) Un algoritmo recursivo de reconocimiento de patrones. Revista Técnica de la Facultad de Ingeniería Universidad del Zulia 40(2):95–104 Puerto E, Aguilar J (2017) Un algoritmo recursivo de reconocimiento de patrones. Revista Técnica de la Facultad de Ingeniería Universidad del Zulia 40(2):95–104
19.
go back to reference Puerto E, Aguilar J, Chavez D (2017) A new recursive patterns matching model inspired in systematic theory of human mind. Int J Adv Comput Technol (IJACT) 9(1):28–39 Puerto E, Aguilar J, Chavez D (2017) A new recursive patterns matching model inspired in systematic theory of human mind. Int J Adv Comput Technol (IJACT) 9(1):28–39
20.
go back to reference Puerto E, Aguilar J (2016) Formal description of a pattern for a recursive process of recognition. In: Proc IEEE Latin American conference on computational intelligence, pp 1–2 Puerto E, Aguilar J (2016) Formal description of a pattern for a recursive process of recognition. In: Proc IEEE Latin American conference on computational intelligence, pp 1–2
21.
go back to reference Puerto E, Aguilar J (2016) Learning algorithm for the recursive pattern recognition model. Appl Artif Intell 30(7):662–678CrossRef Puerto E, Aguilar J (2016) Learning algorithm for the recursive pattern recognition model. Appl Artif Intell 30(7):662–678CrossRef
22.
go back to reference Kurzweil R (2013) How to make mind. The Futurist 47(2):14–17 Kurzweil R (2013) How to make mind. The Futurist 47(2):14–17
23.
go back to reference Puerto E, Aguilar J, Chávez D (2018) A recursive patterns matching model for the dynamic pattern recognition problem. Appl Artif Intell 32(4):419–432CrossRef Puerto E, Aguilar J, Chávez D (2018) A recursive patterns matching model for the dynamic pattern recognition problem. Appl Artif Intell 32(4):419–432CrossRef
24.
go back to reference Liu H, Motoda H (1998) Feature extraction, construction and selection: a data mining perspective, vol 453. Springer, BerlinCrossRefMATH Liu H, Motoda H (1998) Feature extraction, construction and selection: a data mining perspective, vol 453. Springer, BerlinCrossRefMATH
25.
go back to reference Khalid S, Khalil T, Nasreen S (2014) A survey of feature selection and feature extraction techniques in machine learning. In: Proc. IEEE science and information conference (SAI), pp 372–378 Khalid S, Khalil T, Nasreen S (2014) A survey of feature selection and feature extraction techniques in machine learning. In: Proc. IEEE science and information conference (SAI), pp 372–378
26.
go back to reference Motoda H, Liu H (2002) Feature selection, extraction and construction. Commun IICM 5:67–72 Motoda H, Liu H (2002) Feature selection, extraction and construction. Commun IICM 5:67–72
27.
go back to reference Yu J, Kuang Z, Zhang B, Zhang W, Lin D, Fan J (2018) Leveraging content sensitiveness and user trustworthiness to recommend fine-grained privacy settings for social image sharing. IEEE Trans Inf Forensics Secur 13(5):1317–1332CrossRef Yu J, Kuang Z, Zhang B, Zhang W, Lin D, Fan J (2018) Leveraging content sensitiveness and user trustworthiness to recommend fine-grained privacy settings for social image sharing. IEEE Trans Inf Forensics Secur 13(5):1317–1332CrossRef
28.
go back to reference Yu J, Liu D, Tao D, Seah H (2012) On combining multiple features for cartoon character retrieval and clip synthesis. IEEE Trans Syst Man Cybern 42(5):1413–1427CrossRef Yu J, Liu D, Tao D, Seah H (2012) On combining multiple features for cartoon character retrieval and clip synthesis. IEEE Trans Syst Man Cybern 42(5):1413–1427CrossRef
29.
go back to reference Lausser L, Szekely R, Schirra L, Kestler H (2018) The influence of multi-class feature selection on the prediction of diagnostic phenotypes. Neural Process Lett 48(2):863–880CrossRef Lausser L, Szekely R, Schirra L, Kestler H (2018) The influence of multi-class feature selection on the prediction of diagnostic phenotypes. Neural Process Lett 48(2):863–880CrossRef
30.
go back to reference Singaravel S, Suykens J, Geyer P (2018) Deep-learning neural-network architectures and methods: using component based models in building-design energy prediction. Adv Eng Inform 38:81–90CrossRef Singaravel S, Suykens J, Geyer P (2018) Deep-learning neural-network architectures and methods: using component based models in building-design energy prediction. Adv Eng Inform 38:81–90CrossRef
31.
go back to reference Pham D, Dimov S, Nguyen C (2005) Selection of K in K-means clustering. In: Proc. inst. mech. eng., pp 103–119 Pham D, Dimov S, Nguyen C (2005) Selection of K in K-means clustering. In: Proc. inst. mech. eng., pp 103–119
32.
go back to reference Pham D, Dimov S, Nguyen C (2004) An incremental K-means algorithm. J Mech Eng Sci 218(7):783–795CrossRef Pham D, Dimov S, Nguyen C (2004) An incremental K-means algorithm. J Mech Eng Sci 218(7):783–795CrossRef
33.
go back to reference Pelleg D, Moore A (2000) X-means: extending K-means with efficient estimation of the number of clusters. In: Proc. of the 17th international conf. on machine learning, pp 727–734 Pelleg D, Moore A (2000) X-means: extending K-means with efficient estimation of the number of clusters. In: Proc. of the 17th international conf. on machine learning, pp 727–734
34.
go back to reference Kass R, Wasserman L (1995) A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. J Am Stat Assoc 90(431):928–934MathSciNetCrossRefMATH Kass R, Wasserman L (1995) A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. J Am Stat Assoc 90(431):928–934MathSciNetCrossRefMATH
36.
go back to reference Balakrishnama A, Ganapathiraju S (1998) Linear discriminant analysis-a brief tutorial. Inst Signal Inf Process 18:1–8 Balakrishnama A, Ganapathiraju S (1998) Linear discriminant analysis-a brief tutorial. Inst Signal Inf Process 18:1–8
37.
go back to reference Ester M, Kriegel H-P, Sander J, Xu X (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proc. 2nd int conf on knowledge discovery and data mining, pp 226–231 Ester M, Kriegel H-P, Sander J, Xu X (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proc. 2nd int conf on knowledge discovery and data mining, pp 226–231
38.
go back to reference Wagner P, Peres S, Lima C, Freitas F, Barros R (2014) Gesture unit segmentation using spatial-temporal information and machine learning. In: Proc. twenty-seventh international Florida artificial intelligence research society conference, pp 101–106 Wagner P, Peres S, Lima C, Freitas F, Barros R (2014) Gesture unit segmentation using spatial-temporal information and machine learning. In: Proc. twenty-seventh international Florida artificial intelligence research society conference, pp 101–106
39.
go back to reference Lichman M (2013) UCI machine learning repository. University of California, Irvine Lichman M (2013) UCI machine learning repository. University of California, Irvine
40.
go back to reference El Kessab B, Daoui C, Boukhalene B, Salouan R (2014) A comparative study between the K-nearest neighbors and the multi-layer perceptron for cursive handwritten arabic numerals recognition. Int J Comput Appl 107(21):25–30 El Kessab B, Daoui C, Boukhalene B, Salouan R (2014) A comparative study between the K-nearest neighbors and the multi-layer perceptron for cursive handwritten arabic numerals recognition. Int J Comput Appl 107(21):25–30
41.
go back to reference Keith MJ et al (2010) The high time resolution universe pulsar survey—I. System configuration and initial discoveries. Mon Not R Astron Soc 409(2):619–627CrossRef Keith MJ et al (2010) The high time resolution universe pulsar survey—I. System configuration and initial discoveries. Mon Not R Astron Soc 409(2):619–627CrossRef
42.
go back to reference Lyon RJ, Stappers BW, Cooper S, Brooke JM, Knowles JD (2016) Fifty years of pulsar candidate selection: from simple filters to a new principled real-time classification approach. Mon Not R Astron Soc 459(1):1104–1123CrossRef Lyon RJ, Stappers BW, Cooper S, Brooke JM, Knowles JD (2016) Fifty years of pulsar candidate selection: from simple filters to a new principled real-time classification approach. Mon Not R Astron Soc 459(1):1104–1123CrossRef
43.
go back to reference Charytanowicz M et al. (2010) A complete gradient clustering algorithm for features analysis of X-ray images. In: Proc. information technologies in biomedicine. Springer, pp 15–24 Charytanowicz M et al. (2010) A complete gradient clustering algorithm for features analysis of X-ray images. In: Proc. information technologies in biomedicine. Springer, pp 15–24
44.
go back to reference Altay H, Acar B, Demiroz G, Cekin A (1997) A supervised machine learning algorithm for arrhythmia analysis. In: Proceedings of the computers in cardiology conference Altay H, Acar B, Demiroz G, Cekin A (1997) A supervised machine learning algorithm for arrhythmia analysis. In: Proceedings of the computers in cardiology conference
45.
go back to reference Bareiss E, Ray E, Porter B (1987) Protos: an exemplar-based learning apprentice. In: Proceedings 4th international workshop on machine learning, pp 12–23 Bareiss E, Ray E, Porter B (1987) Protos: an exemplar-based learning apprentice. In: Proceedings 4th international workshop on machine learning, pp 12–23
46.
go back to reference Puerto E, Aguilar J, Reyes J, Sarkar D (2018) Deep learning architecture for the recursive patterns recognition model. J Phys Conf Ser 1126:012035CrossRef Puerto E, Aguilar J, Reyes J, Sarkar D (2018) Deep learning architecture for the recursive patterns recognition model. J Phys Conf Ser 1126:012035CrossRef
47.
go back to reference Van M, Van L (2011) Combining RGB and ToF cameras for real-time 3D hand gesture interaction. In: Proc. IEEE workshop on applications of computer vision, pp 66–72 Van M, Van L (2011) Combining RGB and ToF cameras for real-time 3D hand gesture interaction. In: Proc. IEEE workshop on applications of computer vision, pp 66–72
48.
go back to reference Nguyen A, Yosinski J, Clune J (2015) Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. In: Proc. IEEE conference on computer vision and pattern recognition, pp 427–436 Nguyen A, Yosinski J, Clune J (2015) Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. In: Proc. IEEE conference on computer vision and pattern recognition, pp 427–436
49.
go back to reference Niu X, Suen C (2012) A novel hybrid CNN–SVM classifier for recognizing handwritten digits. Pattern Recogn 45(4):1318–1325CrossRef Niu X, Suen C (2012) A novel hybrid CNN–SVM classifier for recognizing handwritten digits. Pattern Recogn 45(4):1318–1325CrossRef
50.
go back to reference Liu L, Wu Y, Wei W, Cao W, Sahin S, Zhang Q (2018) Benchmarking deep learning frameworks: design considerations, metrics and beyond. In: IEEE 38th international conference on distributed computing systems (ICDCS), pp 1258–1269 Liu L, Wu Y, Wei W, Cao W, Sahin S, Zhang Q (2018) Benchmarking deep learning frameworks: design considerations, metrics and beyond. In: IEEE 38th international conference on distributed computing systems (ICDCS), pp 1258–1269
51.
go back to reference Wu D, Pigou L, Kindermans P, Do-Hoang N, Shao L, Dambre J, Odobez J (2016) Deep dynamic neural networks for multimodal gesture segmentation and recognition. IEEE Trans Pattern Anal Mach Intell 38(8):1583–1597CrossRef Wu D, Pigou L, Kindermans P, Do-Hoang N, Shao L, Dambre J, Odobez J (2016) Deep dynamic neural networks for multimodal gesture segmentation and recognition. IEEE Trans Pattern Anal Mach Intell 38(8):1583–1597CrossRef
52.
go back to reference Wang P, Li W, Liu S, Zhang Y, Gao Z, Ogunbona P (2016) Large-scale continuous gesture recognition using convolutional neural networks. In: 23rd International conference on pattern recognition, pp 13–18 Wang P, Li W, Liu S, Zhang Y, Gao Z, Ogunbona P (2016) Large-scale continuous gesture recognition using convolutional neural networks. In: 23rd International conference on pattern recognition, pp 13–18
54.
go back to reference Köpüklü O, Gunduz A, Kose N, Rigoll G (2019) Real-time hand gesture detection and classification using convolutional neural networks. arXiv preprint arXiv:1901.10323 Köpüklü O, Gunduz A, Kose N, Rigoll G (2019) Real-time hand gesture detection and classification using convolutional neural networks. arXiv preprint arXiv:​1901.​10323
55.
go back to reference Strezoski G, Stojanovski D, Dimitrovsk I, Madjarov G (2016) Hand gesture recognition using deep convolutional neural networks. In: International conference on ICT innovations, pp 49–58 Strezoski G, Stojanovski D, Dimitrovsk I, Madjarov G (2016) Hand gesture recognition using deep convolutional neural networks. In: International conference on ICT innovations, pp 49–58
Metadata
Title
An Ar2p Deep Learning Architecture for the Discovery and the Selection of Features
Authors
E. Puerto
J. Aguilar
R. Vargas
J. Reyes
Publication date
13-07-2019
Publisher
Springer US
Published in
Neural Processing Letters / Issue 1/2019
Print ISSN: 1370-4621
Electronic ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-019-10062-4

Other articles of this Issue 1/2019

Neural Processing Letters 1/2019 Go to the issue