Skip to main content
Top
Published in: International Journal of Machine Learning and Cybernetics 1/2021

24-06-2020 | Original Article

A new framework of multi-objective evolutionary algorithms for feature selection and multi-label classification of video data

Authors: Gizem Nur Karagoz, Adnan Yazici, Tansel Dokeroglu, Ahmet Cosar

Published in: International Journal of Machine Learning and Cybernetics | Issue 1/2021

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

There are few studies in the literature to address the multi-objective multi-label feature selection for the classification of video data using evolutionary algorithms. Selecting the most appropriate subset of features is a significant problem while maintaining/improving the accuracy of the prediction results. This study proposes a framework of parallel multi-objective Non-dominated Sorting Genetic Algorithms (NSGA-II) for exploring a Pareto set of non-dominated solutions. The subsets of non-dominated features are extracted and validated by multi-label classification techniques, Binary Relevance (BR), Classifier Chains (CC), Pruned Sets (PS), and Random k-Labelset (RAkEL). Base classifiers such as Support Vector Machines (SVM), J48-Decision Tree (J48), and Logistic Regression (LR) are performed in the classification phase of the algorithms. Comprehensive experiments are carried out with local feature descriptors extracted from two multi-label data sets, the well-known MIR-Flickr dataset and a Wireless Multimedia Sensor (WMS) dataset that we have generated from our video recordings. The prediction accuracy levels are improved by 6.36% and 25.7% for the MIR-Flickr and WMS datasets respectively while the number of features is significantly reduced. The results verify that the algorithms presented in this new framework outperform the state-of-the-art algorithms.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Show more products
Literature
1.
go back to reference Alpaydin E (2014) Introduction to machine learning. MIT Press, LondonMATH Alpaydin E (2014) Introduction to machine learning. MIT Press, LondonMATH
2.
go back to reference Miao J, Niu L (2016) A survey on feature selection. Proc Comput Sci 91:919–926CrossRef Miao J, Niu L (2016) A survey on feature selection. Proc Comput Sci 91:919–926CrossRef
3.
go back to reference Srivastava MS, Joshi MN, Gaur M (2014) A review paper on feature selection methodologies and their applications. IJCSNS 14(5):78 Srivastava MS, Joshi MN, Gaur M (2014) A review paper on feature selection methodologies and their applications. IJCSNS 14(5):78
4.
go back to reference Bach FR (2008) Bolasso: model consistent lasso estimation through the bootstrap. In: Proceedings of the 25th international conference on machine learning, pp 33–40 Bach FR (2008) Bolasso: model consistent lasso estimation through the bootstrap. In: Proceedings of the 25th international conference on machine learning, pp 33–40
5.
go back to reference Cerri R, Basgalupp MP, Barros RC, de Carvalho AC (2019) Inducing hierarchical multi-label classification rules with genetic algorithms. Appl Soft Comput 77:584–604CrossRef Cerri R, Basgalupp MP, Barros RC, de Carvalho AC (2019) Inducing hierarchical multi-label classification rules with genetic algorithms. Appl Soft Comput 77:584–604CrossRef
6.
go back to reference Gargiulo F, Silvestri S, Ciampi M, De Pietro G (2019) Deep neural network for hierarchical extreme multi-label text classification. Appl Soft Comput 79:125–138CrossRef Gargiulo F, Silvestri S, Ciampi M, De Pietro G (2019) Deep neural network for hierarchical extreme multi-label text classification. Appl Soft Comput 79:125–138CrossRef
7.
go back to reference Deb K, Pratap A, Agarwal S, Meyarivan TAMT (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2):182–197CrossRef Deb K, Pratap A, Agarwal S, Meyarivan TAMT (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2):182–197CrossRef
8.
go back to reference Huiskes MJ, Lew MS (2008) The MIR flickr retrieval evaluation. In: Proceedings of the 1st ACM international conference on multimedia information retrieval, pp 39–43 Huiskes MJ, Lew MS (2008) The MIR flickr retrieval evaluation. In: Proceedings of the 1st ACM international conference on multimedia information retrieval, pp 39–43
9.
go back to reference Costa AF, Traina AJM, Traina Jr C (2014) MFS-Map: efficient context and content combination to annotate images. In: Proceedings of the 29th annual ACM symposium on applied computing, pp 945–950 Costa AF, Traina AJM, Traina Jr C (2014) MFS-Map: efficient context and content combination to annotate images. In: Proceedings of the 29th annual ACM symposium on applied computing, pp 945–950
10.
go back to reference Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3(Mar):1157–1182MATH Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3(Mar):1157–1182MATH
11.
go back to reference Yin J, Tao T, Xu J (2015) A multi-label feature selection algorithm based on multi-objective optimization. In: 2015 international joint conference on neural networks (IJCNN), pp 1–7. IEEE, New York Yin J, Tao T, Xu J (2015) A multi-label feature selection algorithm based on multi-objective optimization. In: 2015 international joint conference on neural networks (IJCNN), pp 1–7. IEEE, New York
12.
go back to reference Zhang Y, Gong DW, Sun XY, Guo YN (2017) A PSO-based multi-objective multi-label feature selection method in classification. Sci Rep 7(1):1–12CrossRef Zhang Y, Gong DW, Sun XY, Guo YN (2017) A PSO-based multi-objective multi-label feature selection method in classification. Sci Rep 7(1):1–12CrossRef
13.
go back to reference Dokeroglu T, Sevinc E, Kucukyilmaz T, Cosar A (2019) A survey on new generation metaheuristic algorithms. Comput Ind Eng 137:106040CrossRef Dokeroglu T, Sevinc E, Kucukyilmaz T, Cosar A (2019) A survey on new generation metaheuristic algorithms. Comput Ind Eng 137:106040CrossRef
14.
go back to reference Vaishali R, Sasikala R, Ramasubbareddy S, Remya S, Nalluri S (2017) Genetic algorithm based feature selection and MOE Fuzzy classification algorithm on Pima Indians Diabetes dataset. In: 2017 international conference on computing networking and informatics (ICCNI), pp 1–5. IEEE, New York Vaishali R, Sasikala R, Ramasubbareddy S, Remya S, Nalluri S (2017) Genetic algorithm based feature selection and MOE Fuzzy classification algorithm on Pima Indians Diabetes dataset. In: 2017 international conference on computing networking and informatics (ICCNI), pp 1–5. IEEE, New York
15.
go back to reference Vignolo LD, Milone DH, Scharcanski J (2013) Feature selection for face recognition based on multi-objective evolutionary wrappers. Expert Syst Appl 40(13):5077–5084CrossRef Vignolo LD, Milone DH, Scharcanski J (2013) Feature selection for face recognition based on multi-objective evolutionary wrappers. Expert Syst Appl 40(13):5077–5084CrossRef
16.
go back to reference Labani M, Moradi P, Jalili M, Yu X (2017) An evolutionary based multi-objective filter approach for feature selection. In: 2017 world congress on computing and communication technologies (WCCCT), pp 151–154. IEEE, New York Labani M, Moradi P, Jalili M, Yu X (2017) An evolutionary based multi-objective filter approach for feature selection. In: 2017 world congress on computing and communication technologies (WCCCT), pp 151–154. IEEE, New York
17.
go back to reference Zhang P, Gao W, Liu G (2018) Feature selection considering weighted relevancy. Appl Intell 48(12):4615–4625CrossRef Zhang P, Gao W, Liu G (2018) Feature selection considering weighted relevancy. Appl Intell 48(12):4615–4625CrossRef
18.
go back to reference Deniz A, Kiziloz HE, Dokeroglu T, Cosar A (2017) Robust multiobjective evolutionary feature subset selection algorithm for binary classification using machine learning techniques. Neurocomputing 241:128–146CrossRef Deniz A, Kiziloz HE, Dokeroglu T, Cosar A (2017) Robust multiobjective evolutionary feature subset selection algorithm for binary classification using machine learning techniques. Neurocomputing 241:128–146CrossRef
19.
go back to reference Saroj J (2014) Multi-objective genetic algorithm approach to feature subset optimization. In: Proceeding of the IEEE international advance computing conference (IACC), pp 544–548 Saroj J (2014) Multi-objective genetic algorithm approach to feature subset optimization. In: Proceeding of the IEEE international advance computing conference (IACC), pp 544–548
20.
go back to reference Hamdani TM, Won JM, Alimi AM, Karray F (2007) Multi-objective feature selection with NSGA II. In: International conference on adaptive and natural computing algorithms, pp 240–247. Springer, Berlin Hamdani TM, Won JM, Alimi AM, Karray F (2007) Multi-objective feature selection with NSGA II. In: International conference on adaptive and natural computing algorithms, pp 240–247. Springer, Berlin
21.
go back to reference Khan MA, Ekbal A, Menca EL, Furnkranz J (2017) Multi-objective optimisation-based feature selection for multi-label classification. In: International conference on applications of natural language to information systems, pp 38–41. Springer, New York Khan MA, Ekbal A, Menca EL, Furnkranz J (2017) Multi-objective optimisation-based feature selection for multi-label classification. In: International conference on applications of natural language to information systems, pp 38–41. Springer, New York
22.
go back to reference Li S, Wu H, Wan D, Zhu J (2011) An effective feature selection method for hyperspectral image classification based on genetic algorithm and support vector machine. Knowl-Based Syst 24(1):40–48CrossRef Li S, Wu H, Wan D, Zhu J (2011) An effective feature selection method for hyperspectral image classification based on genetic algorithm and support vector machine. Knowl-Based Syst 24(1):40–48CrossRef
23.
go back to reference Gaspar-Cunha A (2010) Feature selection using multi-objective evolutionary algorithms: application to cardiac SPECT diagnosis. In: Advances in bioinformatics, pp 85-92. Springer, Berlin Gaspar-Cunha A (2010) Feature selection using multi-objective evolutionary algorithms: application to cardiac SPECT diagnosis. In: Advances in bioinformatics, pp 85-92. Springer, Berlin
24.
go back to reference Xue B, Zhang M, Browne WN (2012) Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans Cybern 43(6):1656–1671CrossRef Xue B, Zhang M, Browne WN (2012) Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans Cybern 43(6):1656–1671CrossRef
25.
go back to reference Zhang Y, Gong DW, Cheng J (2015) Multi-objective particle swarm optimization approach for cost-based feature selection in classification. IEEE/ACM Trans Comput Biol Bioinform 14(1):64–75CrossRef Zhang Y, Gong DW, Cheng J (2015) Multi-objective particle swarm optimization approach for cost-based feature selection in classification. IEEE/ACM Trans Comput Biol Bioinform 14(1):64–75CrossRef
26.
go back to reference Tangherloni A, Spolaor S, Cazzaniga P, Besozzi D, Rundo L, Mauri G, Nobile MS (2019) Biochemical parameter estimation vs. benchmark functions: a comparative study of optimization performance and representation design. Appl Soft Comput 81:105494CrossRef Tangherloni A, Spolaor S, Cazzaniga P, Besozzi D, Rundo L, Mauri G, Nobile MS (2019) Biochemical parameter estimation vs. benchmark functions: a comparative study of optimization performance and representation design. Appl Soft Comput 81:105494CrossRef
27.
go back to reference Nalluri MR, Kannan K, Gao XZ, Roy DS (2019) Multiobjective hybrid monarch butterfly optimization for imbalanced disease classification problem. Int J Mach Learn Cybern 1–29 Nalluri MR, Kannan K, Gao XZ, Roy DS (2019) Multiobjective hybrid monarch butterfly optimization for imbalanced disease classification problem. Int J Mach Learn Cybern 1–29
28.
go back to reference Rundo L, Tangherloni A, Nobile MS, Militello C, Besozzi D, Mauri G, Cazzaniga P (2019) MedGA: a novel evolutionary method for image enhancement in medical imaging systems. Expert Syst Appl 119:387–399CrossRef Rundo L, Tangherloni A, Nobile MS, Militello C, Besozzi D, Mauri G, Cazzaniga P (2019) MedGA: a novel evolutionary method for image enhancement in medical imaging systems. Expert Syst Appl 119:387–399CrossRef
29.
go back to reference Rundo L, Tangherloni A, Cazzaniga P, Nobile MS, Russo G, Gilardi MC, Militello C (2019) A novel framework for MR image segmentation and quantification by using MedGA. Comput Methods Programs Biomed 176:159–172CrossRef Rundo L, Tangherloni A, Cazzaniga P, Nobile MS, Russo G, Gilardi MC, Militello C (2019) A novel framework for MR image segmentation and quantification by using MedGA. Comput Methods Programs Biomed 176:159–172CrossRef
30.
go back to reference Thabtah FA, Cowling P, Peng Y (2004) MMAC: a new multi-class, multi-label associative classification approach. In: Fourth IEEE international conference on data mining (ICDM’04), pp 217–224. IEEE, New York Thabtah FA, Cowling P, Peng Y (2004) MMAC: a new multi-class, multi-label associative classification approach. In: Fourth IEEE international conference on data mining (ICDM’04), pp 217–224. IEEE, New York
31.
go back to reference Charte F, del Jesus MJ, Rivera AJ (2016) Multilabel classification: problem analysis, metrics and techniques. Springer, New York Charte F, del Jesus MJ, Rivera AJ (2016) Multilabel classification: problem analysis, metrics and techniques. Springer, New York
32.
go back to reference Read J, Pfahringer B, Holmes G (2008) Multi-label classification using ensembles of pruned sets. In: 2008 eighth IEEE international conference on data mining, pp 995–1000. IEEE, New York Read J, Pfahringer B, Holmes G (2008) Multi-label classification using ensembles of pruned sets. In: 2008 eighth IEEE international conference on data mining, pp 995–1000. IEEE, New York
33.
go back to reference Tsoumakas G, Vlahavas I (2007) Random k-labelsets: an ensemble method for multilabel classification. In: European conference on machine learning, pp 406–417. Springer, Berlin Tsoumakas G, Vlahavas I (2007) Random k-labelsets: an ensemble method for multilabel classification. In: European conference on machine learning, pp 406–417. Springer, Berlin
34.
go back to reference Lobato FS, Steffen V (2017) Multi-objective optimization problem. In: Multi-objective optimization problems, pp 9–23. Springer, Cham Lobato FS, Steffen V (2017) Multi-objective optimization problem. In: Multi-objective optimization problems, pp 9–23. Springer, Cham
35.
go back to reference Zhou A, Qu BY, Li H, Zhao SZ, Suganthan PN, Zhang Q (2011) Multiobjective evolutionary algorithms: a survey of the state of the art. Swarm Evol Comput 1(1):32–49CrossRef Zhou A, Qu BY, Li H, Zhao SZ, Suganthan PN, Zhang Q (2011) Multiobjective evolutionary algorithms: a survey of the state of the art. Swarm Evol Comput 1(1):32–49CrossRef
36.
go back to reference Stadler W (1979) A survey of multicriteria optimization or the vector maximum problem, part I: 1776–1960. J Optim Theory Appl 29(1):1–52MathSciNetCrossRef Stadler W (1979) A survey of multicriteria optimization or the vector maximum problem, part I: 1776–1960. J Optim Theory Appl 29(1):1–52MathSciNetCrossRef
37.
go back to reference Miettinen K (2012) Nonlinear multiobjective optimization, vol 12. Springer, New YorkMATH Miettinen K (2012) Nonlinear multiobjective optimization, vol 12. Springer, New YorkMATH
38.
go back to reference Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333MathSciNetCrossRef Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333MathSciNetCrossRef
39.
go back to reference Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297MATH Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297MATH
40.
go back to reference Zeng ZQ, Yu HB, Xu HR, Xie YQ, Gao J (2008) Fast training support vector machines using parallel sequential minimal optimization. In: 2008 3rd international conference on intelligent system and knowledge engineering, vol 1, pp 997–1001. IEEE, New York Zeng ZQ, Yu HB, Xu HR, Xie YQ, Gao J (2008) Fast training support vector machines using parallel sequential minimal optimization. In: 2008 3rd international conference on intelligent system and knowledge engineering, vol 1, pp 997–1001. IEEE, New York
41.
go back to reference Bishop CM (2006) Pattern recognition and machine learning. Springer, New YorkMATH Bishop CM (2006) Pattern recognition and machine learning. Springer, New YorkMATH
42.
go back to reference Bhargava N, Sharma G, Bhargava R, Mathuria M (2013) Decision tree analysis on j48 algorithm for data mining. In: Proceedings of international journal of advanced research in computer science and software engineering, vol 3(6) Bhargava N, Sharma G, Bhargava R, Mathuria M (2013) Decision tree analysis on j48 algorithm for data mining. In: Proceedings of international journal of advanced research in computer science and software engineering, vol 3(6)
43.
go back to reference Kaur G, Chhabra A (2014) Improved J48 classification algorithm for the prediction of diabetes. Int J Comput Appl 98(22) Kaur G, Chhabra A (2014) Improved J48 classification algorithm for the prediction of diabetes. Int J Comput Appl 98(22)
44.
go back to reference Dokeroglu T, Sevinc E (2019) Evolutionary parallel extreme learning machines for the data classification problem. Comput Ind Eng 130:237–249CrossRef Dokeroglu T, Sevinc E (2019) Evolutionary parallel extreme learning machines for the data classification problem. Comput Ind Eng 130:237–249CrossRef
45.
go back to reference Cantu-Paz E (1998) A survey of parallel genetic algorithms. Calc Paralleles Reseaux Syst Repartis 10(2):141–171 Cantu-Paz E (1998) A survey of parallel genetic algorithms. Calc Paralleles Reseaux Syst Repartis 10(2):141–171
46.
47.
go back to reference Costa AF, Humpire-Mamani G, Traina AJM (2012) An efficient algorithm for fractal analysis of textures. In: 2012 25th SIBGRAPI conference on graphics, patterns and images, pp 39–46. IEEE, New York Costa AF, Humpire-Mamani G, Traina AJM (2012) An efficient algorithm for fractal analysis of textures. In: 2012 25th SIBGRAPI conference on graphics, patterns and images, pp 39–46. IEEE, New York
48.
go back to reference Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110CrossRef Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110CrossRef
49.
go back to reference Bradski G, Kaehler A (2008) Learning OpenCV: computer vision with the OpenCV library. O’Reilly Media, Inc., Massachusetts Bradski G, Kaehler A (2008) Learning OpenCV: computer vision with the OpenCV library. O’Reilly Media, Inc., Massachusetts
50.
go back to reference Read J, Reutemann P, Pfahringer B, Holmes G (2016) Meka: a multi-label/multi-target extension to weka. J Mach Learn Res 17(1):667–671MathSciNetMATH Read J, Reutemann P, Pfahringer B, Holmes G (2016) Meka: a multi-label/multi-target extension to weka. J Mach Learn Res 17(1):667–671MathSciNetMATH
51.
go back to reference Tan Q, Yu G, Domeniconi C, Wang J, Zhang Z (2018) Incomplete multi-view weak-label learning. In: IJCAI, pp 2703–2709 Tan Q, Yu G, Domeniconi C, Wang J, Zhang Z (2018) Incomplete multi-view weak-label learning. In: IJCAI, pp 2703–2709
52.
go back to reference Pearson K (1901) LIII. On lines and planes of closest fit to systems of points in space. Lond Edinb Dublin Philos Mag J Sci 2(11):559–572CrossRef Pearson K (1901) LIII. On lines and planes of closest fit to systems of points in space. Lond Edinb Dublin Philos Mag J Sci 2(11):559–572CrossRef
53.
go back to reference Jolliffe IT, Cadima J (2016) Principal component analysis: a review and recent developments. Philos Trans R Soc A Math Phys Eng Sci 374(2065):20150202MathSciNetCrossRef Jolliffe IT, Cadima J (2016) Principal component analysis: a review and recent developments. Philos Trans R Soc A Math Phys Eng Sci 374(2065):20150202MathSciNetCrossRef
Metadata
Title
A new framework of multi-objective evolutionary algorithms for feature selection and multi-label classification of video data
Authors
Gizem Nur Karagoz
Adnan Yazici
Tansel Dokeroglu
Ahmet Cosar
Publication date
24-06-2020
Publisher
Springer Berlin Heidelberg
Published in
International Journal of Machine Learning and Cybernetics / Issue 1/2021
Print ISSN: 1868-8071
Electronic ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-020-01156-w

Other articles of this Issue 1/2021

International Journal of Machine Learning and Cybernetics 1/2021 Go to the issue