Hang Xu
ABSTRACT
In classification, feature selection mainly aims at reducing the dataset dimensionality and increasing the classification accuracy, which also results in higher computational efficiency than using the original full set of features. Population-based meta-heuristic, evolutionary algorithms have been widely used to solve the bi-objective feature selection problem, which minimizes the number of selected features and the error of classification model. However, most of them are not specifically designed for feature selection, and disregard many of its complex characteristics. In this paper, we propose a generic approach that focuses on improving the initialization effectiveness and offspring quality, in order to boost the performance of existing evolutionary algorithms for bi-objective feature selection. To be more specific, a segmented initialization mechanism is used to enhance the exploration width, while an offspring modification mechanism is proposed to ensure the exploitation depth. Combining them together will make a good trade-off between the diversity and convergence. In the experiments, we plug the proposed approach into three different types of multi-objective evolutionary algorithms, and test them on 18 classification datasets with two widely-used performance metrics. The empirical results prove the significant contribution of the proposed approach on the optimization and classification performance.
- J. Bader and E. Zitzler. 2011. HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization. Evolutionary Computation 19, 1 (March 2011), 45--76. Google Scholar
Digital Library
- Qi Chen, Bing Xue, Ben Niu, and Mengjie Zhang. 2016. Improving generalisation of genetic programming for high-dimensional symbolic regression with feature selection. In 2016 IEEE Congress on Evolutionary Computation (CEC). IEEE, 3793--3800.Google ScholarDigital Library
- Carlos A Coello Coello, Gary B Lamont, David A Van Veldhuizen, et al. 2007. Evolutionary algorithms for solving multi-objective problems. Vol. 5. Springer, New York.Google Scholar
- K. Deb and H. Jain. 2014. An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints. IEEE Transactions on Evolutionary Computation 18, 4 (Aug 2014), 577--601. Google ScholarCross Ref
- K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6, 2 (Apr 2002), 182--197. Google ScholarDigital Library
- Kalyanmoy Deb, Lothar Thiele, Marco Laumanns, and Eckart Zitzler. 2005. Scalable Test Problems for Evolutionary Multiobjective Optimization. Springer, London, 105--145. Google ScholarCross Ref
- Dheeru Dua and Casey Graff. 2017. UCI Machine Learning Repository. (2017). http://archive.ics.uci.edu/mlGoogle Scholar
- Christos Emmanouilidis, Andrew Hunter, and John MacIntyre. 2000. A multiobjective evolutionary setting for feature selection and a commonality-based crossover operator. In Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No. 00TH8512), Vol. 1. IEEE, 309--316.Google ScholarCross Ref
- Raquel Hernández Gómez and Carlos A Coello Coello. 2013. MOMBI: A new metaheuristic for many-objective optimization based on the R2 indicator. In Proceedings of the 2013 IEEE Congress on Evolutionary Computation. Cancun, Mexico, 2488--2495.Google ScholarCross Ref
- Simon Huband, Luigi Barone, Lyndon While, and Phil Hingston. 2005. A scalable multi-objective test problem toolkit. In 3rd International Conference on Evolutionary Multi-Criterion Optimization. Springer, 280--295.Google ScholarDigital Library
- Ron Kohavi and George H John. 1997. Wrappers for feature subset selection. Artificial intelligence 97, 1-2 (1997), 273--324.Google Scholar
- Cosmin Lazar, Jonatan Taminau, Stijn Meganck, David Steenhoff, Alain Coletta, Colin Molter, Virginie de Schaetzen, Robin Duque, Hugues Bersini, and Ann Nowe. 2012. A survey on filter techniques for feature selection in gene expression microarray analysis. IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB) 9, 4 (2012), 1106--1119.Google ScholarDigital Library
- K. Li, K. Deb, Q. Zhang, and S. Kwong. 2015. An Evolutionary Many-Objective Optimization Algorithm Based on Dominance and Decomposition. IEEE Transactions on Evolutionary Computation 19, 5 (Oct 2015), 694--716. Google ScholarDigital Library
- Bach Hoai Nguyen, Bing Xue, Peter Andreae, Hisao Ishibuchi, and Mengjie Zhang. 2019. Multiple Reference Points based Decomposition for Multi-objective Feature Selection in Classification: Static and Dynamic Mechanisms. IEEE Transactions on Evolutionary Computation (2019).Google Scholar
- Hoai Bach Nguyen, Bing Xue, Ivy Liu, Peter Andreae, and Mengjie Zhang. 2016. New mechanism for archive maintenance in PSO-based multi-objective feature selection. Soft Computing 20, 10 (2016), 3927--3946.Google ScholarDigital Library
- Nidamarthi Srinivas and Kalyanmoy Deb. 1994. Muiltiobjective optimization using nondominated sorting in genetic algorithms. Evolutionary computation 2, 3 (1994), 221--248.Google Scholar
- Kay Chen Tan, Eik Fun Khor, and Tong Heng Lee. 2006. Multiobjective evolutionary algorithms and applications. Springer Science & Business Media.Google Scholar
- Ye Tian, Ran Cheng, Xingyi Zhang, and Yaochu Jin. 2017. PlatEMO: A MATLAB Platform for Evolutionary Multi-Objective Optimization. IEEE Computational Intelligence Magazine 12, 4 (2017), 73--87.Google ScholarDigital Library
- Binh Tran, Bing Xue, Mengjie Zhang, and Su Nguyen. 2016. Investigation on particle swarm optimisation for feature selection on high-dimensional data: Local search and selection bias. Connection Science 28, 3 (2016), 270--294.Google ScholarDigital Library
- Leandro D Vignolo, Diego H Milone, and Jacob Scharcanski. 2013. Feature selection for face recognition based on multi-objective evolutionary wrappers. Expert Systems with Applications 40, 13 (2013), 5077--5084.Google ScholarCross Ref
- Lyndon While, Philip Hingston, Luigi Barone, and Simon Huband. 2006. A faster algorithm for calculating Hypervolume. IEEE Transactions on Evolutionary Computation 10, 1 (2006), 29--38.Google ScholarDigital Library
- H. Xu, W. Zeng, X. Zeng, and G. G. Yen. 2019. An Evolutionary Algorithm Based on Minkowski Distance for Many-Objective Optimization. IEEE Transactions on Cybernetics 49, 11 (Nov 2019), 3968--3979. Google ScholarCross Ref
- H. Xu, W. Zeng, X. Zeng, and G. G. Yen. 2020. A Polar-Metric-Based Evolutionary Algorithm. IEEE Transactions on Cybernetics Early Access (2020), 1--12.Google Scholar
- H. Xu, W. Zeng, D. Zhang, and X. Zeng. 2019. MOEA/HD: A Multiobjective Evolutionary Algorithm Based on Hierarchical Decomposition. IEEE Transactions on Cybernetics 49, 2 (Feb 2019), 517--526. Google ScholarCross Ref
- Bing Xue, Mengjie Zhang, and Will N Browne. 2012. Particle swarm optimization for feature selection in classification: A multi-objective approach. IEEE transactions on cybernetics 43, 6 (2012), 1656--1671.Google Scholar
- Bing Xue, Mengjie Zhang, and Will N Browne. 2014. Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms. Applied soft computing 18 (2014), 261--276.Google Scholar
- Bing Xue, Mengjie Zhang, Will N Browne, and Xin Yao. 2015. A survey on evolutionary computation approaches to feature selection. IEEE Transactions on Evolutionary Computation 20, 4 (2015), 606--626.Google ScholarDigital Library
- Q. Zhang and H. Li. 2007. MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition. IEEE Transactions on Evolutionary Computation 11, 6 (Dec 2007), 712--731. Google ScholarDigital Library
- Aimin Zhou, Bo Yang Qu, Hui Li, Shi Zheng Zhao, Ponnuthurai Nagaratnam Suganthan, and Qingfu Zhang. 2011. Multiobjective evolutionary algorithms: A survey of the state of the art. Swarm and Evolutionary Computation 1, 1 (2011), 32--49.Google ScholarCross Ref
- Donald W Zimmerman and Bruno D Zumbo. 1993. Relative power of the Wilcoxon test, the Friedman test, and repeated-measures ANOVA on ranks. The Journal of Experimental Education 62, 1 (1993), 75--86.Google ScholarCross Ref
- Eckart Zitzler, Lothar Thiele, Marco Laumanns, Carlos M Fonseca, and Viviane Grunert Da Fonseca. 2003. Performance assessment of multiobjective optimizers: An analysis and review. IEEE Transactions on Evolutionary Computation 7, 2 (2003), 117--132.Google ScholarDigital Library
Index Terms
- Segmented initialization and offspring modification in evolutionary algorithms for bi-objective feature selection
Recommendations
Differential evolution (DE) for multi-objective feature selection in classification
GECCO Comp '14: Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary ComputationFeature selection has two main conflicting objectives, which are to minimise the number of features and maximise the classification accuracy. Evolutionary computation techniques are particularly suitable for solving mult-objective tasks. Based on ...
A Clustering Strategy-Based Evolutionary Algorithm for Feature Selection in Classification
Advances and Trends in Artificial Intelligence. Theory and ApplicationsAbstractFeature selection is a technique used in data pre-processing to select the most relevant subset of features from a larger set, with the goal of improving classification performance. Evolutionary algorithms have been commonly proposed to solve ...
A Population Initialization Method Based on Similarity and Mutual Information in Evolutionary Algorithm for Bi-objective Feature Selection
Feature selection (FS) is an important data pre-processing technique in classification. It aims to remove redundant and irrelevant features from the data, which reduces the dimensionality of data and improves the performance of the classifier. Thus, FS is ...
Comments