Top

International Journal of Machine Learning and Cybernetics

Published in:

01-11-2021 | Original Article

A new two-stage hybrid feature selection algorithm and its application in Chinese medicine

Authors: Zhiqin Li, Jianqiang Du, Bin Nie, Wangping Xiong, Guoliang Xu, Jigen Luo

Published in: International Journal of Machine Learning and Cybernetics | Issue 5/2022

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

High-dimensional small sample data are prone to the curse of dimensionality and overfitting and contain many irrelevant and redundant features. In order to solve these feature selection problems, a new Two-stage Hybrid Feature Selection Algorithm (Ts-HFSA) is proposed. The first stage uses the Filter method combined with the Wrapper method to adaptively remove irrelevant features. In the second stage, a De-redundancy Algorithm of Fusing Approximate Markov Blanket with L1 Regular Term (DA²MBL1) is used to solve the AMB’s problem of information loss when deleting redundant features and potential redundancy in the subset of features obtained by AMB. The experimental results on multiple UCI public data sets and datasets from the material foundation of Chinese medicine showed that the Ts-HFSA better deleted irrelevant features and redundant features, found smaller and higher quality feature subsets, and improved stability, indicating that it offers more advantages than AMB, FCBF, RF, GBDT, XGBoost, Lasso, and CI_AMB. Moreover, in the face of data of the material foundation of Chinese medicine, with higher feature dimensions and fewer sample sizes, Ts-HFSA performed better, which can also improve the precision of the model after greatly reducing the dimension. The results indicated that Ts-HFSA is an effective method for feature selection of high-dimensional small samples and an excellent research method for the material foundation of Chinese medicine.

previous article Towards unified on-road object detection and depth estimation from a single image

next article Ranking defects and solving countermeasures for Pythagorean fuzzy sets with hesitant degree

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

ATZelectronics worldwide

ATZlectronics worldwide is up-to-speed on new trends and developments in automotive electronics on a scientific level with a high depth of information.

Order your 30-days-trial for free and without any commitment.

inform now

ATZelektronik

Die Fachzeitschrift ATZelektronik bietet für Entwickler und Entscheider in der Automobil- und Zulieferindustrie qualitativ hochwertige und fundierte Informationen aus dem gesamten Spektrum der Pkw- und Nutzfahrzeug-Elektronik.

Lassen Sie sich jetzt unverbindlich 2 kostenlose Ausgabe zusenden.

inform now

Xu D, Zhang J, Xu H et al (2020) Multi-scale supervised clustering-based feature selection for tumor classification and identification of biomarkers and targets on genomic data. BMC Genomics 21(1):650MathSciNetCrossRef

Zhang P, Gao W (2020) Feature selection considering uncertainty change ratio of the class label. Appl Soft Comput 95:106537CrossRef

Ferdinando DM, Sabrina S (2020) Balancing the user-driven feature selection and their incidence in the clustering structure formation. Appl Soft Comput 98:106854

Al-Rimy BAS, Maarof MA, Shaid SZM et al (2021) Redundancy coefficient gradual up-weighting-based mutual information feature selection technique for crypto-ransomware early detection. Fut Gener Comput Syst 115:641–658CrossRef

Rostami O, Kaveh M (2021) Optimal feature selection for SAR image classification using biogeography-based optimization (BBO), artificial bee colony (ABC) and support vector machine (SVM): a combined approach of optimization and machine learning. Comput Geosci. https://doi.org/10.1007/s10596-020-10030-1CrossRefMATH

Yildirim S, Kaya Y, Kl F (2021) A modified feature selection method based on metaheuristic algorithms for speech emotion recognition. Appl Acoust. https://doi.org/10.1016/j.apacoust.2020.107721CrossRef

Zhou Y, Zhang W, Kang J et al (2020) A problem-specific non-dominated sorting genetic algorithm for supervised feature selection. Inf Sci 547:841–859MathSciNetCrossRef

Han M, Liu X (2012) Forward feature selection based on approximate Markov blanket. In: International conference on advances in neural networks. Springer

Yu L, Liu H (2004) Efficient feature selection via analysis of relevance and redundancy. J Mach Learn Res 5:1205–1224MathSciNetMATH

10.

Kira K, Rendell LA (1992) The feature selection problem: traditional methods and a new algorithm. In: Proceedings of the 10th national conference on artificial intelligence, San Jose, CA, July 12–16. AAAI Press, pp 129–134

11.

John G H, Kohavi R, Pfleger K (1994) Irrelevant features and the subset selection problem. In: Proceedings of the eleventh international conference, Rutgers University, New Brunswick, NJ, July 10–13, 1994:121–129. https://doi.org/10.1016/B978-1-55860-335-6.50023-4.

12.

Koller D, Sahami M (1996) Toward optimal feature selection. In: Proceedings of the 20th international conference on machine learning, Bari, Italy, pp 284–292

13.

Dash M, Liu H (1997) Feature selection for classification. Intell Data Anal 1:131–156CrossRef

14.

Reshef DN, Reshef YA, Finucane HK et al (2011) Detecting novel associations in large data sets. Science 334(6062):1518–1524CrossRef

15.

Peng H, Long F, Ding C (2005) Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 27(8):1226–1238CrossRef

16.

Wang X, Yan Y, Ma X (2020) Feature selection method based on differential correlation information entropy. Neural Process Lett. https://doi.org/10.1007/s11063-020-10307-7CrossRef

17.

Emary E, Zawbaa HM, Hassanien AE (2016) Binary ant lion approaches for feature selection. Neurocomputing 213:54–65CrossRef

18.

Zawbaa HM, Emary E (2018) Applications of flower pollination algorithm in feature selection and knapsack problems. Nat Inspired Algorithms Appl Optim 744:217–243CrossRef

19.

Mafarja M, Mirjalili S (2018) Whale optimization approaches for wrapper feature selection. Appl Soft Comput 62:441–453CrossRef

20.

Dhiman G, Oliva D, Kaur A et al (2021) BEPO: A novel binary emperor penguin optimizer for automatic feature selection. Knowl Based Syst 211:106560CrossRef

21.

Abdel-Basset M, El-Shahat D, El-henawy I et al (2020) A new fusion of grey wolf optimizer algorithm with a two-phase mutation for feature selection. Expert Syst Appl. https://doi.org/10.1016/j.eswa.2019.112824cCrossRef

22.

Zhang J et al (2019) A new hybrid filter/wrapper algorithm for feature selection in classification. Anal Chim Acta 1080:43–54CrossRef

23.

Huang Z, Yang C, Zhou X et al (2019) A hybrid feature selection method based on binary state transition algorithm and ReliefF. IEEE J Biomed Health Inform 23:1888–1898CrossRef

24.

Guo W, Li B, Shen S et al (2019) An intelligent grinding burn detection system based on two-stage feature selection and stacked sparse autoencoder. Int J Adv Manuf Technol 103:2837–2847CrossRef

25.

Chaudhuri A, Sahu TP (2021) A hybrid feature selection method based on Binary Jaya algorithm for micro-array data classification. Comput Electr Eng 90:106963CrossRef

26.

Ruiz R, Riquelme JC, Aguilar-Ruiz JS (2006) Incremental wrapper-based gene selection from microarray data for cancer classiflcation. Pattern Recogn 39(12):2383–2392CrossRef

27.

Song Q, Ni J, Wang G (2013) A fast clustering-based feature subset selection algorithm for high-dimensional data. IEEE Trans Knowl Data Eng 25(1):1–14CrossRef

28.

Zhang L, Wang C, Guo W (2018) A feature selection algorithm for maximum relevance minimum redundancy using approximate Markov blanket. J Xi’an Jiaotong Univ 52(10):147–151

29.

Zhang Y, Zhang Z (2012) Feature subset selection with cumulate conditional mutual information minimization. Expert Syst Appl 39(5):6078–6088CrossRef

30.

Huang C, Li K, Du J et al (2020) Research on hybrid feature selection method based on iterative approximation Markov blanket. Comput Math Methods Med 2020:1–11. https://doi.org/10.1155/2020/8308173CrossRef

31.

Wang H, Ling Z, Yu K, Wu X (2020) Towards efficient and effective discovery of Markov blankets for feature selection. Inf Sci. https://doi.org/10.1016/j.ins.2019.09.010CrossRef

32.

Hua Z, Zhou J, Hua Y et al (2020) Strong approximate Markov blanket and its application on filter-based feature selection. Appl Soft Comput 87:105957CrossRef

33.

Davies S, Russell S (1994) NP-completeness of searches for smallest possible feature sets. In: Proceedings of the 1994 AAAI fall symposium on relevance, pp 37–39

34.

Tibshirani R (1996) Regression shrinkage and selection via the Lasso. J R Stat Soc Ser B Methodol 58(1):267–288MathSciNetMATH

35.

Huang C, Du J, Nie B et al (2019) Feature selection method based on partial least squares and analysis of traditional chinese medicine data. Comput Math Methods Med 2019:1–11. https://doi.org/10.1155/2019/9580126MathSciNetCrossRefMATH

36.

Shao F, Li K, Xu X (2016) Railway accidents analysis based on the improved algorithm of the maximal information coefficient. Intell Data Anal 20(3):597–613CrossRef

37.

Sun G, Song Z, Liu J et al (2017) Feature selection method based on maximum information coefficient and approximate Markov blanket. Acta Automatica Sinica 43(5):795–805

Title: A new two-stage hybrid feature selection algorithm and its application in Chinese medicine
Authors: Zhiqin Li
Jianqiang Du
Bin Nie
Wangping Xiong
Guoliang Xu
Jigen Luo
Publication date: 01-11-2021
Publisher: Springer Berlin Heidelberg
Published in: International Journal of Machine Learning and Cybernetics / Issue 5/2022
Print ISSN: 1868-8071
Electronic ISSN: 1868-808X
DOI: https://doi.org/10.1007/s13042-021-01445-y

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

ATZelectronics worldwide

ATZelektronik

Other articles of this Issue 5/2022

Correntropy metric-based robust low-rank subspace clustering for motion segmentation

Hypergraph based semi-supervised support vector machine for binary and multi-category classifications

A novel method for image segmentation: two-stage decoding network with boundary attention

Deep semantic-aware network for zero-shot visual urban perception

A further study on optimal scale selection in dynamic multi-scale decision information systems based on sequential three-way decisions

TSLOD: a coupled generalized subsequence local outlier detection model for multivariate time series