Skip to main content
SpringerLink
Log in

Feature subset selection based on fuzzy entropy measures for handling classification problems

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In this paper, we present a new method for dealing with feature subset selection based on fuzzy entropy measures for handling classification problems. First, we discretize numeric features to construct the membership function of each fuzzy set of a feature. Then, we select the feature subset based on the proposed fuzzy entropy measure focusing on boundary samples. The proposed method can select relevant features to get higher average classification accuracy rates than the ones selected by the MIFS method (Battiti, R. in IEEE Trans. Neural Netw. 5(4):537–550, 1994), the FQI method (De, R.K., et al. in Neural Netw. 12(10):1429–1455, 1999), the OFEI method, Dong-and-Kothari’s method (Dong, M., Kothari, R. in Pattern Recognit. Lett. 24(9):1215–1225, 2003) and the OFFSS method (Tsang, E.C.C., et al. in IEEE Trans. Fuzzy Syst. 11(2):202–213, 2003).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Baim PW (1988) A method for attribute selection in inductive learning systems. IEEE Trans Pattern Anal Mach Intell 10(6):888–896

    Article  Google Scholar 

  2. Battiti R (1994) Using mutual information for selecting features in supervised neural net learning. IEEE Trans Neural Netw 5(4):537–550

    Article  Google Scholar 

  3. Caruana R, Freitag D (1994) Greedy attribute selection. In: Proceedings of international conference on machine learning, New Brunswick, NJ, pp 28–36

  4. Chaikla N, Qi Y (1999) Genetic algorithms in feature selection. In: Proceedings of the 1999 IEEE international conference on systems, man, and cybernetics, Tokyo, Japan, vol 5, pp 538–540

  5. Chen SM, Chang CH (2005) A new method to construct membership functions and generate weighted fuzzy rules from training instances. Cybern Syst 36(4):397–414

    Article  MATH  MathSciNet  Google Scholar 

  6. Chen SM, Chen YC (2002) Automatically constructing membership functions and generating fuzzy rules using genetic algorithms. Cybern Syst 33(8):841–862

    Article  Google Scholar 

  7. Chen SM, Kao CH, Yu CH (2002) Generating fuzzy rules from training data containing noise for handling classification problems. Cybern Syst 33(7):723–748

    Article  MATH  Google Scholar 

  8. Chen SM, Shie JD (2005) A new method for feature subset selection for handling classification problems. In: Proceedings of the 2005 IEEE international conference on fuzzy systems, Reno, NV, pp 183-188

  9. Chen SM (1988) A new approach to handling fuzzy decision-making problems. IEEE Trans Syst Man Cybern 18(6):1012–1016

    Article  MATH  Google Scholar 

  10. De RK, Basak J, Pal SK (1999) Neuro-fuzzy feature evaluation with theoretical analysis. Neural Netw 12(10):1429–1455

    Article  Google Scholar 

  11. De RK, Pal NR, Pal SK (1997) Feature analysis: neural network and fuzzy set theoretic approaches. Pattern Recognit 30(10):1579–1590

    Article  MATH  Google Scholar 

  12. Dong M, Kothari R (2003) Feature subset selection using a new definition of classifiability. Pattern Recognit Lett 24(9):1215–1225

    Article  MATH  Google Scholar 

  13. Fodor IK, Kamath C (2002) Dimension reduction techniques and the classification of bent double galaxies. Comput Stat Data Anal 41(1):91–122

    Article  MATH  MathSciNet  Google Scholar 

  14. Hartigan JA, Wong MA (1979) A k-means clustering algorithm. J Roy Stat Soc Ser C 28(1):100–108

    MATH  Google Scholar 

  15. John GH, Kohavi R, Pfleger K (1994) Irrelevant features and the subset selection problem. In: Proceedings of the eleventh international conference on machine learning, San Francisco, CA, pp 121–129

  16. Kosko B (1986) Fuzzy entropy and conditioning. Inf Scie 40(2):165–174

    Article  MATH  MathSciNet  Google Scholar 

  17. Landwehr N, Hall M, Frank E (2003) Logistic model trees. In: Proceedings of the 14th European conference on machine learning, Cavtat-Dubrovnik, Croatia, pp 241–252

  18. Lee HM, Chen CM, Chen JM, Jou YL (2001) An efficient fuzzy classifier with feature selection based on fuzzy entropy. IEEE Trans Syst Man Cybern Part B Cybern 31(3):426–432

    Article  Google Scholar 

  19. Luca AD, Termini S (1972) A definition of a non-probabilistic entropy in the setting of fuzzy set theory. Inf Control 20(4):301–312

    Article  MATH  Google Scholar 

  20. Platt JC (1999) Using analytic QP and sparseness to speed training of support vector machines. In: Proceedings of the thirteenth annual conference on neural information processing systems, Denver, CO, pp 557–563

  21. Pal SK, De RK, Bsask J (2000) Unsupervised feature selection: a neuro-fuzzy approach. IEEE Trans Neural Netw 11(2):366–376

    Article  Google Scholar 

  22. Quinlan JR (1993) C4.5: programs for machine learning. Kaufmann, San Francisco

    Google Scholar 

  23. Schaffer C (1993) Overfitting avoidance as bias. Mach Learn 10(2):153–178

    MathSciNet  Google Scholar 

  24. Shannon CE (1948) A mathematical theory of communication. Bell Syst Techn J 27(3):379–423

    MathSciNet  Google Scholar 

  25. Shie JD, Chen SM (2006) A new approach for handling classification problems based on fuzzy information gain measures. In: Proceedings of the 2006 IEEE international conference on fuzzy systems, Vancouver, BC, Canada, pp 5427–5434

  26. Tsang ECC, Yeung DS, Wang XZ (2003) OFFSS: optimal fuzzy-valued feature subset selection. IEEE Trans Fuzzy Syst 11(2):202–213

    Article  Google Scholar 

  27. Zadeh LA (1965) Fuzzy sets. Inf Control 8:338–353

    Article  MATH  MathSciNet  Google Scholar 

  28. Zadeh LA (1965) Probability measures of fuzzy events. J Math Anal Appl 23(2):421–427

    Article  MathSciNet  Google Scholar 

  29. Zadeh LA (1975) The concept of linguistic variable and its application to approximate reasoning, I. Inf Sci 8(3):199–249

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, 43, Section 4, Keelung Road, Taipei, 106, Taiwan

    Jen-Da Shie & Shyi-Ming Chen

Authors
  1. Jen-Da Shie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shyi-Ming Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shyi-Ming Chen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shie, JD., Chen, SM. Feature subset selection based on fuzzy entropy measures for handling classification problems. Appl Intell 28, 69–82 (2008). https://doi.org/10.1007/s10489-007-0042-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-007-0042-6

Keywords

Search

Navigation