Skip to main content
SpringerLink
Log in

An Analytical Study on Reasoning of Extreme Learning Machine for Classification from Its Inductive Bias

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Since extreme learning machine (ELM) was proposed, hundreds of studies have been conducted on this subject in various areas, from theoretical researches to practical applications. However, there are very few papers in the literature to reveal the reasons why in ELM classification the class with the highest output value is being chosen as the predicted class for a given input. In order to give a clear insight into this question, this paper analyzes the rationality of ELM reasoning from the perspective of its inductive bias. The analysis results show that the choice of highest output in ELM is reasonable for both binary and multiclass classification problems. In addition, to deal with multiclass problems ELM uses the well-known one-against-all strategy, in which unclassifiable regions may exist. This paper also gives a clear explanation on how ELM resolves the unclassifiable regions, through both analysis and experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Huang GB, Zhu QY, Siew CK. Extreme learning machine: theory and applications. Neurocomputing. 2006;70:489–501.

    Article  Google Scholar 

  2. Huang G-B, Chen L. Convex incremental extreme learning machine. Neurocomputing. 2007;70:3056–62.

    Article  Google Scholar 

  3. Huang G, Huang GB, Song SJ, You KY. Trends in extreme learning machines: a review. Neural Netw. 2015;61:32–48.

    Article  PubMed  Google Scholar 

  4. Feng G, Huang G-B, Lin Q, Gay R. Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw. 2009;20:1352–7.

    Article  PubMed  Google Scholar 

  5. Liu N, Wang H. Ensemble based extreme learning machine. IEEE Signal Process Lett. 2010;17:754–7.

    Article  Google Scholar 

  6. Zong WW, Huang GB, Chen YQ. Weighted extreme learning machine for imbalance learning. Neurocomputing. 2013;101:229–42.

    Article  Google Scholar 

  7. Gao H, Shiji S, Gupta JND, Cheng W. Semi-supervised and unsupervised extreme learning machines. IEEE Trans Cybern. 2014;44:2405–17.

    Article  Google Scholar 

  8. Zhang F, Qi L, Chen E. Extended extreme learning machine for biometric signal classification. J Comput Theor Nanosci. 2015;12:1247–51.

    Article  CAS  Google Scholar 

  9. Liu H, Tian H-Q, Li Y-F. Four wind speed multi-step forecasting models using extreme learning machines and signal decomposing algorithms. Energy Convers Manage. 2015;100:16–22.

    Article  Google Scholar 

  10. Li W, Chen C, Su H, Du Q. Local binary patterns and extreme learning machine for hyperspectral imagery classification. IEEE Trans Geosci Remote Sens. 2015;53:3681–93.

    Article  Google Scholar 

  11. Vong CM, Tai KI, Pun CM, Wong PK. Fast and accurate face detection by sparse Bayesian extreme learning machine. Neural Comput Appl. 2015;26:1149–56.

    Article  Google Scholar 

  12. Huang GB, Zhou HM, Ding XJ, Zhang R. Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B Cybern. 2012;42:513–29.

    Article  Google Scholar 

  13. Zhang T. Statistical analysis of some multi-category large margin classification methods. J Mach Learn Res. 2004;5:1225–51.

    Google Scholar 

  14. Abe S. Support vector machines for pattern classification. London: Springer; 2010.

    Book  Google Scholar 

  15. Mitchell TM. The need for biases in learning generalizations. New Jersey: Department of Computer Science, Laboratory for Computer Science Research, Rutgers University; 1980.

    Google Scholar 

  16. Mitchell TM. Machine Learning. Burr Ridge: McGraw Hill; 1997.

    Google Scholar 

Download references

Acknowledgments

This study was funded by the University of Macau Research Grant (Grant Nos: MYRG2014-00178-FST, MYRG2014-00083-FST and MYRG075(Y1-L2)-FST13-VCM) and the Science and Technology Development Fund of Macau (Grant No: FDCT/050/2015/A). Pak Kin Wong and Chi Man Vong have received research grants from the University of Macau and the Science and Technology Development Fund of Macau S.A.R.

Author information

Authors and Affiliations

  1. Department of Electromechanical Engineering, University of Macau, Macau, China

    Pak Kin Wong, Xiang Hui Gao & Ka In Wong

  2. Department of Computer and Information Science, University of Macau, Macau, China

    Chi Man Vong

Authors
  1. Pak Kin Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiang Hui Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ka In Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chi Man Vong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pak Kin Wong.

Ethics declarations

Conflict of Interest

Pak Kin Wong, Xiang Hui Gao, Ka In Wong and Chi Man Vong declare that they have no conflict of interest.

Informed Consent

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008 (5). Additional informed consent was obtained from all patients for which identifying information is included in this article.

Human and Animal Rights

This article does not contain any studies with human participants or animals performed by any of the authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wong, P.K., Gao, X.H., Wong, K.I. et al. An Analytical Study on Reasoning of Extreme Learning Machine for Classification from Its Inductive Bias. Cogn Comput 8, 746–756 (2016). https://doi.org/10.1007/s12559-016-9414-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-016-9414-8

Keywords

Search

Navigation