Top

Published in:

2024 | OriginalPaper | Chapter

14. A Better Random Forest Classifier: Labels Guided Mondrian Forest

Authors : Ismaël Koné, Adama Samaké, Behou Gérard N’Guessan, Lahsen Boulmane

Published in: Mathematics of Computer Science, Cybersecurity and Artificial Intelligence

Publisher: Springer Nature Switzerland

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

A novel class of Random Forests (RFs), namely Mondrian Forests (MFs), which are an ensemble of Mondrian Trees, achieves competitive performance relatively to classical Breiman RFs. They have attractive properties like performing Bayesian inference at the tree level and being trainable online. However, they perform poorly in the presence of less or low predictive power features. Thus, we propose to extend MF by using label information during splits in order to make them more accurate and robust. We showed an increase in performance when using labels during splits on four datasets where we notice a big improvement on a dataset containing many non-predictive features which is very important as feature relevancy is unknown at first. Additionally, this extension yields equal or superior performance relatively to classical RFs.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Robustness of Imputation Methods with Backpropagation Algorithm in Nonlinear Multiple Regression

next chapter Remote Sensing of Artisanal Mines Buried in the Ground by Infrared Thermography Using UAV

Available only for authorised users

https://github.com/iskode/mondrianforest-labelsplit/

https://github.com/balajiln/mondrianforest

Lakshminarayanan B., Roy D. M., and Teh Y. W., Mondrian Forests: Efficient Online Random Forests, Advances in Neural Information Processing Systems, 27, 3140–3148 (2014)

Breiman, L. Random Forests, Machine Learning, 45, 5–32 (2001)CrossRef

Siddharth Misra, Yaokun Wu, Machine Learning for Subsurface Characterization, 289–314. Gulf Professional Publishing, United States (2020)

Niculescu-Mizil A. and Caruana R., Predicting Good Probabilities with Supervised Learning, Proceedings of the International Conference on Machine Learning - ICML, 22, (2005)

Lakshminarayanan B., Thesis: Decision Trees and Forests: A Probabilistic Perspective, UCL, United Kingdom (2016)

Shannon C. E., A Mathematical Theory of Communication, The Bell System Technical Journal, 27, 379–423 (1948)MathSciNetCrossRef

Daniel M. Roy, Yee Teh, The Mondrian Process, Advances in Neural Information Processing Systems, 21, 1377–138 (2008)

Hull J. J., A database for handwritten text recognition research, IEEE Transactions on Pattern Analysis and Machine Intelligence, 16, 550–554 (1994)CrossRef

Feng C., Sutherland A., King S., Muggleton S. & Henery R., Comparison of Machine Learning Classifiers to Statistics and Neural Networks, AI & Stats Conference, 93(1993).

10.

Srinivasan A., Statlog (Landsat Satellite). UCI Machine Learning Repository (1993)

11.

Slate D., Letter Recognition. UCI Machine Learning Repository (1991).

12.

Michalski,R.S. and Chilausky,R.L.., Soybean (Large). UCI Machine Learning Repository (1988).

13.

Vanschoren J., Rijn J. N. V., Bischl B. and Torgo L., OpenML: networked science in machine learning. SIGKDD Explorations 15(2), pp 49–60, 2013CrossRef

14.

Pedregosa et al., Journal of Machine Learning Research, 12, 2825–2830 (2011)

15.

Breiman L., “Bagging predictors”, Machine Learning, 24(2), 123–140, 1996.CrossRef

16.

Dietterich T. G, Kong E. B., Machine Learning Bias, Statistical Bias, and Statistical Variance of Decision Tree Algorithms, 1–14. Technical Report. Department of Computer Science, Oregon State University (1995)

17.

Breiman L., Bias, variance, and arcing classifiers, Technical Report, 460, Statistics Department, University of California, Berkeley, CA, USA (1996)

18.

Criminisi A., Shotton J., Konukoglu E., Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning, Foundations and Trends Computer Graphics Vision, 7, 81–227 (2012)CrossRef

19.

Geurts P., Ernst. D., and Wehenkel L., Extremely randomized trees, Machine Learning, 63, 3–42 (2006)CrossRef

20.

Chipman H. A., George E. I., and McCulloch R. E., Bayesian CART model search, 93, Journal of the American Statistical Association, 935–948 (1998)

21.

Denison D. G. T., Mallick B. K., and Smith A. F. M., A Bayesian CART algorithm, Biometrika, 85, 363–377 (1998)MathSciNetCrossRef

22.

Chipman H. A., George E. I., and McCulloch R. E., BART: Bayesian Additive Regression Trees, The Annals of Applied Statistics, 4, 266–298 (2010)MathSciNetCrossRef

23.

Domingos P., Bayesian Averaging of Classifiers and the Overfitting Problem, International Conference of Machine Learning, 747, 223–230 (2000)

24.

Saffari A., Leistner C., Santner J., Godec M., Bischof H., On-line random forests. Computer Vision Workshops - ICCV Workshops, 12, 1393–1400 (2009)

25.

Denil M., Matheson D., Freitas N., Consistency of online random forests, International Conference of Machine Learning, 28, 1256–1264 (2013)

26.

Lakshminarayanan B., Roy D. M., and Teh Y. W., Mondrian Forests for Large-Scale Regression when Uncertainty Matters, Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, JMLR: W&CP, 51, 1478–1487 (2016)

Title: A Better Random Forest Classifier: Labels Guided Mondrian Forest
Authors: Ismaël Koné
Adama Samaké
Behou Gérard N’Guessan
Lahsen Boulmane
Publisher: Springer Nature Switzerland
Book: Mathematics of Computer Science, Cybersecurity and Artificial Intelligence
Print ISBN: 978-3-031-66221-8

Electronic ISBN: 978-3-031-66222-5

Copyright Year: 2024
DOI: https://doi.org/10.1007/978-3-031-66222-5_14

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"