Skip to main content
Top

2021 | OriginalPaper | Chapter

20. Expanded Basis Sets for the Manipulation of Random Forests

Author : T. L. Keevers

Published in: Data and Decision Sciences in Action 2

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Random Forests is considered one of the best off-the-shelf algorithms for data mining. However, it suffers from poor interpretability and an opaque decision structure. In this paper, we develop a method for generating an “expanded basis set” for a Random Forest model that captures every possible decision rule and vastly improves the transparency of the classifier. The expanded basis set allows the structure of a Random Forest model to be algebraically manipulated and facilitates a number of operations, including inverse mapping from outputs to the domain of inputs, systematic identification of every decision boundary, and comparison of Random Forest models. The expanded basis set facilitates visualization of the global behaviour of a Random Forest classifier and a data set by combining parallel coordinates with a non-linear binning transformation. The global visualization allows classifier performance to be compared against domain expertise, and areas of underfitting and overfitting to be readily identified. Additionally, the expanded basis set underpins the generation of counterfactuals and anchors—combinations of variables that control the local outputs of a Random Forest model. The basis states can also be used to place bounds on the model stability in response to single or multi-feature perturbations. These stability bounds are especially useful when the model inputs may be uncertain or subject to variation over time.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Friedman J, Hastie T, Tibshirani R (2001) The elements of statistical learning, vol 1. Springer series in statistics New York, NY, USA Friedman J, Hastie T, Tibshirani R (2001) The elements of statistical learning, vol 1. Springer series in statistics New York, NY, USA
2.
go back to reference Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat, pp 1189–1232 Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat, pp 1189–1232
3.
go back to reference Goldstein A, Kapelner A, Bleich J, Pitkin E (2015) Peeking inside the black box: visualizing statistical learning with plots of individual conditional expectation. J Computat Graph Stat 24(1):44–65 MathSciNetCrossRef Goldstein A, Kapelner A, Bleich J, Pitkin E (2015) Peeking inside the black box: visualizing statistical learning with plots of individual conditional expectation. J Computat Graph Stat 24(1):44–65 MathSciNetCrossRef
4.
go back to reference Ribeiro M, Sameer S, Carlos G (2016) Model-agnostic interpretability of machine learning Ribeiro M, Sameer S, Carlos G (2016) Model-agnostic interpretability of machine learning
6.
go back to reference Friedman Jerome H, Popescu Bogdan E et al (2008) Predictive learning via rule ensembles. Ann Appl Stat 2(3):916–954 MathSciNetCrossRef Friedman Jerome H, Popescu Bogdan E et al (2008) Predictive learning via rule ensembles. Ann Appl Stat 2(3):916–954 MathSciNetCrossRef
7.
go back to reference Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Anchors: High-precision model-agnostic explanations. In AAAI Conference on Artificial Intelligence, 2018 Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Anchors: High-precision model-agnostic explanations. In AAAI Conference on Artificial Intelligence, 2018
8.
go back to reference Pedregosa Fabian, Varoquaux Gaël, Gramfort Alexandre, Michel Vincent, Thirion Bertrand, Grisel Olivier, Blondel Mathieu, Prettenhofer Peter, Weiss Ron, Dubourg Vincent, Vanderplas Jake, Passos Alexandre, Cournapeau David, Brucher Matthieu, Perrot Matthieu, Duchesnay Édouard (2011) Scikit-learn: Machine learning in python. J. Mach. Learn. Res. 12:2825–2830 MathSciNetMATH Pedregosa Fabian, Varoquaux Gaël, Gramfort Alexandre, Michel Vincent, Thirion Bertrand, Grisel Olivier, Blondel Mathieu, Prettenhofer Peter, Weiss Ron, Dubourg Vincent, Vanderplas Jake, Passos Alexandre, Cournapeau David, Brucher Matthieu, Perrot Matthieu, Duchesnay Édouard (2011) Scikit-learn: Machine learning in python. J. Mach. Learn. Res. 12:2825–2830 MathSciNetMATH
9.
go back to reference Thomas G. Dietterich. Ensemble methods in machine learning. In International workshop on multiple classifier systems, pages 1–15. Springer, 2000 Thomas G. Dietterich. Ensemble methods in machine learning. In International workshop on multiple classifier systems, pages 1–15. Springer, 2000
10.
go back to reference Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2574–2582, 2016 Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2574–2582, 2016
11.
go back to reference Kathrin Grosse, Nicolas Papernot, Praveen Manoharan, Michael Backes, and Patrick McDaniel. Adversarial examples for malware detection. In European Symposium on Research in Computer Security, pages 62–79. Springer, 2017 Kathrin Grosse, Nicolas Papernot, Praveen Manoharan, Michael Backes, and Patrick McDaniel. Adversarial examples for malware detection. In European Symposium on Research in Computer Security, pages 62–79. Springer, 2017
12.
go back to reference Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:​1605.​07277, 2016 Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:​1605.​07277, 2016
13.
go back to reference Goodfellow IJ, Jonathon S, Christian S (2015) Explaining and harnessing adversarial examples. In: Proceedings of the 2015 International Conference on Learning Representations Goodfellow IJ, Jonathon S, Christian S (2015) Explaining and harnessing adversarial examples. In: Proceedings of the 2015 International Conference on Learning Representations
Metadata
Title
Expanded Basis Sets for the Manipulation of Random Forests
Author
T. L. Keevers
Copyright Year
2021
DOI
https://doi.org/10.1007/978-3-030-60135-5_20