nach oben

Erschienen in:

2016 | OriginalPaper | Buchkapitel

DeepRED – Rule Extraction from Deep Neural Networks

verfasst von : Jan Ruben Zilke, Eneldo Loza Mencía, Frederik Janssen

Erschienen in: Discovery Science

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Neural network classifiers are known to be able to learn very accurate models. In the recent past, researchers have even been able to train neural networks with multiple hidden layers (deep neural networks) more effectively and efficiently. However, the major downside of neural networks is that it is not trivial to understand the way how they derive their classification decisions. To solve this problem, there has been research on extracting better understandable rules from neural networks. However, most authors focus on nets with only one single hidden layer. The present paper introduces a new decompositional algorithm – DeepRED – that is able to extract rules from deep neural networks.

The evaluation of the proposed algorithm shows its ability to outperform a pedagogical baseline on several tasks, including the successful extraction of rules from a neural network realizing the XOR function.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Unsupervised Anomaly Detection in Noisy Business Process Event Logs Using Denoising Autoencoders

Nächstes Kapitel Ligand Affinity Prediction with Multi-pattern Kernels

The merging may produce rules of the form \(i_1<0.1\) AND \(i_1>0.2\), or \(i_1>0.4\) AND \(i_1>0.5\).

Input instances are drawn randomly from \({x} \in \{ 0, 0.5, 1 \} \times \{ 0, 0.25, 0.5, 0.75, 1 \} \times [0, 1]^3\). For artif-I \({y}=\lambda _1\) if \(x_1 = x_2\), if \(x_1 > x_2\) AND \(x_3 > 0.4\), or if \(x_3 > x_4\) AND \(x_4 > x_5\) AND \(x_2 > 0\), else \({y} = \lambda _2\), whereas for artif-I \({y}=\lambda _1\) if \(x_1 = x_2\), if \(x_1 > x_2\) AND \( x_3 > 0.4\), or IF \(x_5 > 0.8\).

You might notice that, earlier, we mentioned that there are 36 experiments per dataset. However, to avoid sophisticating the outcomes, we discard those experiments where the RxREN pruning results in no pruned inputs at all.

An abortion could either be the case if the experiment exceeds the allocated memory space (10000 MB) or if DeepRED needs more than the maximum execution time (24 h).

An example of a sufficient training set with the instance notation \({x} = x_{1} x_{2} x_{3} x_{4}\) would be 0011, 1101, 1000, and 0110. It contains all combinations of \(x_1\)/\(x_2\) and \(x_3\)/\(x_4\).

Andrews, R., Diederich, J., Tickle, A.B.: Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowl. Based Syst. 8(6), 373–389 (1995)CrossRefMATH

Augasta, M.G., Kathirvalavakumar, T.: Reverse engineering the neural networks for rule extraction in classification problems. Neural Process. Lett. 35(2), 131–150 (2012)CrossRef

Benítez, J.M., Castro, J.L., Requena, I.: Are artificial neural networks black boxes? IEEE Trans. Neural Netw. 8(5), 1156–1164 (1997)CrossRef

Craven, M., Shavlik, J.W.: Using sampling and queries to extract rules from trained neural networks. In: ICML, pp. 37–45 (1994)

Craven, M.W., Shavlik, J.W.: Extracting tree-structured representations of trained networks. In: Advances in Neural Information Processing Systems, pp. 24–30 (1996)

Frey, P.W., Slate, D.J.: Letter recognition using Holland-style adaptive classifiers. Mach. Learn. 6(2), 161–182 (1991)

Fu, L.: Rule generation from neural networks. IEEE Trans. Syst. Man Cybern. 24(8), 1114–1124 (1994)CrossRef

Johansson, U., Lofstrom, T., Konig, R., Sonstrod, C., Niklasson, L.: Rule extraction from opaque models-a slightly different perspective. In: 5th International Conference on Machine Learning and Applications, ICMLA 2006, pp. 22–27. IEEE (2006)

LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRef

10.

Quinlan, J.R.: C4.5: Programs for Machine Learning, vol. 1. Morgan Kaufmann, San Francisco (1993)

11.

Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach. Pearson Education, New York (1995)MATH

12.

Sato, M., Tsukimoto, H.: Rule extraction from neural networks via decision tree induction. In: Proceedings of the International Joint Conference on Neural Networks, IJCNN 2001, vol. 3, pp. 1870–1875. IEEE (2001)

13.

Schmitz, G.P., Aldrich, C., Gouws, F.S.: ANN-DT: an algorithm for extraction of decision trees from artificial neural networks. IEEE Trans. Neural Netw. 10(6), 1392–1401 (1999)CrossRef

14.

Sethi, K.K., Mishra, D.K., Mishra, B.: KDRuleEx: a novel approach for enhancing user comprehensibility using rule extraction. In: 2012 Third International Conference on Intelligent Systems, Modelling and Simulation (ISMS), pp. 55–60. IEEE (2012)

15.

Setiono, R., Leow, W.K.: FERNN: an algorithm for fast extraction of rules from neural networks. Appl. Intell. 12(1–2), 15–25 (2000)CrossRef

16.

Taha, I.A., Ghosh, J.: Symbolic interpretation of artificial neural networks. IEEE Trans. Knowl. Data Eng. 11(3), 448–463 (1999)CrossRef

17.

Thrun, S.: Extracting provably correct rules from artificial neural networks. Technical report, University of Bonn, Institut für Informatik III (1993)

18.

Thrun, S.: Extracting rules from artificial neural networks with distributed representations. In: Advances in neural information processing systems, pp. 505–512 (1995)

19.

Towell, G.G., Shavlik, J.W.: Extracting refined rules from knowledge-based neural networks. Mach. Learn. 13(1), 71–101 (1993)

20.

Tsukimoto, H.: Extracting rules from trained neural networks. IEEE Trans. Neural Netw. 11(2), 377–389 (2000)CrossRef

21.

Zhou, Z.H., Chen, S.F., Chen, Z.Q.: A statistics based approach for extracting priority rules from trained neural networks. In: Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks, IJCNN 2000, vol. 3, pp. 401–406. IEEE (2000)

Titel: DeepRED – Rule Extraction from Deep Neural Networks
verfasst von: Jan Ruben Zilke
Eneldo Loza Mencía
Frederik Janssen
Verlag: Springer International Publishing
Buch: Discovery Science
Print ISBN: 978-3-319-46306-3

Electronic ISBN: 978-3-319-46307-0

Copyright-Jahr: 2016
DOI: https://doi.org/10.1007/978-3-319-46307-0_29

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner