Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
ATZ-Fachkongress

Chassis.tech plus 2024


4 Kongresse in einer Veranstaltung

Treffen Sie in München die Fachleute von Chassis, Lenkung, Bremsen sowie Reifen/Räder.
Erweiterte Suche
Anmelden
nach oben
Journal of Computer and Systems Sciences International
Erschienen in: Journal of Computer and Systems Sciences International 6/2021

01.11.2021 | ARTIFICIAL INTELLIGENCE

Review of Research in the Field of Developing Methods to Extract Rules From Artificial Neural Networks

verfasst von: A. N. Averkin, S. A. Yarushev

Erschienen in: Journal of Computer and Systems Sciences International | Ausgabe 6/2021

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

A large-scale review and analysis of the existing methods and approaches to extract rules from artificial neural networks, including deep learning neural networks, is carried out. A wide range of methods and approaches to extract rules and related approaches to develop explainable artificial intelligence (AI) systems are considered. The taxonomy and several directions in studies of explainable neural networks related to the extraction of rules from neural networks, which allow the user to get an idea of how the neural network uses the input data, and also, using rules, to reveal the hidden relationships of the input data and the results found, are explored. This review focuses on the relationship of the most common rule-based explanation systems in AI with the most powerful machine learning algorithms using neural networks. In addition to rule extraction, other methods of constructing explainable AI systems are considered based on the construction of special modules that interpret each step of changing the neural network’s weights. A comprehensive analysis of the existing research makes it possible to draw conclusions about the appropriateness of using certain approaches. The results of the analysis will allow us to get a detailed picture of the state of research in this area and create our own applications based on neural networks, the results of which can be studied in detail and their reliability evaluated. The development of such systems is necessary for the development of the digital economy in Russia and the creation of applications that allow making responsible and explainable management decisions in critical areas of the national economy.
Vorheriger Artikel Analysis of Critical Damage in the Communication Network. IV: MultiCriteria Estimations of Cluster Vulnerability
Nächster Artikel Support for Managing the Survivability of Energy Systems Based on a Combinatorial Approach

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Literatur
1.
M. Bilgic and R. J. Mooney, “Explaining recommendations: Satisfaction vs. promotion,” in Proceedings of the Beyond Personalization Workshop, 2005, Vol. 5, p. 153.
2.
W. R. Swartout and J. D. Moore, “Explanation in second generation expert systems,” in Second Generation Expert Systems (Springer, Berlin, 1993), pp. 543–585.CrossRef
3.
B. Chandrasekaran, M. C. Tanner, and J. R. Josephson, “Explaining control strategies in problem solving,” IEEE Expert 4 (1), 9–15 (1989).CrossRef
4.
J. S. Dhaliwal and I. Benbasat, “The use and effects of knowledge-based system explanations: theoretical foundations and a framework for empirical evaluation,” Inform. Syst. Res. 7, 342–362 (1996).CrossRef
5.
M. M. Eining and P. B. Dorr, “The impact of expert system usage on experiential learning in an auditing setting,” Inform. Syst. 5, 1–16 (1991).
6.
D. S. Murphy, “Expert system use and the development of expertise in auditing: A preliminary investigation,” Inform. Syst. 4, 18–35 (1990).
7.
D. M. Lamberti and W. A. Walace, “Intelligent interface design: An empirical assessment of knowledge presentation in expert systems,” MIS Quart. 14, 279–311 (1990).CrossRef
8.
Artificial Intelligence, The Reference Book, Ed. by V. N. Zakharov, E. V. Popov, D. A. Pospelov, and V. F. Khoroshevskii (Radio Svyaz’, Moscow, 1990) [in Russian].
9.
E. V. Popov, Expert Systems: Solving Unformalized Tasks in a Dialogue with a Computer (Nauka, Moscow, 1987) [in Russian].
10.
W. R. Swartout, “A digitalis therapy advisor with explanations,” in Proceedings of the 5th International Joint Conference on Artificial Intelligence (Cambridge, 1977), Vol. 2, pp. 819–825.
11.
J. L. Weiner, “BLAH, A system that explains its reasoning,” Artif. Intell. 15, 19–48 (1980).CrossRef
12.
W. R. Swartout, C. Paris, and J. Moore, “Explanations in knowledge systems: Design for explainable expert systems,” IEEE Expert 6 (3), 58–64 (1991).CrossRef
13.
W. J. Clancey, Intelligent Tutoring Systems: A Tutorial Survey (Stanford Univ. Dep. Comput. Sci., Stanford, 1986).
14.
R. Sinha and K. Swearingen, “The role of transparency in recommender systems,” in Extended Abstracts of CHI'02 Conference on Human Factors in Computing Systems (Minneapolis, 2002), pp. 830–831.
15.
T. Gruber, “Learning why by being told what,” IEEE Expert 6 (4), 65–75 (1991).CrossRef
16.
W. J. Clancey, “Details of the revised therapy algorithm,” in Rule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project (Addison-Wesley, Reading, MA, 1984), pp. 133–146.
17.
W. J. Clancey, “From GUIDON to NEOMYCIN and HERACLES in twenty short lessons,” AI Magazine 7 (3), 40 (1986).
18.
A. Arioua, P. Buche, and M. Croitoru, “Explanatory dialogs with argumentative faculties over inconsistent knowledge bases,” Expert Syst. Appl. 80, 244–262 (2017).CrossRef
19.
U. Johansson, T. Lofstrom, R. Konig, C. Sonstro, and L. Nilsson, “Rule extraction from opaque models-a slightly different perspective,” in Proceedings of the 5th International Conference on Machine Learning and Applications (ICMLA’06), Orlando, FL, USA, 2006, pp. 22–27.
20.
M. Craven and J. Shavlik, “Rule extraction: Where do we go from here,” in University of Wisconsin Machine Learning Research Group Working Paper (Wisconsin, 1999), pp. 99–108.
21.
R. Andrew, J. Diederich, and A. B. Tickle, “Survey and critique of techniques for extracting rules from trained artificial networks,” Knowledge-based Syst. 8, 373–389 (1995).CrossRef
22.
S. Thrum, “Extracting provably correct rules from artificial neural networks,” Technical Report (Inst. Inform. III, Bonn, 1993).
23.
M. Craven and J. W. Shavlik, “Using sampling and queries to extract rules from trained neural networks,” in Proceedings of the 11th International Conference, Rutgers Univ., New Brunswick, USA, 1994, pp. 37–45.
24.
L. Fu, “Rule generation from neural networks,” IEEE Trans. Syst. Man Cybern. 24, 1114–1124 (1994).CrossRef
25.
M. Sato and H. Tsukimoto, “Rule extraction from neural networks via decision tree induction,” in Proceedings of International Joint Conference on Neural Networks (IJCNN'01) (Washington, DC, 2001), Vol. 3, pp. 1870–1875.
26.
A. B. Tickle, R. Andrew, M. Golea, and J. Diederich, “The truth will come to light: Directions and challenges in extracting the knowledge embedded within trained artificial neural networks,” IEEE Trans. Neural Networks 9, 1057–1068 (1998).CrossRef
27.
K. K. Sethi, D. K. Mishra, and B. Mishra, “KDRuleEx: A novel approach for enhancing user comprehensibility using rule extraction,” in Intelligent Systems, Modelling and Simulation (ISMS), Proceedings of the 3rd International Conference, Kota Kinabalu, Malaysia, 2012, pp. 55–60.
28.
U. Johansson, T. Lofstrom, R. Konig, C. Sonstrod, and L. Niklasson, “Rule extraction from opaque models-a slightly different perspective,” in Proceedings of the 5th International Conference on Machine Learning and Applications, ICMLA’06, Orlando, FL, USA, 2006, pp. 22–27.
29.
M. Rangwala and G. R. Weckman, “Extracting rules from artificial neural networks utilizing TREPAN,” in Proceedings of IIE Annual Conference, Orlando, Florida, 2006.
30.
T. Hailesilassie, “Extraction algorithm for deep neural networks: A review,” Int. J. Comput. Sci. Inform. Secur. 14, 376–381 (2016).
31.
A. Averkin and S. Yarushev, “Hybrid neural networks and time series forecasting,” Commun. Comput. Inform. Sci. 934, 230–239 (2018).CrossRef
32.
G. Pilato, S. A. Yarushev, and A. N. Averkin, “Prediction and detection of user emotions based on neuro-fuzzy neural networks in social networks,” in Proceedings of the 3rd International Scientific Conference on Intelligent Information Technologies for Industry IITI'18, Sochi, Russia, Adv. Intell. Syst. Comput. 875, 118–126 (2018).
33.
A. N. Averkin, G. Pilato, and S. A. Yarushev, “An approach for prediction of user emotions based on ANFIS in social networks,” in Proceedings of the 2nd International Scientific and Practical Conference on Fuzzy Technologies in the Industry FTI 2018–CEUR Workshop, Ostrava-Prague, Czech Republic, 2018, pp. 126–134.
34.
X.-H. Jin, “Neurofuzzy decision support system for efficient risk allocation in public-private partnership infrastructure projects,” J. Comput. Civ. Eng. 24, 525–538 (2010).CrossRef
35.
X.-H. Jin, “Model for efficient risk allocation in privately financed public infrastructure projects using neuro-fuzzy techniques,” J. Constr. Eng. Manag., 1003–1014 (2011).
36.
V. V. Borisov, A. S. Fedulov, and M. M. Zernov, Fundamentals of Hybridization of Fuzzy Models, Vol. 9 of Fundamentals of Fuzzy Mathematics Series (Goryachaya Liniya-Telekom, Moscow, 2017) [in Russian].
37.
D. Rutkowska, M. Piliński, and L. Rutkowski, Neural Networks, Genetic Algorithms and Fuzzy Systems (Naukowe PWN, Warszawa, 2008) [in Polish].
38.
S. Rajab and V. Sharma, “A review on the applications of neuro-fuzzy systems in business,” Artif. Intell. Rev. 49, 481–510 (2018).CrossRef
39.
S. Mitra and Y. Hayashi, “Neuro-fuzzy rule generation: Survey in soft computing framework,” IEEE Trans. Neural Network 11, 748–768 (2000).CrossRef
40.
J. Vieira, F. Morgado-Dias, and A. Mota, “Neuro-fuzzy systems: A survey,” WSEAS Trans. Syst. 3, 414–419 (2004).
41.
J. Kim and N. Kasabov, “HyFIS: Adaptive neuro-fuzzy inference systems and their application to nonlinear dynamical systems,” Neural Network 12, 1301–1319 (1999).CrossRef
42.
K. V. Shihabudheen and G. N. Pillai, “Recent advances in neuro-fuzzy system: A survey,” Knowl.-Based Syst. 152, 136–162 (2018).CrossRef
43.
I. Z. Batyrshin, A. O. Nedosekin, A. A. Stetsko, V. B. Tarasov, A. V. Yazenin, and N. G. Yarushkina, Fuzzy Hybride Systems: Theory and Practice, Ed. by N. G. Yarushkina (Fizmatlit, Moscow, 2007) [in Russian].MATH
44.
Z. J. Viharos and K. B. Kis, “Survey on neuro-fuzzy systems and their applications in technical diagnostics and measurement,” Measurement 67, 126–136 (2015).CrossRef
45.
C. T. Lin and C. S. G. Lee, “Neural network based fuzzy logic control and decision system,” IEEE Trans. Comput. 40, 1320–1336 (1991).MathSciNetCrossRef
46.
J.-S. R. Jang, “ANFIS: Adaptive-network-based fuzzy inference system,” IEEE Trans. Syst. Cybern. 23, 665–685 (1993).CrossRef
47.
H. Naderpour and M. Mirrashid, “Shear failure capacity prediction of concrete beam-column joints in terms of ANFIS and GMDH,” Pract. Period. Struct. Des. Constr. 24 (2) (2019).
48.
L. Fan, “Revisit fuzzy neural network: Demystifying batch normalization and ReLU with generalized hamming network,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, 2017, pp. 1920–1929.
49.
H. R. Bherenji and P. Khedkar, “Learning and tuning fuzzy logic controllers through reinforcements,” IEEE Trans. Neural Networks 3, 724–740 (1992).CrossRef
50.
D. Nauck and R. Kruse, “Neuro-fuzzy systems for function approximation,” Fuzzy Sets Syst. 101, 261–271 (1999).MathSciNetCrossRef
51.
S. Tano, T. Oyama, and T. Arnould, “Deep combination of fuzzy inference and neural network in fuzzy inference,” Fuzzy Sets Syst. 82, 151–160 (1996).CrossRef
52.
J. Ch. Feng and L. Ch. Teng, “An online self constructing neural fuzzy inference network and its applications,” IEEE Trans. Fuzzy Syst. 6, 12–32 (1998).CrossRef
53.
N. Kasabov and Qun Song, “Dynamic evolving fuzzy neural networks with 'm-out-of-n' activation nodes for on-line adaptive systems,” Technical Report TR99/04 (Department of Inform. Sci., Univ. Otago, Otago, 1999).
54.
D. Gunning and D. Aha, “DARPA’S explainable artificial intelligence (XAI) program,” AI Magazine 40 (2), 44–58 (2019).CrossRef
55.
A. N. Gorban’, “Errors of data-based intelligence,” in Proceedings of the International Conference on Intelligent Systems in Science and Technology, and the 6th All-Russian Scientific and Practical Conference on Artificial Intelligence in Solving Urgent Social and Economic Problems of the XXI Century, Perm, 2020, pp. 11–13.
56.
R. Hu, J. Andreas, M. Rohrbach, T. Darrell, and K. Saenko, “Learning to reason: End-to-end module networks for visual question answering,” in Proceedings of the IEEE International Conference on Computer Vision (IEEE, New York, 2017), pp. 804–813.
57.
J. Kim and J. Canny, “Interpretable learning for self-driving cars by visualizing causal attention,” in Proceedings of the International Conference on Computer Vision (IEEE, New York, 2017), pp. 2942–2950.
58.
L. A. Hendricks, T. Darrell, and Z. Akata, “Grounding visual explanations,” in Proceedings of the European Conference of Computer Vision (ECCV), Munich, Germany (Springer, 2018).
59.
K. Marazopoulou, M. Maier, and D. Jensen, “Learning the structure of causal models with relational and temporal dependence,” in Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence, Association for Uncertainty in Artificial Intelligence, Amsterdam, Netherlands, 2015, pp. 572–581.
60.
A. Pfeffer, Practical Probabilistic Programming (Manning, Greenwich, CT, 2016).
61.
M. Harradon, J. Druce, and B. Ruttenberg, “Causal learning and explanation of deep neural networks via autoencoded activations,” arXiv: 1802.00541v1 [cs.AI] (2018).
62.
L. She and J. Y. Chai, “Interactive learning for acquisition of grounded verb semantics towards human-robot communication,” in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017, Vol. 1, pp. 1634–1644.
63.
Z. Qi and F. Li, “Learning explainable embeddings for deep networks,” in Proceedings of the NIPS Workshop on Interpreting, Explaining and Visualizing Deep Learning, Long Beach, 2017.
64.
J. Dodge, S. Penney, C. Hilderbrand, A. Anderson, and M. Burnett, “How the experts do it: Assessing and explaining agent behaviors in real-time strategy games,” in Proceedings of the CHI Conference on Human Factors in Computing Systems (Assoc. for Comput. Machinery, New York, 2018), pp. 1–12.
65.
F. Belbute-Peres and J. Z. Kolter, “A modular differentiable rigid body physics engine,” in Neural Information Processing Systems, Deep Reinforcement Learning Symposium, Long Beach, CA, 2017.
66.
A. Hefny, Z. Marinho, W. Sun, S. Srinivasa, and G. Gordon, “Recurrent predictive state policy networks,” in Proceedings of the 35th International Conference on Machine Learning (Int. Machine Learning Soc., Stockholm, Sweden, 2018), pp. 1949–1958.
67.
P. Vicol, M. Tapaswi, L. Castrejon, and S. Fidler, “MovieGraphs: Towards understanding human-centric situations from videos,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, New York, 2018), pp. 4631–4640.
68.
B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, “Object detectors emerge in deep scene CNNs,” in Proceedings of the International Conference on Learning Representations, San Diego, CA, 2015.
69.
V. Gogate and P. Domingos, “Probabilistic theorem proving,” Commun. ACM 59 (7), 107–15 (2016).CrossRef
70.
M. Du, N. Liu, Q. Song, and X. Hu, “Towards Explanation of DNN-based prediction and guided feature inversion,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Assoc. Comput. Machinery, New York, 2018), pp. 1358–1367.
71.
J. Gao, N. Liu, M. Lawley, and X. Hu, “An interpretable classification framework for information extraction from online healthcare forums,” J. Healthcare Eng. 2017, 2460174 (2017).
72.
S. C.-H. Yang and P. Shafto, “Explainable artificial intelligence via Bayesian teaching,” in Proceedings of the 31st Conference on Neural Information Processing Systems Workshop on Teaching Machines, Robots and Humans, Long Beach, CA, 2017.
Metadaten
Titel
Review of Research in the Field of Developing Methods to Extract Rules From Artificial Neural Networks
verfasst von
A. N. Averkin
S. A. Yarushev
Publikationsdatum
01.11.2021
Verlag
Pleiades Publishing
Erschienen in
Journal of Computer and Systems Sciences International / Ausgabe 6/2021
Print ISSN: 1064-2307
Elektronische ISSN: 1555-6530
DOI
https://doi.org/10.1134/S1064230721060046

Weitere Artikel der Ausgabe 6/2021

Journal of Computer and Systems Sciences International 6/2021 Zur Ausgabe

CONTROL SYSTEMS OF MOVING OBJECTS

Optimal Orientation Control of a Space Vehicle with Restrictions on the Control and Phase Variables

COMPUTER METHODS

Decompositional Method for the Optimization Problem of Effective Shooting

COMPLEX TECHNICAL CONTROL SYSTEMS AND INFORMATION CONTROL COMPLEXES

Support for Managing the Survivability of Energy Systems Based on a Combinatorial Approach

OPTIMAL CONTROL

The Necessary Conditions for Optimal Hybrid Systems of Variable Dimensions

COMPUTER METHODS

Using Solvable Real Time Classes for Jitter-Minimized Planning

COMPUTER METHODS

Signatures of Extremal 2-Unifrom Hypergraphs

Premium Partner

    Bildnachweise