Survey and critique of techniques for extracting rules from trained artificial neural networks

doi:10.1016/0950-7051(96)81920-4

Knowledge-Based Systems

Volume 8, Issue 6, December 1995, Pages 373-389

https://doi.org/10.1016/0950-7051(96)81920-4 Get rights and content

Abstract

It is becoming increasingly apparent that, without some form of explanation capability, the full potential of trained artificial neural networks (ANNs) may not be realised. This survey gives an overview of techniques developed to redress this situation. Specifically, the survey focuses on mechanisms, procedures, and algorithms designed to insert knowledge into ANNs (knowledge initialisation), extract rules from trained ANNs (rule extraction), and utilise ANNs to refine existing rule bases (rule refinement). The survey also introduces a new taxonomy for classifying the various techniques, discusses their modus operandi, and delineates criteria for evaluating their efficacy.

References (37)

R. Davis et al.
Production rules as a representation for a knowledge-based consultation program
Artificial Intelligence
(1977)
H.R. Berenji
Refinement of approximate reasoning-based controllers by reinforcement learning
S.K. Halgamuge et al.
Neural networks in designing fuzzy systems for real world applications
Fuzzy Sets and Systems
(1994)
S. Mitra
Fuzzy MLP based expert system for medical diagnosis
Fuzzy Sets and Systems
(1994)
S.B. Thrun
Extracting provably correct rules from artificial neural networks
G. Towell et al.
The extraction of refined rules from knowledge based neural networks
Machine Learning
(1993)
S. Fahlman et al.
The cascade-correlation learning architecture
C.L. Giles et al.
Extraction, insertion, and refinement of symbolic rules in dynamically driven recurrent networks
Connection Science
(1993)
C.L. Giles et al.
Learning and extracting finite state automata with second-order recurrent neural networks
Neural Computation
(1992)
R.L. Watrous et al.
Induction of finite-state languages using second-order recurrent networks
Neural Computation
(1992)

S. Gallant

Connectionist expert systems

Communications of the ACM

(1988)

N. Gilbert

Explanation and dialogue

Knowledge Engineering Review

(1989)

J.D. Moore

A reactive approach to explanation in expert and advice-giving systems

J.D. Moore et al.

A reactive approach to explanation

W.R. Swartout

Lecture on explanation

(1989)

M.W. Craven et al.

Using sampling and queries to extract rules from trained neural networks

K. Saito et al.

Medical diagnostic; expert system based on PDP model

S. Sestito et al.

The use of sub-symbolic methods for the automation of knowledge acquisition for expert systems

Cited by (966)

Linguistic complexity consideration for advanced risk decision making and handling
2024, Research in International Business and Finance
This study presents a fusion architecture to predict enterprises’ operating efficiency via linguistic cues from the accounting narrative disclosures. We specifically apply topic modelling to decompose annual reports into some topics, preserve those topics that relate to business operation risks (including financial, operational, strategic, and hazard), and then incorporate the preserved topics with a readability score (i.e., linguistic cues) to conjecture managers’ attitude toward their firm’s risks. After examining the association between those four risk types derived from annual reports and operating efficiency calculated by a two-step integration model (i.e., it combines balanced scorecards (BSCs) with data envelopment analysis (DEA)), the results show that the model with linguistic cues does improve forecasting quality. This finding provides deep insight for users to form better judgments, and directly supports the recent reporting regulation in Taiwan that encourages enterprises to add a new section in their annual reports to disclose any risk encountered.
Explaining answers generated by knowledge graph embeddings
2024, International Journal of Approximate Reasoning
Completion of large-scale knowledge bases, such as DBPedia or Freebase, often relies on embedding models that turn symbolic relations into vector-based representations. Such embedding models are rather opaque to the human user. Research in interpretability has emphasized non-relational classifiers, such as deep neural networks, and has devoted less effort to opaque models extracted from relational structures, such as knowledge graph embeddings. We introduce techniques that produce explanations, expressed as logical rules, for predictions based on the embeddings of knowledge graphs. Algorithms build explanations out of paths in an input knowledge graph, searched through contextual and heuristic cues.
Rule extraction based on PROMETHEE-assisted multi-objective genetic algorithm for generating interpretable neural networks
2024, Applied Soft Computing
Rule extraction from artificial neural networks (ANNs) refers to the process of identifying the underlying decision-making logic used by the network in its predictions. This involves converting the complex, non-linear relationships between inputs and outputs in the network into a set of interpretable rules. The main objective of rule extraction is to enhance the interpretability of decision-making in artificial neural networks (ANNs) and make it more understandable for human experts, especially in critical domains like healthcare, where clear decision-making is essential. Despite the ability of machine learning methods to interpret ANNs, interpreting complex neural networks remains limited, as most studies tend to sacrifice one criterion, related to rule quality, in favor of another. Rule extraction methods applied directly to complex neural networks may produce overly specific rules that are tailored to the training data. Such rules can suffer from overfitting and may not generalize well to unseen data. Also, The use of multiobjective genetic algorithms for rule extraction can result in a large number of rules, making it challenging for decision-makers to interpret them. This study proposes an innovative approach to address challenges in rule extraction from complex artificial neural networks (ANNs) by integrating a multiobjective genetic algorithm with the PROMETHEE method. By integrating these techniques, the study aims to identify meaningful relationships between inputs and outputs of ANNs, generating concise and reliable rules that are ranked based on support, confidence, and lift values. The use of PROMETHEE's rule ranking mechanism enabled the consideration of rule priority during candidate selection, guiding the genetic algorithm towards discovering high-quality rule sets and efficiently exploring the search space. This approach not only enhances the quality of extracted rules but also facilitates efficient decision-making by striking a balance between rule accuracy, fidelity, and comprehensibility, thus contributing to advancing the understanding of complex neural networks. By equally weighting the criteria (support, confidence, and lift) during the rule ranking mechanism, our approach achieved high coverage rates ranging from 94.88 % to 100 % and generated a manageable number of rules that could be easily interpreted by human experts, ranging from 5 to 10.5. These findings demonstrate the potential of our approach to significantly improve decision-making accuracy and interpretability across various real-world applications, making it a promising tool for applications in healthcare, finance, and other fields.
Computational approaches to Explainable Artificial Intelligence: Advances in theory, applications and trends
2023, Information Fusion
Deep Learning (DL), a groundbreaking branch of Machine Learning (ML), has emerged as a driving force in both theoretical and applied Artificial Intelligence (AI). DL algorithms, rooted in complex and non-linear artificial neural systems, excel at extracting high-level features from data. DL has demonstrated human-level performance in real-world tasks, including clinical diagnostics, and has unlocked solutions to previously intractable problems in virtual agent design, robotics, genomics, neuroimaging, computer vision, and industrial automation. In this paper, the most relevant advances from the last few years in Artificial Intelligence (AI) and several applications to neuroscience, neuroimaging, computer vision, and robotics are presented, reviewed and discussed. In this way, we summarize the state-of-the-art in AI methods, models and applications within a collection of works presented at the 9 $th$ International Conference on the Interplay between Natural and Artificial Computation (IWINAC). The works presented in this paper are excellent examples of new scientific discoveries made in laboratories that have successfully transitioned to real-life applications.
Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence
2023, Information Fusion
Artificial intelligence (AI) is currently being utilized in a wide range of sophisticated applications, but the outcomes of many AI models are challenging to comprehend and trust due to their black-box nature. Usually, it is essential to understand the reasoning behind an AI model’s decision-making. Thus, the need for eXplainable AI (XAI) methods for improving trust in AI models has arisen. XAI has become a popular research subject within the AI field in recent years. Existing survey papers have tackled the concepts of XAI, its general terms, and post-hoc explainability methods but there have not been any reviews that have looked at the assessment methods, available tools, XAI datasets, and other related aspects. Therefore, in this comprehensive study, we provide readers with an overview of the current research and trends in this rapidly emerging area with a case study example. The study starts by explaining the background of XAI, common definitions, and summarizing recently proposed techniques in XAI for supervised machine learning. The review divides XAI techniques into four axes using a hierarchical categorization system: (i) data explainability, (ii) model explainability, (iii) post-hoc explainability, and (iv) assessment of explanations. We also introduce available evaluation metrics as well as open-source packages and datasets with future research directions. Then, the significance of explainability in terms of legal demands, user viewpoints, and application orientation is outlined, termed as XAI concerns. This paper advocates for tailoring explanation content to specific user types. An examination of XAI techniques and evaluation was conducted by looking at 410 critical articles, published between January 2016 and October 2022, in reputed journals and using a wide range of research databases as a source of information. The article is aimed at XAI researchers who are interested in making their AI models more trustworthy, as well as towards researchers from other disciplines who are looking for effective XAI methods to complete tasks with confidence while communicating meaning from data.
A survey on neural-symbolic learning systems
2023, Neural Networks
In recent years, neural systems have demonstrated highly effective learning ability and superior perception intelligence. However, they have been found to lack effective reasoning and cognitive ability. On the other hand, symbolic systems exhibit exceptional cognitive intelligence but suffer from poor learning capabilities when compared to neural systems. Recognizing the advantages and disadvantages of both methodologies, an ideal solution emerges: combining neural systems and symbolic systems to create neural-symbolic learning systems that possess powerful perception and cognition. The purpose of this paper is to survey the advancements in neural-symbolic learning systems from four distinct perspectives: challenges, methods, applications, and future directions. By doing so, this research aims to propel this emerging field forward, offering researchers a comprehensive and holistic overview. This overview will not only highlight the current state-of-the-art but also identify promising avenues for future research.

View all citing articles on Scopus

View full text

Survey and critique of techniques for extracting rules from trained artificial neural networks

Abstract

Artificial Intelligence

Fuzzy Sets and Systems

Fuzzy Sets and Systems

Extracting provably correct rules from artificial neural networks

The extraction of refined rules from knowledge based neural networks

Machine Learning

The cascade-correlation learning architecture

Extraction, insertion, and refinement of symbolic rules in dynamically driven recurrent networks

Connection Science

Learning and extracting finite state automata with second-order recurrent neural networks

Neural Computation

Induction of finite-state languages using second-order recurrent networks