Rudresh Dwivedi
Skip Abstract Section
Abstract
As our dependence on intelligent machines continues to grow, so does the demand for more transparent and interpretable models. In addition, the ability to explain the model generally is now the gold standard for building trust and deployment of artificial intelligence systems in critical domains. Explainable artificial intelligence (XAI) aims to provide a suite of machine learning techniques that enable human users to understand, appropriately trust, and produce more explainable models. Selecting an appropriate approach for building an XAI-enabled application requires a clear understanding of the core ideas within XAI and the associated programming frameworks. We survey state-of-the-art programming techniques for XAI and present the different phases of XAI in a typical machine learning development process. We classify the various XAI approaches and, using this taxonomy, discuss the key differences among the existing XAI techniques. Furthermore, concrete examples are used to describe these techniques that are mapped to programming frameworks and software toolkits. It is the intention that this survey will help stakeholders in selecting the appropriate approaches, programming frameworks, and software toolkits by comparing them through the lens of the presented taxonomy.
- [1] . 2020. Towards explainable deep neural networks (xDNN). Neural Networks 130 (2020), 185–194. Google Scholar
Cross Ref
- [2] . 2016. Visualizing the effects of predictor variables in black box supervised learning models. arXiv preprint arXiv:1612.08468.Google Scholar
- [3] . 2005. Dimensions of neural-symbolic integration— structured survey. In We Will Show Them! Essays in Honour of Dov Gabbay, Volume One, , , , , and (Eds.). College Publications, 167–194.Google Scholar
- [4] . 2020. Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion 58 (2020), 82–115. Google ScholarDigital Library
- [5] . 2014. RDF 1.1 Turtle. World Wide Web Consortium 2014 (2014), 18–31.Google Scholar
- [6] . 2017. Neural-symbolic learning and reasoning: A survey and interpretation. CoRR abs/1711.03902.Google Scholar
- [7] . 2020. Explainable machine learning in deployment. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency.ACM, New York, NY, 648–657. Google ScholarDigital Library
- [8] . 2016. DL-Learner—A framework for inductive learning on the semantic web. Journal of Web Semantics 39 (2016), 15–24.Google ScholarDigital Library
- [9] . 2019. Machine learning interpretability: A survey on methods and metrics. Electronics 8, 8 (2019), 832.Google ScholarCross Ref
- [10] 2015. NVIDIA Collective Communications Library (NCCL). Retrieved July 15, 2021 from https://developer.nvidia.com/nccl.Google Scholar
- [11] . 2012. Large scale distributed deep networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems (NIPS’12). 1223–1231.Google Scholar
- [12] . 2018. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In Advances in Neural Information Processing Systems. 592–603.Google Scholar
- [13] . 2017. What does explainable AI really mean? A new conceptualization of perspectives. In Proceedings of the First International Workshop on Comprehensibility and Explanation in AI and ML 2017 co-located with 16th International Conference of the Italian Association for Artificial Intelligence (AI*IA 2017), Bari, Italy, November 16th and 17th, 2017(
CEUR Workshop Proceedings , Vol. 2071), and (Eds.). CEUR-WS.org. http://ceur-ws.org/Vol-2071/CExAIIA_2017_paper_2.pdf.Google Scholar - [14] . 2017. Towards a rigorous science of interpretable machine learning. arxiv:1702.08608 [stat.ML].Google Scholar
- [15] . 2018. Explainable artificial intelligence: A survey. In Proceedings of the 2018 41st International Convention on Information and Communication Technology, Electronics, and Microelectronics (MIPRO’18). 0210–0215. Google ScholarCross Ref
- [16] . 2014. SPARQL query verbalization for explaining semantic search engine queries. In Proceedings of the European Semantic Web Conference. 426–441.Google ScholarCross Ref
- [17] . 2018. Model class reliance: Variable importance measures for any machine learning model class, from the “Rashomon” perspective. arXiv preprint arXiv:1801.01489.Google Scholar
- [18] . 2001. Greedy function approximation: A gradient boosting machine. Annals of Statistics 29, 5 (2001), 1189–1232.Google ScholarCross Ref
- [19] . 2008. Predictive learning via rule ensembles. Annals of Applied Statistics 2, 3 (2008), 916–954.Google ScholarCross Ref
- [20] . 2020. On the integration of knowledge graphs into deep learning models for a more comprehensible AI—Three challenges for future research. Information 11, 2 (2020), 122.Google ScholarCross Ref
- [21] . 2019. Explainable AI in industry. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’19). ACM, New York, NY, 3203–3204. Google ScholarDigital Library
- [22] . 2020. Shapley additive explanations for NO2 forecasting. Ecological Informatics 56 (2020), 101039.Google ScholarCross Ref
- [23] . 2009. OWL 2 Web Ontology Language New Features and Rationale. W3C Working Draft 11 June 2009. Retrieved September 9, 2022 from http://www.w3.org/TR/2009/WD-owl2-new-features-20090611/.Google Scholar
- [24] . 2015. Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation. Journal of Computational and Graphical Statistics 24, 1 (2015), 44–65.Google ScholarCross Ref
- [25] . 2017. pdp: An R package for constructing partial dependence plots. R Journal 9, 1 (2017), 421–436.Google ScholarCross Ref
- [26] . 2019. DARPA’s explainable artificial intelligence program. AI Magazine 40, 2 (2019), 44–58.Google ScholarDigital Library
- [27] . 2019. AutoML: A survey of the state-of-the-art. arxiv:1908.00709 [cs.LG].Google Scholar
- [28] . 2020. Neural-symbolic integration and the semantic web. Semantic Web 11, 1 (2020), 3–11.Google ScholarDigital Library
- [29] . 2009. OWL 2 web ontology language primer. W3C Recommendation 27, 1 (2009), 123.Google Scholar
- [30] . 2017. What do we need to build explainable AI systems for the medical domain? arXiv preprint arXiv:1712.09923.Google Scholar
- [31] . 2019. GPipe: Efficient training of giant neural networks using pipeline parallelism. Advances in Neural Information Processing Systems 32 (2019), 103–112.Google Scholar
- [32] . 2022. Data Analytics with SNOMED CT. Retrieved June 29, 2022 from https://confluence.ihtsdotools.org/display/DOCANLYT/Data+Analytics+with+SNOMED+CT.Google Scholar
- [33] Bahador Khaleghi. 2020. An Explanation of What, Why, and How of eXplainable AI (XAI). Retrieved June 9, 2022 from https://towardsdatascience.com/an-explanation-of-what-why-and-how-of-explainable-ai-xai-117d9c441265.Google Scholar
- [34] . 2019. Alibi: Algorithms for Monitoring and Explaining Machine Learning Models. Retrieved September 9, 2022 from https://github.com/SeldonIO/alibi.Google Scholar
- [35] . 2016. Adversarial machine learning at scale. CoRR abs/1611.01236.Google Scholar
- [36] . 2020. Understanding Machines: Explainable AI. Technical Report. Retrieved September 9, 2022 from https://www.accenture.com/_acnmedia/pdf-85/accenture-understanding-machines-explainable-ai.pdf.Google Scholar
- [37] . 2014. Scaling distributed machine learning with the parameter server. In Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI’14). 583–598.Google Scholar
- [38] . 2011. The Translational Medicine Ontology and Knowledge Base: Driving personalized medicine by bridging the gap between bench and bedside. Journal of Biomedical Semantics 2 (2011), 1–21.Google Scholar
- [39] . 2020. From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence 2, 1 (2020), 2522–5839.Google ScholarCross Ref
- [40] . 2017. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems. 4765–4774.Google Scholar
- [41] . 2019. The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. arXiv preprint arXiv:1904.12584.Google Scholar
- [42] . 2018. A hierarchical model for device placement. In Proceedings of the International Conference on Learning Representations.Google Scholar
- [43] . 2017. Device placement optimization with reinforcement learning. In Proceedings of the International Conference on Machine Learning. 2430–2439.Google Scholar
- [44] . 2019. Interpretable Machine Learning. Retrieved September 9, 2022 from https://christophm.github.io/interpretable-ml-book/.Google Scholar
- [45] . 2019. Explaining machine learning classifiers through diverse counterfactual explanations. CoRR abs/1905.07697.Google Scholar
- [46] . 2018. Anchors: High-precision model-agnostic explanations. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 1527–1535.Google Scholar
- [47] . 2017. Explaining trained neural networks with semantic web technologies: First steps. In Proceedings of the Twelfth International Workshop on Neural-Symbolic Learning and Reasoning, NeSy 2017, London, UK, July 17-18, 2017(
CEUR Workshop Proceedings , Vol. 2003), , , and (Eds.). CEUR-WS.org. http://ceur-ws.org/Vol-2003/NeSy17_paper4.pdf.Google Scholar - [48] . 2021. Neuro-symbolic artificial intelligence: Current trends. arXiv preprint arXiv:2105.05330.Google Scholar
- [49] . 2019. Semantic web technologies for explainable machine learning models: A literature review. PROFILES/SEMEX@ ISWC 2465 (2019), 1–16.Google Scholar
- [50] . 2017. Grad-CAM: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision. 618–626.Google ScholarCross Ref
- [51] . 2018. Horovod: Fast and easy distributed deep learning in TensorFlow. arXiv preprint arXiv:1802.05799.Google Scholar
- [52] . 2017. Learning important features through propagating activation differences. arXiv preprint arXiv:1704.02685.Google Scholar
- [53] . 2019. Interpretability Methods in Machine Learning: A Brief Survey. Retrieved September 9, 2022 from https://www.twosigma.com/insights/article/interpretability-methods-in-machine-learning-a-brief-survey/.Google Scholar
- [54] . 2017. SmoothGrad: Removing noise by adding noise. arXiv preprint arXiv:1706.03825.Google Scholar
- [55] . 2019. Explainable AI: The Basics, Policy Briefing. Retrieved September 9, 2022 from https://royalsociety.org/topics-policy/projects/explainable-ai/.Google Scholar
- [56] . 2019. The automatic statistician. In Automated Machine Learning: Methods, Systems, Challenges, , , and (Eds.). Springer, 161–164. Google ScholarCross Ref
- [57] . 2017. Axiomatic attribution for deep networks. arXiv preprint arXiv:1703.01365.Google Scholar
- [58] . 2020. Interpretable AI, Building Explainable Machine Learning Systems. Manning Publications. Google Scholar
- [59] . 2003. Three problems in computer science. Journal of the ACM 50, 1 (2003), 96–99.Google ScholarDigital Library
- [60] . 2019. Designing theory-driven user-centric explainable AI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–15.Google ScholarDigital Library
- [61] . 2015. Petuum: A new platform for distributed machine learning on big data. IEEE Transactions on Big Data 1, 2 (2015), 49–67.Google ScholarCross Ref
- [62] . 2018. Explainable recommendation: A survey and new perspectives. CoRR abs/1804.11192.Google Scholar
- [63] Vijay Arya, Rachel K. E. Bellamy, Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Q. Vera Liao, Ronny Luss, Aleksandra Mojsilović, Sami Mourad, Pablo Pedemonte, Ramya Raghavendra, John Richards, Prasanna Sattigeri, Karthikeyan Shanmugam, Moninder Singh, Kush R. Varshney, Dennis Wei, Yunfeng Zhang. 2019. One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques. https://arxiv.org/abs/1909.03012.Google Scholar
Index Terms
- Explainable AI (XAI): Core Ideas, Techniques, and Solutions
Recommendations
The role of explainable AI in the context of the AI Act
FAccT '23: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and TransparencyThe proposed EU regulation for Artificial Intelligence (AI), the AI Act, has sparked some debate about the role of explainable AI (XAI) in high-risk AI systems. Some argue that black-box AI models will have to be replaced with transparent ones, others ...
Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities
AbstractThe past decade has seen significant progress in artificial intelligence (AI), which has resulted in algorithms being adopted for resolving a variety of problems. However, this success has been met by increasing model complexity and ...
Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence
AbstractArtificial intelligence (AI) is currently being utilized in a wide range of sophisticated applications, but the outcomes of many AI models are challenging to comprehend and trust due to their black-box nature. Usually, it is essential to ...
Highlights- A novel four-axis framework to examine a model for robustness and explainability.
- Formulation of research questions at each axis and its corresponding taxonomy.
- Discussion of different explainability assessment methods.
- A novel ...
Comments