nach oben

Erschienen in:

2020 | OriginalPaper | Buchkapitel

Evaluating Interpretability in Machine Teaching

verfasst von : Lars Holmberg, Paul Davidsson, Per Linde

Erschienen in: Highlights in Practical Applications of Agents, Multi-Agent Systems, and Trust-worthiness. The PAAMS Collection

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Building interpretable machine learning agents is a challenge that needs to be addressed to make the agents trustworthy and align the usage of the technology with human values. In this work, we focus on how to evaluate interpretability in a machine teaching setting, a setting that involves a human domain expert as a teacher in relation to a machine learning agent. By using a prototype in a study, we discuss the interpretability definition and show how interpretability can be evaluated on a functional-, human- and application level. We end the paper by discussing open questions and suggestions on how our results can be transferable to other domains.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Industrial Federated Learning – Requirements and System Design

Nächstes Kapitel VRacter: Investigating Character-Based Avatar Preference in Education

Merriam-Webster dictionary, accessed 2020-01-24.

fast.ai.

https://github.com/k3larra/commuter#personas.

https://github.com/k3larra/commuter/blob/master/machine_teaching/mt.ipynb.

Biran, O., Cotton, C.: Explanation and justification in machine learning: a survey. IJCAI Workshop on Explain. AI (XAI) 8(August), 8–14 (2017)

Boukhelifa, N., Bezerianos, A., Lutton, E.: Evaluation of interactive machine learning systems, pp. 1–20 (2018)

Doshi-Velez, F., Kim, B.: Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017)

Doshi-Velez, F., Kim, B.: Considerations for evaluation and generalization in interpretable machine learning. In: Escalante, H.J., et al. (eds.) Explainable and Interpretable Models in Computer Vision and Machine Learning. TSSCML, pp. 3–17. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98131-4_1CrossRef

Dudley, J.J., Kristensson, P.O.: A review of user interface design for interactive machine learning (2018). https://doi.org/10.1145/3185517

Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M., Kagal, L.: Explaining explanations: an overview of interpretability of machine learning. In: Proceedings - 2018 IEEE 5th International Conference on Data Science and Advanced Analytics, DSAA 2018 (2019). https://doi.org/10.1109/DSAA.2018.00018

Graneheim, U., Lundman, B.: Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ. Today 24(2), 105–112 (2004). https://doi.org/10.1016/J.NEDT.2003.10.001CrossRef

Hind, M., et al.: TED: teaching AI to explain its decisions. In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 123–129 (2018). https://doi.org/10.1145/3306618.3314273

HLEG: Ethics Guidelines for Trustworthy AI (European Commission, 2019). Technical report, High-Level Expert Group on Artificial Intelligence (2019). https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai

10.

Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Netw. 2(5), 359–366 (1989). https://doi.org/10.1016/0893-6080(89)90020-8CrossRefMATH

11.

Lakkaraju, H., Bach, S.H., Leskovec, J.: Interpretable decision sets: a joint framework for description and prediction. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, vol. 13–17-August, pp. 1675–1684 (2016). https://doi.org/10.1145/2939672.2939874

12.

Lakkaraju, H., Kamar, E., Caruana, R., Leskovec, J.: Faithful and customizable explanations of black box models. In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 131–138. ACM (2019). www.aaai.org

13.

Lecun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015). https://doi.org/10.1038/nature14539CrossRef

14.

Lindvall, M., Molin, J., Löwgren, J.: From machine learning to machine teaching. Interactions 25(6), 52–57 (2018). https://doi.org/10.1145/3282860CrossRef

15.

Lipton, Z.C.: The mythos of model interpretability. In: ICML Workshop on Human Interpretability in Machine Learning, WHI (2016)

16.

Lou, Y., Caruana, R., Gehrke, J.: Intelligible models for classification and regression. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 150–158 (2012). https://doi.org/10.1145/2339530.2339556

17.

Lundberg, S., Lee, S.I.: An unexpected unity among methods for interpreting model predictions. arXiv preprint arXiv:1611.07478 (2016)

18.

Miller, T.: Explanation in artificial intelligence: insights from the social sciences (2019). https://doi.org/10.1016/j.artint.2018.07.007

19.

Nielsen, L.: Personas - User Focused Design. Springer, London (2013). https://doi.org/10.1007/978-1-4471-4084-9CrossRef

20.

Ribeiro, M.T., Singh, S., Guestrin, C.: Why should I trust you? In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2016, pp. 1135–1144. ACM Press, New York (2016). https://doi.org/10.1145/2939672.2939778

21.

Simard, P.Y., et al.: Machine teaching: a new paradigm for building machine learning systems. Technical report, Microsoft Research (2017). http://arxiv.org/abs/1707.06742

22.

Teso, S., Kersting, K.: Explanatory interactive machine learning. In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 239–245. ACM (2019). https://doi.org/10.1145/3306618.3314293

Titel: Evaluating Interpretability in Machine Teaching
verfasst von: Lars Holmberg
Paul Davidsson
Per Linde
Verlag: Springer International Publishing
Buch: Highlights in Practical Applications of Agents, Multi-Agent Systems, and Trust-worthiness. The PAAMS Collection
Print ISBN: 978-3-030-51998-8

Electronic ISBN: 978-3-030-51999-5

Copyright-Jahr: 2020
DOI: https://doi.org/10.1007/978-3-030-51999-5_5

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"