skip to main content
research-article
Free Access

The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery.

Published:01 June 2018Publication History
Skip Abstract Section

Abstract

Supervised machine-learning models boast remarkable predictive capabilities. But can you trust your model? Will it work in deployment? What else can it tell you about the world?

References

  1. Athey, S., Imbens, G. W. 2015 Machine-learning methods https://arxiv.org/abs/1504.01132v1 (see also ref. 7).Google ScholarGoogle Scholar
  2. Caruana, R., Kangarloo, H., Dionisio, J. D, Sinha, U., Johnson, D. 1999. Case-based explanation of non-case- based learning methods. In Proceedings of the American Medical Informatics Association (AMIA) Symposium: 212-215.Google ScholarGoogle Scholar
  3. Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., Elhadad, N. 2015. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21st Annual SIGKDD International Conference on Knowledge Discovery and Data Mining, 1721-1730. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Chang, J., Gerrish, S., Wang, C., Boyd-Graber, J. L., Blei, D. M. 2009. Reading tea leaves: how humans interpret topic models. In Proceedings of the 22nd International Conference on Neural Information Processing Systems (NIPS), 288-296. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Doshi-Velez, F., Wallace, B., Adams, R. 2015. Graph- sparse lDA: a topic model with structured sparsity. In Proceedings of the 29th Association for the Advancement of Artificial Intelligence (AAAI) Conference, 2575-2581. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. FICO (Fair Isaac Corporation). 2011. Introduction to model builder scorecard; http://www.fico.com/en/latest-thinking/white-papers/introduction-to-model-builder-scorecard.Google ScholarGoogle Scholar
  7. Goodman, B., Flaxman, S. 2016. European Union regulations on algorithmic decision-making and a "right to explanation." https://arxiv.org/abs/1606.08813v3.Google ScholarGoogle Scholar
  8. Huysmans, J., Dejaeger, K., Mues, C., Vanthienen, J., Baesens, B. 2011. An empirical evaluation of the comprehensibility of decision table, tree- and rule- based predictive models. Journal of Decision Support Systems 51(1), 141-154. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Kim, B. 2015. Interactive and interpretable machine- learning models for human-machine collaboration. Ph.D. thesis. Massachusetts Institute of Technology.Google ScholarGoogle Scholar
  10. Kim, B., Rudin, C., Shah, J. A. 2014. The Bayesian case model: A generative approach for case-based reasoning and prototype classification. In Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS), volume 2, 1952-1960. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Kim, B., Glassman, E., Johnson, B., Shah, J. 2015. iBCM: Interactive Bayesian case model empowering humans via intuitive interaction. Massachusetts Institute of Technology, Cambridge, MA.Google ScholarGoogle Scholar
  12. Krening, S., Harrison, B., Feigh, K., Isbell, C., Riedl, M., Thomaz, A. 2017. Learning from explanations using sentiment and advice in RL. IEEE Transactions on Cognitive and Developmental Systems 9(1), 41-55.Google ScholarGoogle ScholarCross RefCross Ref
  13. Lipton, Z. C., Kale, D. C., Wetzel, R. 2016. Modeling missing data in clinical time series with RNNs. In Proceedings of Machine Learning for Healthcare.Google ScholarGoogle Scholar
  14. Liu, C., Rani, P., Sarkar, N. 2006. An empirical study of machine-learning techniques for affect recognition in human-robot interaction. Pattern Analysis and Applications 9(1): 58-69. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Lou, Y., Caruana, R., Gehrke, J. 2012. Intelligible models for classification and regression. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 150-158. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Lou, Y., Caruana, R., Gehrke, J., Hooker, G. 2013. Accurate intelligible models with pairwise interactions. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 623-631. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Mahendran, A., Vedaldi, A. 2015. Understanding deep image representations by inverting them. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1-9.Google ScholarGoogle ScholarCross RefCross Ref
  18. McAuley, J., Leskovec, J. 2013. Hidden factors and hidden topics: understanding rating dimensions with review text. In Proceedings of the 7th ACM Conference on Recommender Systems, 165-172. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., Dean, J. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS), volume 2, 3111?3119. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Mordvintsev, A., Olah, C., Tyka, M. 2015. Inceptionism: going deeper into neural networks. Google AI Blog; https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html.Google ScholarGoogle Scholar
  21. Mounk, Y. 2014. Is Harvard unfair to Asian-Americans? New York Times (Nov. 24); http://www.nytimes.com/2014/11/25/opinion/is-harvard-unfair-to-asian-americans.html.Google ScholarGoogle Scholar
  22. Pearl, J. 2009. Causality. Cambridge University Press.Google ScholarGoogle Scholar
  23. Ribeiro, M. T., Singh, S., Guestrin, C. 2016. "Why should I trust you?": explaining the predictions of any classifier. In Proceedings of the 22nd SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135-1144. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Ridgeway, G., Madigan, D., Richardson, T., O'Kane, J. 1998. Interpretable boosted naïve Bayes classification. In Proceedings of the 4th International Conference on Knowledge Discovery and Data Mining: 101-104. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Simonyan, K., Vedaldi, A., Zisserman, A. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. https://arxiv.org/abs/1312.6034 (see notes to refs 1, 7).Google ScholarGoogle Scholar
  26. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R. 2013. Intriguing properties of neural networks. https://arxiv.org/abs/1312.6199 (see refs 1, 7, 25).Google ScholarGoogle Scholar
  27. Tibshirani, R. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 58(1), 267-288.Google ScholarGoogle ScholarCross RefCross Ref
  28. Van der Maaten, L., Hinton, G. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, 2579-2605.Google ScholarGoogle Scholar
  29. Wang, H.-X., Fratiglioni, L., Frisoni, G. B., Viitanen, M., Winblad, B. 1999. Smoking and the occurrence of Alzheimer's disease: cross-sectional and longitudinal data in a population-based study. American Journal of Epidemiology 149(7), 640-644.Google ScholarGoogle ScholarCross RefCross Ref
  30. Wang, Z., Freitas, N., Lanctot, M. 2016. Dueling network architectures for deep reinforcement learning. Proceedings of the 33rd International Conference on Machine Learning 48, 1995-2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
Index terms have been assigned to the content through auto-classification.

Recommendations

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Sign in

Full Access

  • Published in

    cover image Queue
    Queue  Volume 16, Issue 3
    Machine Learning
    May-June 2018
    118 pages
    ISSN:1542-7730
    EISSN:1542-7749
    DOI:10.1145/3236386
    Issue’s Table of Contents

    Copyright © 2018 ACM

    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    • Published: 1 June 2018

    Permissions

    Request permissions about this article.

    Request Permissions

    Check for updates

    Qualifiers

    • research-article
    • Popular
    • Editor picked

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format .

View HTML Format