Stefano Teso
ABSTRACT
Although interactive learning puts the user into the loop, the learner remains mostly a black box for the user. Understanding the reasons behind predictions and queries is important when assessing how the learner works and, in turn, trust. Consequently, we propose the novel framework of explanatory interactive learning where, in each step, the learner explains its query to the user, and the user interacts by both answering the query and correcting the explanation. We demonstrate that this can boost the predictive and explanatory powers of, and the trust into, the learned model, using text (e.g. SVMs) and image classification (e.g. neural networks) experiments as well as a user study.
- Josh Attenberg et al. 2010. A unified approach to active dual supervision for labeling features and examples. In Proc. of ECML/PKDD. Springer, 40--55. Google Scholar
Digital Library
- Maria-Florina Balcan et al. 2010. The true sample complexity of active learning. Machine learning, Vol. 80, 2 (2010), 111--139. Google ScholarDigital Library
- Osbert Bastani et al. 2017. Interpreting blackbox models via model extraction. arXiv preprint arXiv:1705.08504 (2017).Google Scholar
- Cristian Bucilu? et al. 2006. Model compression. In Proc. of KDD. ACM, 535--541. Google ScholarDigital Library
- Maya Cakmak et al. 2011. Mixed-initiative active learning. ICML 2011 Workshop on Combining Learning Strategies to Reduce Label Cost (2011).Google Scholar
- Rui M Castro et al. 2006. Upper and lower error bounds for active learning. In The 44th Annual Allerton Conference on Communication, Control and Computing, Vol. 2. 1.Google Scholar
- Li Chen et al. 2012. Critiquing-based recommenders: survey and emerging trends. User Modeling and User-Adapted Interaction, Vol. 22, 1--2 (2012), 125--150. Google ScholarDigital Library
- Corinna Cortes et al. 1995. Support-vector networks. Machine learning, Vol. 20, 3 (1995), 273--297. Google ScholarDigital Library
- Gerald DeJong et al. 2011. Explanation-based learning. In Encyclopedia of Machine Learning. Springer, 388--392.Google Scholar
- Gregory Druck et al. 2008. Learning from labeled features using generalized expectation criteria. In Proc. of SIGIR. 595--602. Google ScholarDigital Library
- Gregory Druck et al. 2009. Active learning by labeling features. In Proc. of EMNLP. 81--90. Google ScholarDigital Library
- Yarin Gal et al. 2017. Deep Bayesian Active Learning with Image Data. In Proc. of ICML. 1183--1192. Google ScholarDigital Library
- Steve Hanneke et al. 2014. Theory of disagreement-based active learning. Foundations and Trends® in Machine Learning, Vol. 7, 2--3 (2014), 131--309. Google ScholarDigital Library
- Robert R Hoffman et al. 2013. Trust in automation. IEEE Intelligent Systems, Vol. 28, 1 (2013), 84--88. Google ScholarDigital Library
- Kshitij Judah et al. 2012. Active Imitation Learning via Reduction to IID Active Learning.. In UAI. 428--437. Google ScholarDigital Library
- Angelika Kimmig et al. 2007. Probabilistic explanation based learning. In Proc. of ECML. 176--187. Google ScholarDigital Library
- Andreas Krause et al. 2007. Nonmyopic active learning of gaussian processes: an exploration-exploitation approach. In ICML. ACM, 449--456. Google ScholarDigital Library
- Todd Kulesza et al. 2015. Principles of explanatory debugging to personalize interactive machine learning. In Proc. of IUI. 126--137. Google ScholarDigital Library
- Scott Lundberg et al. 2016. An unexpected unity among methods for interpreting model predictions. arXiv preprint arXiv:1611.07478 (2016).Google Scholar
- Tom M Mitchell et al. 1986. Explanation-based generalization: A unifying view. Machine learning, Vol. 1, 1 (1986), 47--80. Google ScholarDigital Library
- Phillip Odom and Sriraam Natarajan. 2018. Human-Guided Learning for Probabilistic Logic Models. Front. Robotics and AI, Vol. 2018 (2018).Google ScholarCross Ref
- Bart Peintner et al. 2008. Preferences in interactive systems: Technical challenges and case studies. AI Magazine, Vol. 29, 4 (2008), 13.Google ScholarCross Ref
- Hema Raghavan et al. 2006. Active learning with feedback on features and instances. JMLR, Vol. 7, Aug (2006), 1655--1686. Google ScholarDigital Library
- Hema Raghavan et al. 2007. An interactive algorithm for asking and incorporating feature feedback into support vector machines. In Proc. of SIGIR. 79--86. Google ScholarDigital Library
- Marco Tulio Ribeiro et al. 2016. Why should i trust you?: Explaining the predictions of any classifier. In Proc. of KDD. 1135--1144. Google ScholarDigital Library
- Marco Tulio Ribeiro et al. 2018. Anchors: High-Precision Model-Agnostic Explanations. In Proc. of AAAI .Google Scholar
- Andrew Slavin Ross et al. 2017. Right for the right reasons: training differentiable models by constraining their explanations. In IJCAI. AAAI Press, 2662--2670. Google ScholarDigital Library
- Nicholas Roy et al. 2001. Toward optimal active learning through monte carlo estimation of error reduction. ICML (2001), 441--448. Google ScholarDigital Library
- Tobias Schnabel et al. 2018. Short-Term Satisfaction and Long-Term Coverage: Understanding How Users Tolerate Algorithmic Exploration. In Proc. of WSDM. ACM, 513--521. Google ScholarDigital Library
- Burr Settles. 2011. Closing the loop: Fast, interactive semi-supervised annotation with queries on features and instances. In Proc. EMNLP. 1467--1478. Google ScholarDigital Library
- Burr Settles. 2012. Active learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, Vol. 6, 1 (2012), 1--114. Google ScholarDigital Library
- Manali Sharma et al. 2015. Active learning with rationales for text classification. In NAACL HLT. 441--451.Google Scholar
- Pannaga Shivaswamy et al. 2015. Coactive Learning. J. Artif. Intell. Res.(JAIR), Vol. 53 (2015), 1--40. Google ScholarDigital Library
- Jeffry A Simpson. 2007. Psychological foundations of trust. Current directions in psychological science, Vol. 16, 5 (2007), 264--268.Google Scholar
- Kevin Small et al. 2011. The constrained weight space svm: learning with ranked features. In ICML. Omnipress, 865--872. Google ScholarDigital Library
- Simone Stumpf et al. 2009. Interacting meaningfully with machine learning systems: Three experiments. International Journal of Human-Computer Studies, Vol. 67, 8 (2009), 639--662. Google ScholarDigital Library
- Simon Tong and Daphne Koller. 2001. Support vector machine active learning with applications to text classification. JMLR, Vol. 2, Nov (2001), 45--66. Google ScholarDigital Library
- Ning Wang et al. 2016. Trust calibration within a human-robot team: Comparing automatically generated explanations. In Proc. of HRI. 109--116. Google ScholarDigital Library
- Adam Waytz et al. 2014. The mind in the machine: Anthropomorphism increases trust in an autonomous vehicle. Journal of Experimental Social Psychology, Vol. 52 (1 5 2014), 113--117.Google Scholar
- Omar Zaidan et al. 2007. Using "Annotator Rationales" to Improve Machine Learning for Text Categorization. In NAACL HLT. 260--267.Google Scholar
- Omar F Zaidan et al. 2008. Modeling annotators: A generative approach to learning from annotator rationales. In Proc. EMNLP. 31--40. Google ScholarDigital Library
- Ji Zhu et al. 2004. 1-norm support vector machines. In Proc. of NIPS. 49--56. Google ScholarDigital Library
Index Terms
- Explanatory Interactive Machine Learning
Recommendations
Explanatory Pluralism in Explainable AI
Machine Learning and Knowledge ExtractionAbstractThe increasingly widespread application of AI models motivates increased demand for explanations from a variety of stakeholders. However, this demand is ambiguous because there are many types of ‘explanation’ with different evaluative criteria. In ...
CSSE - An agnostic method of counterfactual, selected, and social explanations for classification models
AbstractIn some contexts, achieving high predictive capability may be sufficient for a machine learning model. However, in many scenarios, it is necessary to understand the model’s decisions to increase confidence in the predictions and direct the ...
Highlights- CSSE generates local explanations for classification models using a genetic algorithm.
- CSSE offers contrastive, selected, and social explanations growing user understanding.
- CSSE presents explanations with diversity, actionality, ...
Applying the Polysynchronous Learning to Foster the Student-centered Learning in the Higher Education Context: A Blended Course Design
Teaching and learning computer programming has posed great challenges for both instructors and students. An emerging blended learning mode-polysynchronous learning has potentials to motivate and engage students in the programming learning process, to ...
Comments