Michelle Vaccaro
Abstract
Collaboration between humans and machines does not necessarily lead to better outcomes.
- Angwin, J., Larson, J. Machine bias. ProPublica (May 23, 2016).Google Scholar
- Case, N. How to become a centaur. J. Design and Science (Jan. 2018).Google Scholar
- Chouldechova, A. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data 5, 2 (2017), 153--163.Google ScholarCross Ref
- Corbett-Davies, S., Pierson, E., Feller, A., Goel, S. and Huq, A. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd ACM SIGKDD Intern. Conf. Knowledge Discovery and Data Mining. ACM Press, 2017, 797--806.Google ScholarDigital Library
- Critcher, C.R. and Gilovich, T. Incidental environmental anchors. J. Behavioral Decision Making 21, 3 (2008), 241--251.Google ScholarCross Ref
- Dressel, J. and Farid, H. The accuracy, fairness, and limits of predicting recidivism. Science Advances 4, 1 (2018), eaao5580.Google ScholarCross Ref
- Englich, B., Mussweiler, T. and Strack, F. Playing dice with criminal sentences: the influence of irrelevant anchors on experts' judicial decision making. Personality and Social Psychology Bulletin 32, 2 (2006), 188--200.Google ScholarCross Ref
- Furnham, A. and Boo, H.C. A literature review of the anchoring effect. The J. Socio-Economics 40, 1 (2011), 35--42.Google ScholarCross Ref
- Goldstein, I.M., Lawrence, J. and Miner, A.S. Human-machine collaboration in cancer and beyond: The Centaur Care Model. JAMA Oncology 3, 10 (2017), 1303.Google Scholar
- Green, K.C. and Armstrong, J.S. Evidence on the effects of mandatory disclaimers in advertising. J. Public Policy & Marketing 31, 2 (2012), 293--304.Google ScholarCross Ref
- Horvitz, E. and Paek, T. Complementary computing: policies for transferring callers from dialog systems to human receptionists. User Modeling and User-Adapted Interaction 17, 1--2 (2007), 159--182.Google ScholarCross Ref
- Johnson, R.C. Overcoming AI bias with AI fairness. Commun. ACM (Dec. 6, 2018).Google Scholar
- Jukier, R. Inside the judicial mind: exploring judicial methodology in the mixed legal system of Quebec. European J. Comparative Law and Governance (Feb. 2014).Google Scholar
- Kahneman, D. Thinking, Fast and Slow. Farrar, Straus and Giroux, 2011.Google Scholar
- Mussweiler, T. and Strack, F. Numeric judgments under uncertainty: the role of knowledge in anchoring. J. Experimental Social Psychology 36, 5 (2000), 495--518.Google ScholarCross Ref
- Northcraft, G.B. and Neale, M.A. Experts, amateurs, and real estate: an anchoring-and-adjustment perspective on property pricing decisions. Organizational Behavior and Human Decision Processes 39, 1 (1987), 84--97.Google ScholarCross Ref
- Shaw, A.D., Horton, J.J. and Chen, D.L. Designing incentives for inexpert human raters. In Proceedings of the ACM Conf. Computer-supported Cooperative Work. ACM Press, 2011, 275--284.Google ScholarDigital Library
- State v Loomis, 2016.Google Scholar
- Tversky, A. and Kahneman, D. Judgment under uncertainty: Heuristics and biases. Science 185, 4157 (1974), 1124--1131.Google ScholarCross Ref
- Wansink, B., Kent, R.J. and Hoch, S.J. An anchoring and adjustment model of purchase quantity decisions. J. Marketing Research 35, 1 (1998), 71.Google ScholarCross Ref
Index Terms
- The effects of mixing machine learning and human judgment
Recommendations
The Effects of Mixing Machine Learning and Human Judgment: Collaboration between humans and machines does not necessarily lead to better outcomes.
Machine Learning, SecurityBased on the theoretical findings from the existing literature, some policymakers and software engineers contend that algorithmic risk assessments such as the COMPAS software can alleviate the incarceration epidemic and the occurrence of violent crimes ...
Ball bonding inspections using a conjoint framework with machine learning and human judgement
AbstractBall bonding inspections with human vision are essential in manufacturing processes of semiconductors devices and integrated circuits (ICs). The inspections are an intensive task which involves human labours to detect poor bonds. ...
Highlights- Ball bonding inspections with human vision are essential in manufacturing ICs.
- ...
Introduction to the Special Issue on Human-Centered Machine Learning
Special Issue on Human-Centered Machine LearningMachine learning is one of the most important and successful techniques in contemporary computer science. Although it can be applied to myriad problems of human interest, research in machine learning is often framed in an impersonal way, as merely ...
Reviews
Jonathan K. Millen
Automated risk assessment systems are often used in situations that require human judgment. One motivation for doing this is to remove human bias. Even when the automated system has been shown to be more accurate than human assessments, a team combining system and human decisions has occasionally been proven to be better for some applications and collaboration modes. The presented experiment involves criminal recidivism assessments using a well-known algorithmic system, COMPAS. Human subjects were recruited according to their interest, rather than their expertise, in criminal justice. As expected, COMPAS was more accurate by itself than the humans, given the same data from real court cases. The pertinent question, however, is whether, and in what way, the human results are influenced by being told the COMPAS results prior to making their own assessments. In the first trial, humans were told the COMPAS recidivism risk scores, and their own scores were (on the average) different and less accurate. In the second trial, the experimenters investigated an "anchoring" effect by providing COMPAS results that were deliberately altered higher or lower. The average human scores differed significantly in the direction of the altered COMPAS scores that they were given. These results are not world shaking, and the article is short, less than seven pages. Yet it was as absorbing as a mystery novel, and raised all sorts of questions that aroused the hope of further work. For example, the cited success of teaming involves a feedback loop between the humans and the system, which was not tried here. Also, would the experiment have come out differently with expert humans on the team There is so much more to learn.
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments