Bruce Glymour
ABSTRACT
Measures of algorithmic bias can be roughly classified into four categories, distinguished by the conditional probabilistic dependencies to which they are sensitive. First, measures of "procedural bias" diagnose bias when the score returned by an algorithm is probabilistically dependent on a sensitive class variable (e.g. race or sex). Second, measures of "outcome bias" capture probabilistic dependence between class variables and the outcome for each subject (e.g. parole granted or loan denied). Third, measures of "behavior-relative error bias" capture probabilistic dependence between class variables and the algorithmic score, conditional on target behaviors (e.g. recidivism or loan default). Fourth, measures of "score-relative error bias" capture probabilistic dependence between class variables and behavior, conditional on score. Several recent discussions have demonstrated a tradeoff between these different measures of algorithmic bias, and at least one recent paper has suggested conditions under which tradeoffs may be minimized.
In this paper we use the machinery of causal graphical models to show that, under standard assumptions, the underlying causal relations among variables forces some tradeoffs. We delineate a number of normative considerations that are encoded in different measures of bias, with reference to the philosophical literature on the wrongfulness of disparate treatment and disparate impact. While both kinds of error bias are nominally motivated by concern to avoid disparate impact, we argue that consideration of causal structures shows that these measures are better understood as complicated and unreliable measures of procedural biases (i.e. disparate treatment). Moreover, while procedural bias is indicative of disparate treatment, we show that the measure of procedural bias one ought to adopt is dependent on the account of the wrongfulness of disparate treatment one endorses. Finally, given that neither score-relative nor behavior-relative measures of error bias capture the relevant normative considerations, we suggest that error bias proper is best measured by score-based measures of accuracy, such as the Brier score.
- Elizabeth S. Anderson. 1999. What Is the Point of Equality? Ethics 109, 2 (January 1999), 287--337.Google Scholar
Cross Ref
- Richard Arneson. 2013. Discrimination, Disparate Impact, and Theories of Justice. In Philosophical Foundations of Discrimination Law. Oxford University Press, Oxford, 87--113. Retrieved August 22, 2018 fromGoogle Scholar
- Solon Barocas and Moritz Hardt. 2017. Fairness in Machine Learning. In Conference on Neural Information Processing Systems, 2017.Google Scholar
- Solon Barocas and Andrew D. Selbst. 2016. Big data's disparate impact. Calif. L. Rev. 104, (2016), 671--732.Google Scholar
- Richard Berk, Hoda Heidari, Shahin Jabbari, Michael Kearns, and Aaron Roth. 2017. Fairness in Criminal Justice Risk Assessments: The State of the Art. arXiv:1703.09207 {stat} (March 2017). Retrieved November 2, 2017 from http://arxiv.org/abs/1703.09207Google Scholar
- Reuben Binns. 2017. Fairness in Machine Learning: Lessons from Political Philosophy. arXiv: 1712.03586 {cs} (December 2017). Retrieved July 12, 2018 from http://arxiv.org/abs/1712.03586Google Scholar
- Bernard R. Boxill. 1992. Blacks and Social Justice (2nd ed.). Rowman & Littlefield Publishers, Lanham, Md.Google Scholar
- Glenn W. Brier. 1950. Verification of forecasts expressed in terms of probability. Mon. Wea. Rev. 78, 1 (January 1950), 1--3.Google ScholarCross Ref
- Alexandra Chouldechova. 2016. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. In arXiv:1610.07524 {cs, stat}. Retrieved November 7, 2017 from http://arxiv.org/abs/1610.07524Google Scholar
- Sam Corbett-Davies and Sharad Goel. 2018. The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning. arXiv:1808.00023 {cs} (July 2018). Retrieved August 22, 2018 from http://arxiv.org/abs/1808.00023Google Scholar
- Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. 2017. Algorithmic decision making and the cost of fairness. arXiv:1701.08230 {cs, stat} (January 2017). Google ScholarDigital Library
- Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Rich Zemel. 2011. Fairness Through Awareness. arXiv:1104.3913 {cs} (April 2011). Retrieved November 17, 2018 from http://arxiv.org/abs/1104.3913Google Scholar
- Ronald Dworkin. 1978. Taking Rights Seriously: With a New Appendix, a Response to Critics. Harvard University Press, Cambridge, Mass.Google Scholar
- Michael Feldman, Sorelle Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2014. Certifying and removing disparate impact. arXiv:1412.3756 {cs, stat} (December 2014). Retrieved November 18, 2018 from http://arxiv.org/abs/1412.3756Google Scholar
- Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. 2016. On the (im)possibility of fairness. arXiv:1609.07236 {cs, stat} (September 2016). Retrieved August 22, 2018 from http://arxiv.org/abs/1609.07236Google Scholar
- Moritz Hardt, Eric Price, and Nathan Srebro. 2016. Equality of Opportunity in Supervised Learning. arXiv:1610.02413 {cs} (October 2016). Retrieved December 4, 2017 from http://arxiv.org/abs/1610.02413Google Scholar
- Deborah Hellman. 2008. When Is Discrimination Wrong? Harvard University Press, Cambridge, MA.Google Scholar
- Richard D. Kahlenberg. 1997. The Remedy: Class, Race, And Affirmative Action. Basic Books.Google Scholar
- John Kekes. 1993. The Injustice of Strong Affirmative Action. In Affirmative Action and the University. Temple University Press, 144--156. Retrieved from http://www.jstor.org/stable/j.ctt14bs9hb.10Google Scholar
- Niki Kilbertus, Mateo Rojas Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Schölkopf. 2017. Avoiding Discrimination through Causal Reasoning. In Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan and R. Garnett (eds.). Curran Associates, Inc., 656--666. Retrieved November 16, 2018 from http://papers.nips.cc/paper/6668-avoiding-discrimination-through-causal-reasoning.pdf Google ScholarDigital Library
- Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2016. Inherent Trade-Offs in the Fair Determination of Risk Scores. In Proceedings of Innovations in Theoretical Computer Science (ITCS). Retrieved November 7, 2017 from http://arxiv.org/abs/1609.05807Google Scholar
- Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual Fairness. In Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan and R. Garnett (eds.). Curran Associates, Inc., 4066--4076. Retrieved November 16, 2018 from http://papers.nips.cc/paper/6995-counterfactual-fairness.pdf Google ScholarDigital Library
- Matt J. Kusner, Chris Russell, Joshua R. Loftus, and Ricardo Silva. 2018. Causal Interventions for Fairness. (June 2018). Retrieved November 16, 2018 from https://arxiv.org/abs/1806.02380Google Scholar
- Kasper Lippert-rasmussen. 2006. The badness of discrimination. Ethic Theory Moral Prac 9, 2 (April 2006), 167--185.Google Scholar
- Kasper Lippert-Rasmussen. 2014. Indirect Discrimination is Not Necessarily Unjust. Journal of Practical Ethics 2, 2 (2014), 33--57.Google Scholar
- Zachary C. Lipton, Alexandra Chouldechova, and Julian McAuley. 2017. Does mitigating ML's impact disparity require treatment disparity? arXiv:1711.07076 {cs, stat} (November 2017). Retrieved March 12, 2018 from http://arxiv.org/abs/1711.07076Google Scholar
- Judea Pearl. 2009. Causality: Models, Reasoning and Inference (2nd ed.). Cambridge University Press, New York. Google ScholarDigital Library
- Philip Pettit. 1999. Republicanism: a theory of freedom and government. Oxford University Press, Oxford.Google Scholar
- T. M. Scanlon. 2010. Moral Dimensions: Permissibility, Meaning, Blame (Reprint edition ed.). Belknap Press, Cambridge, Mass.Google Scholar
- Richard Schemes. 1997. An Introduction to Causal Inference. In Causality in Crisis? Statistical Methods and the Search for Causal Knowledge in the Social Sciences. University of Notre Dame Press, South Bend, IN. Retrieved from https:/www.cmu.edu/dietrich/philosophy/docs/spirtes/notredame.psGoogle Scholar
- Shlomi Segall. 2012. What's so Bad about Discrimination? Utilitas 24, 1 (March 2012), 82--100.Google ScholarCross Ref
- Patrick S. Shin. 2009. The Substantive Principle of Equal Treatment. Legal Theory 15, 2 (June 2009), 149--172.Google ScholarCross Ref
- Peter Spirtes, Clark Glymour, and Richard Schemes. 2001. Causation, Prediction, and Search (2nd ed.). MIT Press, Cambridge, Mass.Google Scholar
- Cass R. Sunstein. 1994. The Anticaste Principle. Michigan Law Review 92, 8 (1994), 2410--2455.Google ScholarCross Ref
- Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi. 2017. Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment. Proceedings of the 26th International Conference on World Wide Web - WWW '17 (2017), 1171--1180. Google ScholarDigital Library
- Indre Zliobaite. 2015. On the relation between accuracy and fairness in binary classification. arXiv:1505.05723 {cs} (May 2015). Retrieved August 23, 2018 from http://arxiv.org/abs/1505.05723Google Scholar
Index Terms
- Measuring the Biases that Matter: The Ethical and Casual Foundations for Measures of Fairness in Algorithms
Recommendations
Measuring Fairness in an Unfair World
AIES '20: Proceedings of the AAAI/ACM Conference on AI, Ethics, and SocietyComputer scientists have made great strides in characterizing different measures of algorithmic fairness, and showing that certain measures of fairness cannot be jointly satisfied. In this paper, I argue that the three most popular families of measures -...
Measuring justice in machine learning
FAT* '20: Proceedings of the 2020 Conference on Fairness, Accountability, and TransparencyHow can we build more just machine learning systems? To answer this question, we need to know both what justice is and how to tell whether one system is more or less just than another. That is, we need both a definition and a measure of justice. ...
Disambiguating Algorithmic Bias: From Neutrality to Justice
AIES '23: Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and SocietyAs algorithms have become ubiquitous in consequential domains, societal concerns about the potential for discriminatory outcomes have prompted urgent calls to address algorithmic bias. In response, a rich literature across computer science, law, and ...
Comments