Ninareh Mehrabi
Fred Morstatter
Skip Abstract Section
Abstract
With the widespread use of artificial intelligence (AI) systems and applications in our everyday lives, accounting for fairness has gained significant importance in designing and engineering of such systems. AI systems can be used in many sensitive environments to make important and life-changing decisions; thus, it is crucial to ensure that these decisions do not reflect discriminatory behavior toward certain groups or populations. More recently some work has been developed in traditional machine learning and deep learning that address such challenges in different subdomains. With the commercialization of these systems, researchers are becoming more aware of the biases that these applications can contain and are attempting to address them. In this survey, we investigated different real-world applications that have shown biases in various ways, and we listed different sources of biases that can affect AI applications. We then created a taxonomy for fairness definitions that machine learning researchers have defined to avoid the existing bias in AI systems. In addition to that, we examined different domains and subdomains in AI showing what researchers have observed with regard to unfair outcomes in the state-of-the-art methods and ways they have tried to address them. There are still many future directions and solutions that can be taken to mitigate the problem of bias in AI systems. We are hoping that this survey will motivate researchers to tackle these issues in the near future by observing existing work in their respective fields.
- Alekh Agarwal, Miroslav Dudik, and Zhiwei Steven Wu. 2019. Fair regression: Quantitative definitions and reduction-based algorithms. In Proceedings of the International Conference on Machine Learning. 120–129.Google Scholar
- Sina Aghaei, Mohammad Javad Azizi, and Phebe Vayanos. 2019. Learning optimal and fair decision trees for non-discriminative decision-making. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 1418–1426.Google ScholarDigital Library
- Nazanin Alipourfard, Peter G. Fennell, and Kristina Lerman. 2018. Can you trust the trend? Discovering Simpson’s paradoxes in social data. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining. ACM, 19–27.Google Scholar
- Nazanin Alipourfard, Peter G. Fennell, and Kristina Lerman. 2018. Using Simpson’s paradox to discover interesting patterns in behavioral data. In Proceedings of the 12th International AAAI Conference on Web and Social Media.Google Scholar
- Alexander Amini, Ava P. Soleimany, Wilko Schwarting, Sangeeta N. Bhatia, and Daniela Rus. 2019. Uncovering and mitigating algorithmic bias through learned latent structure. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (AIES’19). Association for Computing Machinery, New York, NY, USA, 289–295. https://doi.org/10.1145/3306618.3314243Google ScholarDigital Library
- Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2019. Machine bias: There’s software used across the country to predict future criminals. and it’s biased against blacks. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.Google Scholar
- A. Asuncion and D. J. Newman. 2007. UCI Machine Learning Repository. Retrieved from http://www.ics.uci.edu/$∖sim$mlearn/{MLR}epository.html.Google Scholar
- Arturs Backurs, Piotr Indyk, Krzysztof Onak, Baruch Schieber, Ali Vakilian, and Tal Wagner. 2019. Scalable fair clustering. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.), Vol. 97. PMLR, 405–413. Retrieved from http://proceedings.mlr.press/v97/backurs19a.html.Google Scholar
- Ricardo Baeza-Yates. 2018. Bias on the web. Commun. ACM 61, 6 (May 2018), 54–61. DOI:DOI:https://doi.org/10.1145/3209581Google ScholarDigital Library
- Samuel Barbosa, Dan Cosley, Amit Sharma, and Roberto M Cesar Jr. 2016. Averaging gone wrong: Using time-aware analyses to better understand behavior. In Proceedings of the 25th International Conference on World Wide Web. 829–841.Google ScholarDigital Library
- Rachel K. E. Bellamy, Kuntal Dey, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovic et al. 2018. AI fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. arXiv preprint arXiv:1810.01943 (2018).Google Scholar
- Emily M. Bender and Batya Friedman. 2018. Data statements for natural language processing: Toward mitigating system bias and enabling better science. Trans. Assoc. Comput. Ling. 6 (2018), 587–604. DOI:DOI:https://doi.org/10.1162/tacl_a_00041Google ScholarCross Ref
- Misha Benjamin, Paul Gagnon, Negar Rostamzadeh, Chris Pal, Yoshua Bengio, and Alex Shee. 2019. Towards standardization of data licenses: The Montreal data license.Google Scholar
- Richard Berk, Hoda Heidari, Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, Seth Neel, and Aaron Roth. 2017. A Convex Framework for Fair Regression. arxiv:cs.LG/1706.02409 (2017).Google Scholar
- Richard Berk, Hoda Heidari, Shahin Jabbari, Michael Kearns, and Aaron Roth. 2021. Fairness in Criminal Justice Risk Assessments: The State of the Art. Sociological Methods & Research 50, 1 (2021), 3–44. https://doi.org/10.1177/0049124118782533Google ScholarCross Ref
- Peter J. Bickel, Eugene A. Hammel, and J. William O’Connell. 1975. Sex bias in graduate admissions: Data from Berkeley. Science 187, 4175 (1975), 398–404.Google Scholar
- R. D. P. Binns. 2018. Fairness in machine learning: Lessons from political philosophy. J. Mach. Learn. Res. (2018).Google Scholar
- Colin R. Blyth. 1972. On Simpson’s paradox and the sure-thing principle. J. Amer. Statist. Assoc. 67, 338 (1972), 364–366.Google ScholarCross Ref
- Miranda Bogen and Aaron Rieke. 2018. Help Wanted: An Examination of Hiring Algorithms, Equity and Bias. Technical Report. Upturn.Google Scholar
- Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou, Venkatesh Saligrama, and Adam T. Kalai. 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 4349–4357.Google Scholar
- Shikha Bordia and Samuel Bowman. 2019. Identifying and reducing gender bias in word-level language models. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop. 7–15.Google ScholarCross Ref
- Avishek Bose and William Hamilton. 2019. Compositional fairness constraints for graph embeddings. In Proceedings of the International Conference on Machine Learning. 715–724.Google Scholar
- Marc-Etienne Brunet, Colleen Alkalay-Houlihan, Ashton Anderson, and Richard Zemel. 2019. Understanding the origins of bias in word embeddings. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.), Vol. 97. PMLR, 803–811. Retrieved from http://proceedings.mlr.press/v97/brunet19a.html.Google Scholar
- Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (Proceedings of Machine Learning Research), Sorelle A. Friedler and Christo Wilson (Eds.), Vol. 81. PMLR, 77–91. Retrieved from http://proceedings.mlr.press/v81/buolamwini18a.html.Google Scholar
- Toon Calders and Sicco Verwer. 2010. Three naive Bayes approaches for discrimination-free classification. Data Mining Knowl. Discov. 21, 2 (2010), 277–292.Google ScholarDigital Library
- Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science 356, 6334 (2017), 183–186.Google Scholar
- Flavio Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R. Varshney. 2017. Optimized pre-processing for discrimination prevention. In Proceedings of the International Conference on Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). Curran Associates, Inc., 3992–4001. Retrieved from http://papers.nips.cc/paper/6988-optimized-pre-processing-for-discrimination-prevention.pdf.Google Scholar
- Manel Capdevila, Marta Ferrer, and Eulália Luque. 2005. La reincidencia en el delito en la justicia de menores. Centro de Estudios jurídicos y formación especializada, Generalitat de Catalunya. Documento no publicado (2005).Google Scholar
- Allison J. B. Chaney, Brandon M. Stewart, and Barbara E. Engelhardt. 2018. How algorithmic confounding in recommendation systems increases homogeneity and decreases utility. In Proceedings of the 12th ACM Conference on Recommender Systems. ACM, 224–232.Google Scholar
- Jiahao Chen, Nathan Kallus, Xiaojie Mao, Geoffry Svacha, and Madeleine Udell. 2019. Fairness under unawareness: Assessing disparity when protected class is unobserved. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, 339–348.Google ScholarDigital Library
- Xingyu Chen, Brandon Fain, Liang Lyu, and Kamesh Munagala. 2019. Proportionally fair clustering. In Proceedings of the International Conference on Machine Learning. 1032–1041.Google Scholar
- S. Chiappa. 2019. Path-specific counterfactual fairness. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 7801–7808.Google ScholarCross Ref
- S. Chiappa and W. S. Isaac. 2019. A causal Bayesian networks viewpoint on fairness. In Privacy and Identity Management. Fairness, Accountability, and Transparency in the Age of Big Data. Privacy and Identity 2018. IFIP Advances in Information and Communication Technology, E. Kosta, J. Pierson, D. Slamanig, S. Fischer-Hübner, S. Krenn (Eds.,) Vol. 547. Springer, Cham.Google Scholar
- Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data 5, 2 (2017), 153–163.Google ScholarCross Ref
- Alexandra Chouldechova, Diana Benavides-Prado, Oleksandr Fialko, and Rhema Vaithianathan. 2018. A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (Proceedings of Machine Learning Research), Sorelle A. Friedler and Christo Wilson (Eds.), Vol. 81. PMLR, 134–148. Retrieved from http://proceedings.mlr.press/v81/chouldechova18a.html.Google Scholar
- Alexandra Chouldechova and Aaron Roth. 2018. The frontiers of fairness in machine learning. arXiv preprint arXiv:1810.08810 (2018).Google Scholar
- John S. Chuang, Olivier Rivoire, and Stanislas Leibler. 2009. Simpson’s paradox in a synthetic microbial system. Science 323, 5911 (2009), 272–275.Google Scholar
- Kevin A. Clarke. 2005. The phantom menace: Omitted variable bias in econometric research. Conflict Manag. Peace Sci. 22, 4 (2005), 341–352.Google ScholarCross Ref
- Lee Cohen, Zachary C. Lipton, and Yishay Mansour. 2019. Efficient candidate screening under multiple tests and implications for fairness. arxiv:cs.LG/1905.11361 (2019).Google Scholar
- United States. Equal Employment Opportunity Commission. 1987. EEOC Compliance Manual. U.S. Equal Employment Opportunity Commission, Washington, D.C..Google Scholar
- Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. 2017. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 797–806.Google ScholarDigital Library
- Elliot Creager, David Madras, Joern-Henrik Jacobsen, Marissa Weis, Kevin Swersky, Toniann Pitassi, and Richard Zemel. 2019. Flexibly fair representation learning by disentanglement. In Proceedings of the International Conference on Machine Learning. 1436–1445.Google Scholar
- Brian d’Alessandro, Cathy O’Neil, and Tom LaGatta. 2017. Conscientious classification: A data scientist’s guide to discrimination-aware classification. Big Data 5, 2 (2017), 120–134.Google ScholarCross Ref
- David Danks and Alex John London. 2017. Algorithmic bias in autonomous systems. In Proceedings of the International Joint Conference on Artificial Intelligence. 4691–4697.Google ScholarCross Ref
- Shai Danziger, Jonathan Levav, and Liora Avnaim-Pesso. 2011. Extraneous factors in judicial decisions. Proc. Nat. Acad. Sci. 108, 17 (2011), 6889–6892.Google ScholarCross Ref
- Julia Dressel and Hany Farid. 2018. The accuracy, fairness, and limits of predicting recidivism. Sci. Adv. 4, 1 (2018). DOI:DOI:https://doi.org/10.1126/sciadv.aao5580Google Scholar
- Dheeru Dua and Casey Graff. 2017. UCI Machine Learning Repository. Retrieved from http://archive.ics.uci.edu/ml.Google Scholar
- Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference (ITCS’12). ACM, New York, NY, 214–226. DOI:DOI:https://doi.org/10.1145/2090236.2090255Google ScholarDigital Library
- Cynthia Dwork, Nicole Immorlica, Adam Tauman Kalai, and Max Leiserson. 2018. Decoupled classifiers for group-fair and efficient machine learning. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (Proceedings of Machine Learning Research), Sorelle A. Friedler and Christo Wilson (Eds.), Vol. 81. PMLR, 119–133. Retrieved from http://proceedings.mlr.press/v81/dwork18a.html.Google Scholar
- Golnoosh Farnadi, Behrouz Babaki, and Lise Getoor. 2018. Fairness in relational domains. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES’18). ACM, New York, NY, 108–114. DOI:DOI:https://doi.org/10.1145/3278721.3278733Google ScholarDigital Library
- Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015. Certifying and removing disparate impact. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’15). Association for Computing Machinery, New York, NY, 259–268. DOI:DOI:https://doi.org/10.1145/2783258.2783311Google ScholarDigital Library
- Joel Escudé Font and Marta R. Costa-Jussà. 2019. Equalizing gender biases in neural machine translation with word embeddings techniques. arXiv preprint arXiv:1901.03116 (2019).Google Scholar
- Batya Friedman and Helen Nissenbaum. 1996. Bias in computer systems. ACM Trans. Inf. Syst. 14, 3 (July 1996), 330–347. DOI:DOI:https://doi.org/10.1145/230538.230561Google ScholarDigital Library
- Anna Fry, Thomas J. Littlejohns, Cathie Sudlow, Nicola Doherty, Ligia Adamska, Tim Sprosen, Rory Collins, and Naomi E. Allen. 2017. Comparison of sociodemographic and health-related characteristics of UK biobank participants with those of the general population. Amer. J. Epidem. 186, 9 (06 2017), 1026–1034. DOI:DOI:https://doi.org/10.1093/aje/kwx246Google ScholarCross Ref
- Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, and Kate Crawford. 2018. Datasheets for datasets.Google Scholar
- C. E. Gehlke and Katherine Biehl. 1934. Certain effects of grouping upon the size of the correlation coefficient in census tract material. J. Amer. Statist. Assoc. 29, 185A (1934), 169–170. DOI:DOI:https://doi.org/10.2307/2277827Google Scholar
- Naman Goel, Mohammad Yaghini, and Boi Faltings. 2018. Non-discriminatory machine learning through convex fairness criteria. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence.Google ScholarDigital Library
- Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. arXiv preprint arXiv:1903.03862 (2019).Google Scholar
- Sandra González-Bailón, Ning Wang, Alejandro Rivero, Javier Borge-Holthoefer, and Yamir Moreno. 2014. Assessing the bias in samples of large online networks. Soc. Netw. 38 (2014), 16–27.Google ScholarCross Ref
- Susan T. Gooden. 2015. Race and Social Equity: A Nervous Area of Government. Routledge.Google Scholar
- Nina Grgic-Hlaca, Muhammad Bilal Zafar, Krishna P. Gummadi, and Adrian Weller. 2016. The case for process fairness in learning: Feature selection for fair decision making. In Proceedings of the NIPS Symposium on Machine Learning and the Law, Vol. 1. 2.Google Scholar
- S. Hajian and J. Domingo-Ferrer. 2013. A methodology for direct and indirect discrimination prevention in data mining. IEEE Trans. Knowl. Data Eng. 25, 7 (July 2013), 1445–1459. DOI:DOI:https://doi.org/10.1109/TKDE.2012.72Google ScholarDigital Library
- Moritz Hardt, Eric Price, Nati Srebro et al. 2016. Equality of opportunity in supervised learning. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 3315–3323.Google Scholar
- Eszter Hargittai. 2007. Whose space? Differences among users and non-users of social network sites. J. Comput.-mediat. Commun. 13, 1 (10 2007), 276–297. DOI:DOI:https://doi.org/10.1111/j.1083-6101.2007.00396.xGoogle Scholar
- Yuzi He, Keith Burghardt, and Kristina Lerman. 2020. A geometric solution to fair representations. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 279–285.Google ScholarDigital Library
- Sarah Holland, Ahmed Hosny, Sarah Newman, Joshua Joseph, and Kasia Chmielinski. 2018. The dataset nutrition label: A framework to drive higher data quality standards. arXiv preprint arXiv:1805.03677 (2018).Google Scholar
- Ayanna Howard and Jason Borenstein. 2018. The ugly truth about ourselves and our robot creations: The problem of bias and social inequity. Sci. Eng. Ethics 24, 5 (2018), 1521–1536.Google ScholarCross Ref
- Gary B. Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller. 2008. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. In Workshop on Faces in ‘Real-Life’ Images: Detection, Alignment, and Recognition, Erik Learned-Miller and Andras Ferencz and Frédéric Jurie, Marseille, France. https://hal.inria.fr/inria-00321923.Google Scholar
- Lingxiao Huang and Nisheeth Vishnoi. 2019. Stable and fair classification. In Proceedings of the International Conference on Machine Learning. 2879–2890.Google Scholar
- Ben Hutchinson and Margaret Mitchell. 2019. 50 Years of test (Un) fairness: Lessons for machine learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, 49–58.Google ScholarDigital Library
- L. Introna and H. Nissenbaum. 2000. Defining the Web: The politics of search engines. Computer 33, 1 (Jan. 2000), 54–62. DOI:DOI:https://doi.org/10.1109/2.816269Google ScholarDigital Library
- Ayush Jaiswal, Yue Wu, Wael AbdAlmageed, and Premkumar Natarajan. 2018. Unsupervised Adversarial Invariance. arxiv:cs.LG/1809.10083 (2018).Google Scholar
- Ray Jiang, Aldo Pacchiano, Tom Stepleton, Heinrich Jiang, and Silvia Chiappa. 2020. Wasserstein fair classification. In Uncertainty in Artificial Intelligence. PMLR, 862–872.Google Scholar
- F. Kamiran and T. Calders. 2009. Classifying without discriminating. In Proceedings of the 2nd International Conference on Computer, Control and Communication. 1–6. DOI:DOI:https://doi.org/10.1109/IC4.2009.4909197Google Scholar
- Faisal Kamiran and Toon Calders. 2010. Classification with no discrimination by preferential sampling. In Proceedings of the 19th Machine Learning Conference. Citeseer, 1–6.Google Scholar
- Faisal Kamiran and Toon Calders. 2012. Data preprocessing techniques for classification without discrimination. Knowl. Inf. Syst. 33, 1 (01 Oct. 2012), 1–33. DOI:DOI:https://doi.org/10.1007/s10115-011-0463-8Google Scholar
- Faisal Kamiran and Indrė Žliobaitė. 2013. Explainable and Non-explainable Discrimination in Classification. Springer Berlin, 155–170. DOI:DOI:https://doi.org/10.1007/978-3-642-30487-3_8Google Scholar
- Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. 2012. Fairness-aware classifier with prejudice remover regularizer. In Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 35–50.Google ScholarCross Ref
- Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. 2018. Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. In Proceedings of the International Conference on Machine Learning. 2569–2577.Google Scholar
- Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. 2019. An empirical study of rich subgroup fairness for machine learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, 100–109.Google ScholarDigital Library
- Rogier Kievit, Willem Eduard Frankenhuis, Lourens Waldorp, and Denny Borsboom. 2013. Simpson’s paradox in psychological science: A practical guide. Front. Psychol. 4 (2013), 513.Google ScholarCross Ref
- Niki Kilbertus, Mateo Rojas Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Schölkopf. 2017. Avoiding discrimination through causal reasoning. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 656–666.Google Scholar
- Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2016. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807 (2016).Google Scholar
- Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Proceedings of the MT Summit, Vol. 5. 79–86.Google Scholar
- Emmanouil Krasanakis, Eleftherios Spyromitros-Xioufis, Symeon Papadopoulos, and Yiannis Kompatsiaris. 2018. Adaptive sensitive reweighting to mitigate bias in fairness-aware classification. In Proceedings of the World Wide Web Conference (WWW’18). International World Wide Web Conferences Steering Committee, 853–862. DOI:DOI:https://doi.org/10.1145/3178876.3186133Google ScholarDigital Library
- Ivan Krasin, Tom Duerig, Neil Alldrin, Vittorio Ferrari, Sami Abu-El-Haija, Alina Kuznetsova, Hassan Rom, Jasper Uijlings, Stefan Popov, Andreas Veit et al. 2017. Openimages: A public dataset for large-scale multi-label and multi-class image classification. Retrieved from https://github.com/openimages.Google Scholar
- Matt J. Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual fairness. In Proceedings of the International Conference on Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). Curran Associates, Inc., 4066–4076. Retrieved from http://papers.nips.cc/paper/6995-counterfactual-fairness.pdf.Google Scholar
- Anja Lambrecht and Catherine Tucker. 2019. Algorithmic bias? an empirical study of apparent gender-based discrimi-nation in the display of stem career ads. Management Science 65, 7 (2019), 2966–2981.Google ScholarCross Ref
- J. Larson, S. Mattu, L. Kirchner, and J. Angwin. 2016. Compas analysis. Retrieved from https://github.com/propublica/compas-analysis.Google Scholar
- Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. 335–340.Google ScholarDigital Library
- Kristina Lerman. 2018. Computational social scientist beware: Simpson’s paradox in behavioral data. J. Comput. Soc. Sci. 1, 1 (2018), 49–58.Google ScholarCross Ref
- Kristina Lerman and Tad Hogg. 2014. Leveraging position bias to improve peer recommendation. PLoS One 9, 6 (2014), e98914. Retrieved from http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.009891%4.Google Scholar
- Zachary C. Lipton, Alexandra Chouldechova, and Julian McAuley. 2017. Does mitigating ML’s disparate impact require disparate treatment?stat 1050 (2017), 19.Google Scholar
- Lydia T. Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. 2018. Delayed impact of fair machine learning. In Proceedings of the 35th International Conference on Machine Learning.Google Scholar
- Joshua R. Loftus, Chris Russell, Matt J. Kusner, and Ricardo Silva. 2018. Causal reasoning for algorithmic fairness. arXiv preprint arXiv:1805.05859 (2018).Google Scholar
- Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard Zemel. 2016. The variational fair autoencoder. stat 1050 (2016), 4.Google Scholar
- Arjun K. Manrai, Birgit H. Funke, Heidi L. Rehm, Morten S. Olesen, Bradley A. Maron, Peter Szolovits, David M. Margulies, Joseph Loscalzo, and Isaac S. Kohane. 2016. Genetic misdiagnoses and the potential for health disparities. New Eng. J. Med. 375, 7 (2016), 655–665. DOI:DOI:https://doi.org/10.1056/NEJMsa1507092Google ScholarCross Ref
- Ray Marshall. 1974. The economics of racial discrimination: A survey. J. Econ. Lit. 12, 3 (1974), 849–871.Google Scholar
- Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019. On measuring social biases in sentence encoders. arXiv preprint arXiv:1903.10561 (2019).Google Scholar
- Ninareh Mehrabi, Thamme Gowda, Fred Morstatter, Nanyun Peng, and Aram Galstyan. 2019. Man is to person as woman is to location: Measuring gender bias in named entity recognition. arXiv preprint arXiv:1910.10872 (2019).Google Scholar
- Ninareh Mehrabi, Fred Morstatter, Nanyun Peng, and Aram Galstyan. 2019. Debiasing community detection: The importance of lowly-connected nodes. arXiv preprint arXiv:1903.08136 (2019).Google Scholar
- Aditya Krishna Menon and Robert C. Williamson. 2018. The cost of fairness in binary classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency (Proceedings of Machine Learning Research), Sorelle A. Friedler and Christo Wilson (Eds.), Vol. 81. PMLR, 107–118. Retrieved from http://proceedings.mlr.press/v81/menon18a.html.Google Scholar
- Michele Merler, Nalini Ratha, Rogerio S. Feris, and John R. Smith. 2019. Diversity in faces. arXiv preprint arXiv:1901.10436 (2019).Google Scholar
- Hannah Jean Miller, Jacob Thebault-Spieker, Shuo Chang, Isaac Johnson, Loren Terveen, and Brent Hecht. 2016. “Blissfully Happy” or “Ready to Fight”: Varying interpretations of emoji. In Proceedings of the 10th International AAAI Conference on Web and Social Media.Google Scholar
- I. Minchev, G. Matijevic, D. W. Hogg, G. Guiglion, M. Steinmetz, F. Anders, C. Chiappini, M. Martig, A. Queiroz, and C. Scannapieco. 2019. Yule-Simpson’s paradox in galactic archaeology. arXiv preprint arXiv:1902.01421 (2019).Google Scholar
- Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT’19). ACM, New York, NY, 220–229. DOI:DOI:https://doi.org/10.1145/3287560.3287596Google ScholarDigital Library
- Fred Morstatter, Jürgen Pfeffer, Huan Liu, and Kathleen M. Carley. 2013. Is the sample good enough? Comparing data from Twitter’s streaming API with Twitter’s Firehose. In Proceedings of the 7th International AAAI Conference on Weblogs and Social Media. AAAI Press.Google Scholar
- Daniel Moyer, Shuyang Gao, Rob Brekelmans, Aram Galstyan, and Greg Ver Steeg. 2018. Invariant representations without adversarial training. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 9084–9093.Google Scholar
- Amitabha Mukerjee, Rita Biswas, Kalyanmoy Deb, and Amrit P. Mathur. 2002. Multi–objective evolutionary algorithms for the risk–return trade–off in bank loan management. Int. Trans. Oper. Res. 9, 5 (2002), 583–597.Google ScholarCross Ref
- David B. Mustard. 2003. Reexamining criminal behavior: The importance of omitted variable bias. Rev. Econ. Statist. 85, 1 (2003), 205–211.Google ScholarCross Ref
- Razieh Nabi, Daniel Malinsky, and Ilya Shpitser. 2018. Learning optimal fair policies. arXiv preprint arXiv:1809.02244 (2018).Google Scholar
- Razieh Nabi and Ilya Shpitser. 2018. Fair inference on outcomes. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence.Google Scholar
- Azadeh Nematzadeh, Giovanni Luca Ciampaglia, Filippo Menczer, and Alessandro Flammini. 2017. How algorithmic popularity bias hinders or promotes quality. arXiv preprint arXiv:1707.00574 (2017).Google Scholar
- Dong-Phuong Nguyen, Rilana Gravel, Rudolf Berend Trieschnigg, and Theo Meder. 2013. “How old do you think I am?”: A study of language and age in Twitter. In Proceedings of the 7th International AAAI Conference on Weblogs and Social Media (ICWSM ’13). AAAI Press, 439–448.Google Scholar
- Anne O’Keeffe and Michael McCarthy. 2010. The Routledge Handbook of Corpus Linguistics. Routledge.Google Scholar
- Alexandra Olteanu, Carlos Castillo, Fernando Diaz, and Emre Kıcıman. 2019. Social data: Biases, methodological pitfalls, and ethical boundaries. Frontiers in Big Data 2 (2019), 13.Google ScholarCross Ref
- Cathy O’Neil. 2016. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Crown Publishing Group, New York, NY.Google ScholarDigital Library
- Luca Oneto, Michele Doninini, Amon Elders, and Massimiliano Pontil. 2019. Taking advantage of multitask learning for fair classification. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 227–237.Google ScholarDigital Library
- Osonde A. Osoba and William Welser IV. 2017. An Intelligence in Our Image: The Risks of Bias and Errors in Artificial Intelligence. Rand Corporation.Google Scholar
- Edmund S. Phelps. 1972. The statistical theory of racism and sexism. Amer. Econ. Rev. 62, 4 (1972), 659–661.Google Scholar
- Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q. Weinberger. 2017. On fairness and calibration. In Proceedings of the International Conference on Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). Curran Associates, Inc., 5680–5689. Retrieved from http://papers.nips.cc/paper/7151-on-fairness-and-calibration.pdf.Google Scholar
- Marcelo O. R. Prates, Pedro H. Avelar, and Luis C. Lamb. 2019. Assessing gender bias in machine translation: a case study with google translate. Neural Computing and Applications (2019), 1–19.Google Scholar
- Bilal Qureshi, Faisal Kamiran, Asim Karim, and Salvatore Ruggieri. 2016. Causal discrimination discovery through propensity score analysis. arXiv preprint arXiv:1608.03735 (2016).Google Scholar
- Inioluwa Deborah Raji and Joy Buolamwini. 2019. Actionable Auditing: Investigating the Impact of Publicly Naming Biased Performance Results of Commercial AI Products. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (AIES’19). Association for Computing Machinery, New York, NY, USA, 429–435. https://doi.org/10.1145/3306618.3314244Google ScholarDigital Library
- M. Redmond. 2011. Communities and crime unnormalized dataset. UCI Machine Learning Repository : Retrieved from http://www.ics.uci.edu/mlearn/MLRepository.html.Google Scholar
- Willy E. Rice. 1996. Race, gender, redlining, and the discriminatory access to loans, credit, and insurance: An historical and empirical analysis of consumers who sued lenders and insurers in federal and state courts, 1950–1995. San Diego L. Rev. 33 (1996), 583.Google Scholar
- Stephanie K. Riegg. 2008. Causal inference and omitted variable bias in financial aid research: Assessing solutions. Rev. Higher Educ. 31, 3 (2008), 329–354.Google ScholarCross Ref
- Lauren A. Rivera. 2012. Hiring as cultural matching: The case of elite professional service firms. Amer. Sociol. Rev. 77, 6 (2012), 999–1022.Google ScholarCross Ref
- Andrea Romei and Salvatore Ruggieri. 2014. A multidisciplinary survey on discrimination analysis. The Knowledge Engineering Review 29, 5 (2014), 582–638.Google ScholarCross Ref
- Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers). Association for Computational Linguistics, 8–14. DOI:DOI:https://doi.org/10.18653/v1/N18-2002Google ScholarCross Ref
- Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein et al. 2015. ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 3 (2015), 211–252.Google ScholarDigital Library
- Pedro Saleiro, Benedict Kuester, Abby Stevens, Ari Anisfeld, Loren Hinkson, Jesse London, and Rayid Ghani. 2018. Aequitas: A bias and fairness audit toolkit. arXiv preprint arXiv:1811.05577 (2018).Google Scholar
- Samira Samadi, Uthaipon Tantipongpipat, Jamie Morgenstern, Mohit Singh, and Santosh Vempala. 2018. The price of fair PCA: One extra dimension. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS’18). Curran Associates Inc., 10999–11010. Retrieved from http://dl.acm.org/citation.cfm?id=3327546.3327755.Google Scholar
- Nripsuta Ani Saxena. 2019. Perceptions of fairness. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES’19). ACM, New York, NY, 537–538. DOI:DOI:https://doi.org/10.1145/3306618.3314314Google ScholarDigital Library
- Nripsuta Ani Saxena, Karen Huang, Evan DeFilippis, Goran Radanovic, David C. Parkes, and Yang Liu. 2019. How do fairness definitions fare?: Examining public attitudes towards algorithmic definitions of fairness. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. ACM, 99–106.Google ScholarDigital Library
- Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations as treatments: Debiasing learning and evaluation. In Proceedings of the International Conference on Machine Learning. 1670–1679.Google Scholar
- Andrew D. Selbst, Danah Boyd, Sorelle A. Friedler, Suresh Venkatasubramanian, and Janet Vertesi. 2019. Fairness and abstraction in sociotechnical systems. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, 59–68.Google ScholarDigital Library
- Shreya Shankar, Yoni Halpern, Eric Breck, James Atwood, Jimbo Wilson, and D. Sculley. 2017. No classification without representation: Assessing geodiversity issues in open data sets for the developing world. stat 1050 (2017), 22.Google Scholar
- Richard Shaw and Manuel Corpas. [n.d.]. Further bias in personal genomics? ([n. d.]).Google Scholar
- Harini Suresh and John V. Guttag. 2019. A framework for understanding unintended consequences of machine learning. arXiv preprint arXiv:1901.10002 (2019).Google Scholar
- Songül Tolan, Marius Miron, Emilia Gómez, and Carlos Castillo. 2019. Why machine learning may lead to unfairness: evidence from risk assessment for juvenile justice in catalonia. In Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law (ICAIL’19). Association for Computing Machinery, New York, NY, USA, 83–92. https://doi.org/10.1145/3322640.3326705Google ScholarDigital Library
- Zeynep Tufekci. 2014. Big questions for social media big data: Representativeness, validity and other methodological pitfalls. In Proceedings of the 8th International AAAI Conference on Weblogs and Social Media.Google Scholar
- Berk Ustun, Yang Liu, and David Parkes. 2019. Fairness without harm: Decoupled classifiers with preference guarantees. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.), Vol. 97. PMLR, 6373–6382. Retrieved from http://proceedings.mlr.press/v97/ustun19a.html.Google Scholar
- Eva Vanmassenhove, Christian Hardmeier, and Andy Way. 2018. Getting gender right in neural machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 3003–3008.Google ScholarCross Ref
- Sahil Verma and Julia Rubin. 2018. Fairness definitions explained. In Proceedings of the IEEE/ACM International Workshop on Software Fairness (FairWare’18). IEEE, 1–7.Google ScholarDigital Library
- Selwyn Vickers, Mona Fouad, and Moon S. Chen Jr. 2014. Enhancing minority participation in clinical trials (EMPaCT): Laying the groundwork for improving minority clinical trial accrual. Cancer 120 (2014), vi–vii.Google Scholar
- Ting Wang and Dashun Wang. 2014. Why Amazon’s ratings might mislead you: The story of herding effects. Big Data 2, 4 (2014), 196–204.Google ScholarCross Ref
- Steven L. Willborn. 1984. The disparate impact model of discrimination: Theory and limits. Amer. UL Rev. 34 (1984), 799.Google Scholar
- Christo Wilson, Bryce Boe, Alessandra Sala, Krishna P. N. Puttaswamy, and Ben Y. Zhao. 2009. User interactions in social networks and their implications. In Proceedings of the 4th ACM European Conference on Computer Systems. ACM, 205–218.Google Scholar
- Blake Woodworth, Suriya Gunasekar, Mesrob I. Ohannessian, and Nathan Srebro. 2017. Learning non-discriminatory predictors. arXiv preprint arXiv:1702.06081 (2017).Google Scholar
- Yongkai Wu, Lu Zhang, and Xintao Wu. 2018. Fairness-aware Classification: Criterion, Convexity, and Bounds. arxiv:cs.LG/1809.04737 (2018).Google Scholar
- Depeng Xu, Shuhan Yuan, Lu Zhang, and Xintao Wu. 2018. FairGAN: Fairness-aware generative adversarial networks. In Proceedings of the IEEE International Conference on Big Data (Big Data’18). IEEE, 570–575.Google ScholarCross Ref
- Irene Y. Chen, Peter Szolovits, and Marzyeh Ghassemi. 2019. Can AI help reduce disparities in general medical and mental health care?AMA J. Ethics 21 (02 2019), E167–179. DOI:DOI:https://doi.org/10.1001/amajethics.2019.167Google Scholar
- Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi. 2017. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the 26th International Conference on World Wide Web. 1171–1180.Google Scholar
- Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi. 2015. Fairness constraints: Mechanisms for fair classification. arXiv preprint arXiv:1507.05259 (2015).Google Scholar
- Lu Zhang and Xintao Wu. 2017. Anti-discrimination learning: A causal modeling-based framework. Int. J. Data Sci. Analyt. 4, 1 (01 Aug. 2017), 1–16. DOI:DOI:https://doi.org/10.1007/s41060-017-0058-xGoogle ScholarCross Ref
- Lu Zhang, Yongkai Wu, and Xintao Wu. 2016. On discrimination discovery using causal networks. In Social, Cultural, and Behavioral Modeling, Kevin S. Xu, David Reitter, Dongwon Lee, and Nathaniel Osgood (Eds.). Springer International Publishing, Cham, 83–93.Google Scholar
- Lu Zhang, Yongkai Wu, and Xintao Wu. 2016. Situation testing-based discrimination discovery: A causal inference approach. In Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI’16). AAAI Press, 2718–2724. Retrieved from http://dl.acm.org/citation.cfm?id=3060832.3061001.Google Scholar
- Lu Zhang, Yongkai Wu, and Xintao Wu. 2017. Achieving non-discrimination in data release. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1335–1344.Google ScholarDigital Library
- Lu Zhang, Yongkai Wu, and Xintao Wu. 2017. A causal framework for discovering and removing direct and indirect discrimination. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI’17). 3929–3935. DOI:DOI:https://doi.org/10.24963/ijcai.2017/549Google ScholarCross Ref
- L. Zhang, Y. Wu, and X. Wu. 2018. Causal modeling-based discrimination discovery and removal: Criteria, bounds, and algorithms. IEEE Trans. Knowl. Data Eng. (2018), 1–1. DOI:DOI:https://doi.org/10.1109/TKDE.2018.2872988Google Scholar
- Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender bias in contextualized word embeddings. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 629–634.Google ScholarCross Ref
- Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proceedings of the Conference on Empirical Methods in Natural Language Processing.Google ScholarCross Ref
- Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. arxiv:cs.CL/1804.06876 (2018).Google Scholar
- Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018. Learning gender-neutral word embeddings. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 4847–4853.Google ScholarCross Ref
- James Zou and Londa Schiebinger. 2018. AI can be sexist and racist it’s time to make it fair. Nature Publishing Group.Google Scholar
Index Terms
- A Survey on Bias and Fairness in Machine Learning
Recommendations
Deep learning, reinforcement learning, and world models
AbstractDeep learning (DL) and reinforcement learning (RL) methods seem to be a part of indispensable factors to achieve human-level or super-human AI systems. On the other hand, both DL and RL have strong connections with our brain functions ...
Artificial learning companionusing machine learning and natural language processing
AbstractArtificial Intelligence, also referred to as AI, is one of the most rapidly evolving branches of Computer Science. The two branches of AI which empowers it to understand and interact with humans are Machine Learning (ML) and Natural Language ...
Representation Learning: A Review and New Perspectives
The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. ...
Comments