ABSTRACT
Algorithms and decision making based on Big Data have become pervasive in all aspects of our daily lives lives (offline and online), as they have become essential tools in personal finance, health care, hiring, housing, education, and policies. It is therefore of societal and ethical importance to ask whether these algorithms can be discriminative on grounds such as gender, ethnicity, or health status. It turns out that the answer is positive: for instance, recent studies in the context of online advertising show that ads for high-income jobs are presented to men much more often than to women [Datta et al., 2015]; and ads for arrest records are significantly more likely to show up on searches for distinctively black names [Sweeney, 2013]. This algorithmic bias exists even when there is no discrimination intention in the developer of the algorithm. Sometimes it may be inherent to the data sources used (software making decisions based on data can reflect, or even amplify, the results of historical discrimination), but even when the sensitive attributes have been suppressed from the input, a well trained machine learning algorithm may still discriminate on the basis of such sensitive attributes because of correlations existing in the data. These considerations call for the development of data mining systems which are discrimination-conscious by-design. This is a novel and challenging research area for the data mining community.
The aim of this tutorial is to survey algorithmic bias, presenting its most common variants, with an emphasis on the algorithmic techniques and key ideas developed to derive efficient solutions. The tutorial covers two main complementary approaches: algorithms for discrimination discovery and discrimination prevention by means of fairness-aware data mining. We conclude by summarizing promising paths for future research.
Supplemental Material
- S. Barocas and A. D. Selbst. Big data's disparate impact. SSRN Pre-Print 2477899, 2014.Google Scholar
- F. Bonchi, S. Hajian, B. Mishra, and D. Ramazzotti. Exposing the probabilistic causal structure of discrimination. arXiv:1510.00552, 2015.Google Scholar
- T. Calders and S. Verwer. Three naive bayes approaches for discrimination-free classi cation. DMKD, 21(2):277--292, 2010. Google ScholarDigital Library
- B. Custers, T. Calders, B. Schermer, and T. Zarsky, editors. Discrimination and Privacy in the Information Society. Springer, 2013. Google ScholarDigital Library
- A. Datta, M. C. Tschantz, and A. Datta. Automated experiments on ad privacy settings. Proc. Privacy Enhancing Technologies, 2015(1):92--112, 2015.Google ScholarCross Ref
- C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel. Fairness through awareness. In ITCS, 2012. Google ScholarDigital Library
- M. Feldman, S. A. Friedler, J. Moeller, C. Scheidegger, and S. Venkatasubramanian. Certifying and removing disparate impact. In KDD, 2015. Google ScholarDigital Library
- S. Hajian and J. Domingo-Ferrer. A methodology for direct and indirect discrimination prevention in data mining. TKDE, 25(7):1445--1459, 2013. Google ScholarDigital Library
- S. Hajian, J. Domingo-Ferrer, and O. Farras. Generalization-based privacy preservation and discrimination prevention in data publishing and mining. DMKD, 28(5-6):1158--1188, 2014. Google ScholarDigital Library
- S. Hajian, J. Domingo-Ferrer, A. Monreale, D. Pedreschi, and F. Giannotti. Discrimination- and privacy-aware patterns. DMKD, 29(6):1733--1782, 2015. Google ScholarDigital Library
- F. Kamiran and T. Calders. Data preprocessing techniques for classi cation without discrimination. KAIS, 33(1):1--33, 2012.Google ScholarDigital Library
- B. T. Luong, S. Ruggieri, and F. Turini. k-nn as an implementation of situation testing for discrimination discovery and prevention. In KDD, 2011. Google ScholarDigital Library
- D. Pedreshi, S. Ruggieri, and F. Turini. Discrimination-aware data mining. In KDD, 2008. Google ScholarDigital Library
- S. Ruggieri, S. Hajian, F. Kamiran, and X. Zhang. Anti-discrimination analysis using privacy attack strategies. In ECML-PKDD, 2014.Google ScholarDigital Library
- N. Savage. When computers stand in the schoolhouse door. Commun. ACM, 59(3):19--21, Feb. 2016. Google ScholarDigital Library
- L. Sweeney. Discrimination in online ad delivery. Queue, 11(3):10, 2013. Google ScholarDigital Library
Index Terms
- Algorithmic Bias: From Discrimination Discovery to Fairness-aware Data Mining
Recommendations
Combating discrimination using Bayesian networks
Discrimination in decision making is prohibited on many attributes (religion, gender, etc ), but often present in historical decisions. Use of such discriminatory historical decision making as training data can perpetuate discrimination, even if the ...
Beyond Algorithmic Bias: A Socio-Computational Interrogation of the Google Search by Image Algorithm
We perform a socio-computational interrogation of the google search by image algorithm, a main component of the google search engine. We audit the algorithm by presenting it with more than 40 thousands faces of all ages and more than four races and ...
(Some) algorithmic bias as institutional bias
AbstractIn this paper I argue that some examples of what we label ‘algorithmic bias’ would be better understood as cases of institutional bias. Even when individual algorithms appear unobjectionable, they may produce biased outcomes given the way that ...
Comments