ABSTRACT
Rankings of people and items are at the heart of selection-making, match-making, and recommender systems, ranging from employment sites to sharing economy platforms. As ranking positions influence the amount of attention the ranked subjects receive, biases in rankings can lead to unfair distribution of opportunities and resources such as jobs or income. This paper proposes new measures and mechanisms to quantify and mitigate unfairness from a bias inherent to all rankings, namely, the position bias which leads to disproportionately less attention being paid to low-ranked subjects. Our approach differs from recent fair ranking approaches in two important ways. First, existing works measure unfairness at the level of subject groups while our measures capture unfairness at the level of individual subjects, and as such subsume group unfairness. Second, as no single ranking can achieve individual attention fairness, we propose a novel mechanism that achieves amortized fairness, where attention accumulated across a series of rankings is proportional to accumulated relevance. We formulate the challenge of achieving amortized individual fairness subject to constraints on ranking quality as an online optimization problem and show that it can be solved as an integer linear program. Our experimental evaluation reveals that unfair attention distribution in rankings can be substantial, and demonstrates that our method can improve individual fairness while retaining high ranking quality.
- Rediet Abebe, Jon Kleinberg, and David C Parkes . 2017. Fair division via social comparison. In AAMAS. Google ScholarDigital Library
- Philip Adler, Casey Falk, Sorelle Friedler, Gabriel Rybeck, Carlos Scheidegger, Brandon Smith, and Suresh Venkatasubramanian . 2016. Auditing Black-Box Models for Indirect Influence. In ICDM.Google Scholar
- Mor Armony and Amy R. Ward . 2010. Fair Dynamic Routing in Large-Scale Heterogeneous-Server Systems. Operations Research (2010). Google ScholarDigital Library
- Asia J Biega, Rishiraj Saha Roy, and Gerhard Weikum . 2017. Privacy through Solidarity: A User-Utility-Preserving Framework to Counter Profiling. In SIGIR. Google ScholarDigital Library
- Ryan Calo and Alex Rosenblat . 2017. The taking economy: Uber, information, and power. Columbia Law Review (2017).Google Scholar
- L Elisa Celis, Damian Straszak, and Nisheeth K Vishnoi . 2017. Ranking with Fairness Constraints. arXiv preprint arXiv:1704.06840.Google Scholar
- Abhijnan Chakraborty, Asia J. Biega, Aniko Hannak, and Krishna P. Gummadi . 2017. Fair Sharing for Sharing Economy Platforms. In FATREC@RecSys Workshop.Google Scholar
- Aleksandr Chuklin, Ilya Markov, and Maarten de Rijke . 2015. Click Models for Web Search. In Morgan & Claypool.Google Scholar
- FAT Conference . {n. d.}. Conference on Fairness, Accountability, and Transparency (FAT*) small http://fatconference.org/resources.html.Google Scholar
- Nick Craswell, Onno Zoeter, Michael Taylor, and Bill Ramsey . 2008. An experimental comparison of click position-bias models WSDM. Google ScholarDigital Library
- Georges Dupret and Benjamin Piwowarski . 2008. A user browsing model to predict search engine click data from past observations SIGIR. Google ScholarDigital Library
- Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel . 2012. Fairness through awareness. In ITCS. Google ScholarDigital Library
- Benjamin Edelman, Michael Ostrovsky, and Michael Schwarz . 2007. Internet advertising and the generalized second-price auction: Selling billions of dollars worth of keywords. American economic review (2007).Google Scholar
- Michael Feldman, Sorelle Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian . 2015. Certifying and Removing Disparate Impact. In KDD. Google ScholarDigital Library
- Ali Ghodsi, Vyas Sekar, Matei Zaharia, and Ion Stoica . 2012. Multi-resource fair queueing for packet processing SIGCOMM. Google ScholarDigital Library
- Ali Ghodsi, Matei Zaharia, Benjamin Hindman, Andy Konwinski, Scott Shenker, and Ion Stoica . 2011. Dominant resource fairness: Fair allocation of multiple resource types NSDI. Google ScholarDigital Library
- Jerald Greenberg . 1987. A taxonomy of organizational justice theories. Academy of Management review (1987).Google Scholar
- Nina Grgic-Hlaca, Muhammad Bilal Zafar, Krishna P Gummadi, and Adrian Weller . 2018. Beyond Distributive Fairness in Algorithmic Decision Making: Feature Selection for Procedurally Fair Learning. In AAAI.Google Scholar
- Fan Guo, Chao Liu, and Yi Min Wang . 2009. Efficient multiple-click models in web search. In WSDM. Google ScholarDigital Library
- Moritz Hardt, Eric Price, and Nati Srebro . 2016. Equality of Opportunity in Supervised Learning. In NIPS. Google ScholarDigital Library
- Thorsten Joachims and Filip Radlinski . 2007. Search Engines that Learn from Implicit Feedback. IEEE Computer (2007). Google ScholarDigital Library
- Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel . 2017. Unbiased Learning-to-Rank with Biased Feedback. In WSDM. Google ScholarDigital Library
- Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma . 2012. Fairness-Aware Classifier with Prejudice Remover Regularizer ECML/PKDD.Google Scholar
- Michael Kearns, Aaron Roth, and Zhiwei Steven Wu . 2017. Meritocratic Fairness for Cross-Population Selection ICML.Google Scholar
- Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan . 2017. Human decisions and machine predictions. The Quarterly Journal of Economics (2017).Google ScholarCross Ref
- Karen Levy and Solon Barocas . 2018. Designing Against Discrimination in Online Markets. Berkeley Technology Law Journal (2018).Google Scholar
- Rishabh Mehrotra, Ashton Anderson, Fernando Diaz, Amit Sharma, Hanna Wallach, and Emine Yilmaz . 2017. Auditing Search Engines for Differential Satisfaction Across Demographics WWW. Google ScholarDigital Library
- Dino Pedreschi, Salvatore Ruggieri, and Franco Turini . 2008. Discrimination-aware data mining. In KDD. Google ScholarDigital Library
- Andrea Romei and Salvatore Ruggieri . 2014. A multidisciplinary survey on discrimination analysis. Knowledge Eng. Review (2014).Google Scholar
- Ashudeep Singh and Thorsten Joachims . 2018. Fairness of Exposure in Rankings. arXiv preprint arXiv:1802.07281.Google Scholar
- Elaine Walster, Ellen Berscheid, and G William Walster . 1973. New directions in equity research. Journal of personality and social psychology (1973).Google Scholar
- Xuanhui Wang, Michael Bendersky, Donald Metzler, and Marc Najork . 2016. Learning to Rank with Selection Bias in Personal Search SIGIR. Google ScholarDigital Library
- Menahem E Yaari and Maya Bar-Hillel . 1984. On dividing justly. Social choice and welfare (1984).Google Scholar
- Ke Yang and Julia Stoyanovich . 2007. Measuring fairness in ranked outputs. In SSDBM. Google ScholarDigital Library
- Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez-Rodriguez, and Krishna P. Gummadi . 2017. Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment. In WWW. Google ScholarDigital Library
- Meike Zehlike, Francesco Bonchi, Carlos Castillo, Sara Hajian, Mohamed Megahed, and Ricardo Baeza-Yates . 2017. FA*IR: A fair top-k ranking algorithm. In CIKM. Google ScholarDigital Library
- Richard S. Zemel, Yu Wu, Kevin Swersky, Toniann Pitassi, and Cynthia Dwork . 2013. Learning Fair Representations. In ICML. Google ScholarDigital Library
Index Terms
- Equity of Attention: Amortizing Individual Fairness in Rankings
Recommendations
Marginal-Certainty-Aware Fair Ranking Algorithm
WSDM '23: Proceedings of the Sixteenth ACM International Conference on Web Search and Data MiningRanking systems are ubiquitous in modern Internet services, including online marketplaces, social media, and search engines. Traditionally, ranking systems only focus on how to get better relevance estimation. When relevance estimation is available, they ...
FARA: Future-aware Ranking Algorithm for Fairness Optimization
CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge ManagementRanking systems are the key components of modern Information Retrieval (IR) applications, such as search engines and recommender systems. Besides the ranking relevance to users, the exposure fairness to item providers has also been considered an ...
What's Fair about Individual Fairness?
AIES '21: Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and SocietyOne of the main lines of research in algorithmic fairness involves individual fairness (IF) methods. Individual fairness is motivated by an intuitive principle, similar treatment, which requires that similar individuals be treated similarly. IF offers a ...
Comments