• Corpus ID: 232352317

Fairness in Ranking: A Survey

@article{Zehlike2021FairnessIR,
  title={Fairness in Ranking: A Survey},
  author={Meike Zehlike and Ke Yang and Julia Stoyanovich},
  journal={ArXiv},
  year={2021},
  volume={abs/2103.14000}
}
In the past few years, there has been much work on incorporating fairness requirements into algorithmic rankers, with contributions coming from the data management, algorithms, information retrieval, and recommender systems communities. In this survey we give a systematic overview of this work, offering a broad perspective that connects formalizations and algorithmic approaches across subfields. An important contribution of our work is in developing a common narrative around the value… 
Fairness in Rankings and Recommendations: An Overview
TLDR
This work aims at presenting a toolkit of definitions, models and methods used for ensuring fairness in rankings and recommendations, and to highlight open challenges and research paths for future work.
Comparing Fair Ranking Metrics
TLDR
This work provides a direct comparative analysis identifying similarities and differences of fair ranking metrics selected for the work, and empirically compare them on the same experimental setup and data set.
Causal Intersectionality and Fair Ranking
TLDR
A causal modeling approach to intersectional fairness, and a flexible, taskspecific method for computing intersectionally fair rankings, that is connected to important real world effects and domain knowledge, and transparent about technical limitations are proposed.
Calibrating Explore-Exploit Trade-off for Fair Online Learning to Rank
TLDR
This work states that different groups of items might receive differential treatments during the course of OL2R, and existing fair ranking solutions usually require the knowledge of result relevance or a performing ranker beforehand, which contradicts with the setting ofOL2R and thus cannot be directly applied to guarantee fairness.
Understanding and Mitigating the Effect of Outliers in Fair Ranking
TLDR
This work formalizes outlierness in a ranking, shows that outliers are present in realistic datasets, and presents the results of an eye-tracking study, showing that users scanning order and the exposure of items are influenced by the presence of outliers.
Evaluating Fairness in Argument Retrieval
TLDR
This work analyzes a range of non-stochastic fairness-aware ranking and diversity metrics to evaluate the extent to which argument stances are fairly exposed in argument retrieval systems and shows that systems with the best effectiveness are not necessarily the most fair or the most diverse in terms of argument stance.
Fairness Through Counterfactual Utilities
TLDR
This work derives two fairness principles that enable a generalized set of group fairness definitions that unambiguously extend to all machine learning environments while still retaining their original fairness notions and provides concrete examples of how this framework resolves known fairness issues in classification, clustering, and reinforcement learning problems.
Inequality and inequity in network-based ranking and recommendation algorithms
TLDR
This work proposes a directed network model with preferential attachment and homophily (DPAH) and demonstrates the influence of network structure on the rank distributions of PageRank and Who-to-Follow and sheds light on the social and algorithmic mechanisms that hinder equality and equity in network-based ranking and recommendation algorithms.
Identifying Biased Subgroups in Ranking and Classification
TLDR
The notion of divergence is introduced to measure this performance difference and it is exploited in the context of classification models and ranking applications to automatically detect data subgroups showing a significant deviation in their behavior.
Fair Decision-Making for Food Inspections
TLDR
This work revisits the application of predictive models by the Chicago Department of Public Health to schedule restaurant inspections and prioritize the detection of critical violations of the food code and finds that the model treats inspections unequally based on the sanitarian who conducted the inspection.
...
...

References

SHOWING 1-10 OF 95 REFERENCES
Fairness in Recommendation Ranking through Pairwise Comparisons
TLDR
This paper offers a set of novel metrics for evaluating algorithmic fairness concerns in recommender systems and shows how measuring fairness based on pairwise comparisons from randomized experiments provides a tractable means to reason about fairness in rankings fromRecommender systems.
Fairness and discrimination in recommendation and retrieval
TLDR
This tutorial will help RecSys researchers to algorithmic fairness, understand how concepts do and do not translate from other settings, and provide an introduction to the growing literature on this topic.
Fairness of Exposure in Rankings
TLDR
This work proposes a conceptual and computational framework that allows the formulation of fairness constraints on rankings in terms of exposure allocation, and develops efficient algorithms for finding rankings that maximize the utility for the user while provably satisfying a specifiable notion of fairness.
Multisided Fairness for Recommendation
TLDR
It is shown that in some recommendation contexts, fairness may be a multisided concept, in which fair outcomes for multiple individuals need to be considered, and a taxonomy of classes of fairness-aware recommender systems is presented.
Fairness and Discrimination in Retrieval and Recommendation
TLDR
This tutorial will help to orient IR researchers to algorithmic fairness, understand how concepts do and do not translate from other settings, and provide an introduction to the growing literature on this topic.
Ranking with Fairness Constraints
TLDR
This work studies the following variant of the traditional ranking problem when the objective satisfies properties that appear in common ranking metrics such as Discounted Cumulative Gain, Spearman's rho or Bradley-Terry.
Policy Learning for Fairness in Ranking
TLDR
This work proposes a general LTR framework that can optimize a wide range of utility metrics while satisfying fairness of exposure constraints with respect to the items, and provides a new LTR algorithm called Fair-PG-Rank for directly searching the space of fair ranking policies via a policy-gradient approach.
Equity of Attention: Amortizing Individual Fairness in Rankings
TLDR
The challenge of achieving amortized individual fairness subject to constraints on ranking quality as an online optimization problem is formulated and solved as an integer linear program and it is demonstrated that the method can improve individual fairness while retaining high ranking quality.
Online Set Selection with Fairness and Diversity Constraints
TLDR
This paper develops algorithms for several problem settings with streaming data, where an online decision must be made on each item as it is presented, and demonstrates that fairness and diversity can be achieved, usually with modest costs in terms of quality.
FairSearch: A Tool For Fairness in Ranked Search Results
TLDR
FairSearch is presented, the first fair open source search API to provide fairness notions in ranked search results and implement two well-known algorithms from the literature, namely FA*IR and DELTR and provide them as stand-alone libraries in Python and Java.
...
...