• Corpus ID: 239016061

Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness

  title={Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness},
  author={Nicola Neophytou and Bhaskar Mitra and Catherine Stinson},
Recommendation algorithms are susceptible to popularity bias: a tendency to recommend popular items even when they fail to meet user needs. A related issue is that the recommendation quality can vary by demographic groups. Marginalized groups or groups that are under-represented in the training data may receive less relevant recommendations from these algorithms compared to others. In a recent study, Ekstrand et al. [15] investigate how recommender performance varies according to popularity and… 

Figures and Tables from this paper


The Connection Between Popularity Bias, Calibration, and Fairness in Recommendation
There is a connection between how different user groups are affected by algorithmic popularity bias and their level of interest in popular items, and a metric called miscalibration is used for measuring how a recommendation algorithm is responsive to users’ true preferences.
Multi-sided Exposure Bias in Recommendation
This paper empirically shows the inherent popularity bias of the algorithms and how this bias impacts different stakeholders such as users and suppliers of the items, and proposes metrics to measure the exposure bias of recommendation algorithms from the perspective of different stakeholders.
All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Recommender Evaluation and Effectiveness
System evaluation protocols that explicitly quantify the degree to which the system is meeting the information needs of all its users are proposed, as well as the need for researchers and operators to move beyond evaluations that favor the needs of larger subsets of the user population while ignoring smaller subsets.
Exploring author gender in book rating and recommendation
This work measures the distribution of the genders of the authors of books in user rating profiles and recommendation lists produced from this data to find that common collaborative filtering algorithms differ in the gender distribution of their recommendation lists, and in the relationship of that output distribution to user profile distribution.
When Users with Preferences Different from Others Get Inaccurate Recommendations
The proposed measures allow to identify reliably a subset of users with atypical preferences, who will actually get inaccurate recommendations with a social approach, one of which is the most accurate, whatever is the recommendation technique.
A methodology for learning, analyzing, and mitigating social influence bias in recommender systems
This work proposes a methodology to learn, learn, analyze, and mitigate the effect of Social Inuence Bias in recommender systems, and found statistically significant evidence of SIB in the California Report Card.
Collaborative Filtering for Implicit Feedback Datasets
This work identifies unique properties of implicit feedback datasets and proposes treating the data as indication of positive and negative preference associated with vastly varying confidence levels, which leads to a factor model which is especially tailored for implicit feedback recommenders.
A Study of Position Bias in Digital Library Recommender Systems
A study in a real-world recommender system that delivered ten million related-article recommendations to the users of the digital library Sowiport, and the reference manager JabRef confirms the findings from other domains, that recommendations in the top positions are more often clicked, regardless of their actual relevance.
"Fulfilling the Needs of Gray-Sheep Users in Recommender Systems, A Clustering Solution"
This paper focuses on graysheep users problem responsible for the increased error rate in collaborative filtering based recommender systems algorithms, and shows that the presence of gray-sheep users can affect the performance— accuracy and coverage— of collaborative filteringbased algorithms, depending on the data sparsity and distribution.
Music Recommendation and Discovery in the Long Tail
The Long Tail curve of artist popularity is model to predict —potentially— interesting and unknown music, hidden in the tail of the popularity curve, which has significant implications for recommender systems that assist users to explore the Long Tail, digging for content they might like.