Share This Author
All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Recommender Evaluation and Effectiveness
System evaluation protocols that explicitly quantify the degree to which the system is meeting the information needs of all its users are proposed, as well as the need for researchers and operators to move beyond evaluations that favor the needs of larger subsets of the user population while ignoring smaller subsets.
Exploring author gender in book rating and recommendation
- Michael D. Ekstrand, Mucun Tian, Mohammed R. Imran Kazi, Hoda Mehrpouyan, Daniel Kluver
- Computer ScienceRecSys
- 22 August 2018
This work measures the distribution of the genders of the authors of books in user rating profiles and recommendation lists produced from this data to find that common collaborative filtering algorithms differ in the gender distribution of their recommendation lists, and in the relationship of that output distribution to user profile distribution.
Estimating Error and Bias in Offline Evaluation Results
It is found that missing data in the rating or observation process causes the evaluation protocol to systematically mis-estimate metric values, and in some cases erroneously determine that a popularity-based recommender outperforms even a perfect personalized recommender.
Counterfactual Learning to Rank using Heterogeneous Treatment Effect Estimation
This work employs heterogeneous treatment effect estimation techniques to estimate position bias when intervention click data is limited and uses such estimations to debias the observed click distribution and re-draw a new de-biased data set, which can be used for any LTR algorithms.
Monte Carlo Estimates of Evaluation Metric Error and Bias
Simulation of the recommender data generation and evaluation processes is used to quantify the extent of evaluation metric errors and assess their sensitivity to various assumptions.
Scripts for All The Cool Kids, How Do They Fit In?
Estimating Error and Bias of Offline Recommender System Evaluation Results
- Mucun Tian
- Computer Science
Script for Estimating Error and Bias in Offline Evaluation Results
This publication contains scripts to reproduce the paper “Estimating Error and Bias in Offline Evaluation Results” by Muncun Tian and Michael D. Ekstrand in Proceedings of the 2020 Conference on …
Dynamic Prediction of Building HVAC Energy Consumption by Ensemble Learning Approach
- Weisong Tian, Chengwei Lei, Mucun Tian
- EngineeringInternational Conference on Computational Science…
- 1 December 2018
This work adopts ensemble learning approaches that consider the load model as a black box and use historical weather data as input and power consumption data as output, to train the HVAC power load model of a building.