When recommenders fail: predicting recommender failure for algorithm selection and combination

@inproceedings{Ekstrand2012WhenRF,
  title={When recommenders fail: predicting recommender failure for algorithm selection and combination},
  author={Michael D. Ekstrand and John Riedl},
  booktitle={RecSys '12},
  year={2012}
}
Hybrid recommender systems --- systems using multiple algorithms together to improve recommendation quality --- have been well-known for many years and have shown good performance in recent demonstrations such as the NetFlix Prize. Modern hybridization techniques, such as feature-weighted linear stacking, take advantage of the hypothesis that the relative performance of recommenders varies by circumstance and attempt to optimize each item score to maximize the strengths of the component… 

Figures and Tables from this paper

Assessing and improving recommender systems to deal with user cold-start problem
TLDR
This thesis proposes 4 approaches to deal with user cold-start problem using existing models available for analysis in the recommender systems and evaluated the proposed approaches in terms of prediction quality and ranking quality in real-world datasets under different recommendation domains.
Letting Users Choose Recommender Algorithms: An Experimental Study
TLDR
This study gives users the ability to change the algorithm providing their movie recommendations and studied how they make use of this power, and examines log data from user interactions with this new feature to under-stand whether and how users switch among recommender algorithms, and select a final algorithm to use.
Rating-Based Collaborative Filtering: Algorithms and Evaluation
TLDR
The concepts, algorithms, and means of evaluation that are at the core of collaborative filtering research and practice are reviewed, and two more recent directions in recommendation algorithms are presented: learning-to-rank and ensemble recommendation algorithms.
On hybrid modular recommendation systems for video streaming
TLDR
The enabler, a hybrid recommendation system which employs various machine-learning algorithms for learning an efficient combination of several recommendation algorithms and selects the best blending for a given input, is proposed.
Recommender System: Personalizing User Experience or Scientifically Deceiving Users?
TLDR
An overview of the recommender system is given, how various components of theRecommender system may be manipulated to allure innocent customers with false ratings are discussed, and the importance of engaging stakeholders to develop a robust recommendersystem is discussed.
Towards Recommender Engineering: tools and experiments for identifying recommender differences
TLDR
The LensKit toolkit for conducting experiments on a wide variety of recommender algorithms and data sets under different experimental conditions, along with new developments in object-oriented software configuration to support this toolkit, and experiments on the configuration options of widely-used algorithms to provide guidance on tuning and configuring them are made.
MetaSelector: Meta-Learning for Recommendation with User-Level Adaptive Model Selection
TLDR
This work proposes a meta-learning framework to facilitate user-level adaptive model selection in recommender systems and achieves improvements over single model baselines and sample-level model selector in terms of AUC and LogLoss.
MetaSelector: Meta-Learning for Recommendation with User-Level Adaptive Model Selection
TLDR
This work proposes a meta-learning framework to facilitate user-level adaptive model selection in recommender systems and achieves improvements over single model baselines and sample-level model selector in terms of AUC and LogLoss.
Online optimization for user-specific hybrid recommender systems
TLDR
A self-learning user-specific hybrid recommender system is described and its ability towards meeting a set of pre-defined requirements relevant to online recommendation scenarios: responsiveness, scalability, system transparency and user control is assessed.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 15 REFERENCES
A Survey of Accuracy Evaluation Metrics of Recommendation Tasks
TLDR
This paper reviews the proper construction of offline experiments for deciding on the most appropriate algorithm, and discusses three important tasks of recommender systems, and classify a set of appropriate well known evaluation metrics for each task.
Hybrid Recommender Systems: Survey and Experiments
  • R. Burke
  • Computer Science
    User Modeling and User-Adapted Interaction
  • 2004
TLDR
This paper surveys the landscape of actual and possible hybrid recommenders, and introduces a novel hybrid, EntreeC, a system that combines knowledge-based recommendation and collaborative filtering to recommend restaurants, and shows that semantic ratings obtained from the knowledge- based part of the system enhance the effectiveness of collaborative filtering.
Improving rating estimation in recommender systems using aggregation- and variance-based hierarchical models
TLDR
It is experimentally shown that the optimal linear combination approach significantly dominates all other special cases, including the classical non-aggregated case and the previously studied aggregate methods, and therefore is the method of choice.
Item-based collaborative filtering recommendation algorithms
TLDR
This paper analyzes item-based collaborative ltering techniques and suggests that item- based algorithms provide dramatically better performance than user-based algorithms, while at the same time providing better quality than the best available userbased algorithms.
Improving regularized singular value decomposition for collaborative filtering
TLDR
Different efficient collaborative filtering techniques and a framework for combining them to obtain a good prediction are described, predicting users’ preferences for movies with error rate 7.04% better on the Netflix Prize dataset than the reference algorithm Netflix Cinematch.
Making recommendations better: an analytic model for human-recommender interaction
TLDR
It is argued that recommenders need a deeper understanding of users and their information seeking tasks and recommender algorithms using a common language and an analytic process model.
An Empirical Analysis of Design Choices in Neighborhood-Based Collaborative Filtering Algorithms
TLDR
An analysis framework is applied that divides the neighborhood-based prediction approach into three components and then examines variants of the key parameters in each component, and identifies the three components identified are similarity computation, neighbor selection, and rating combination.
Rethinking the recommender research ecosystem: reproducibility, openness, and LensKit
TLDR
The utility of LensKit is demonstrated by replicating and extending a set of prior comparative studies of recommender algorithms, and a question recently raised by a leader in the recommender systems community on problems with error-based prediction evaluation is investigated.
Automatically building research reading lists
TLDR
This work explores several methods for augmenting existing collaborative and content-based filtering algorithms with measures of the influence of a paper within the web of citations, including a novel method for using importance scores to influence collaborative filtering.
Enhancing digital libraries with TechLens
TLDR
This paper presents and experiments with hybrid recommender algorithms that combine collaborative filtering and content-based filtering to recommend research papers to users and shows that users value paper recommendations, that the hybrid algorithms can be successfully combined, and that these results can be applied to develop recommender systems for other types of digital libraries.
...
1
2
...