Evaluating the Accuracy and Utility of Recommender Systems

@inproceedings{Said2013EvaluatingTA,
  title={Evaluating the Accuracy and Utility of Recommender Systems},
  author={A. Said},
  year={2013}
}
  • A. Said
  • Published 2013
  • Computer Science
Recommender systems have become a ubiquitous feature on the World Wide Web. Today, most websites use some form of recommendation to heighten their users’ experience. Over the last decade, vast advancements in recommendation have been done, this has however not been matched in the processes involved in evaluating these systems. The evaluation methods and metrics currently used for this have originated in other related fields, e.g. information retrieval, statistics, etc. For most cases, these… Expand
Improving Recommender Systems Precision with Multiple Metadata using Ensemble Methods
Recommender systems have become increasingly popular and widely adopted by many sites and services. They are important tools in assisting users to filter what is relevant for them in this complexExpand
Towards reproducibility in recommender-systems research
TLDR
The recommender-system community needs to survey other research fields and learn from them, find a common understanding of reproducibility, identify and understand the determinants that affect reproduCibility, conduct more comprehensive experiments, and establish best-practice guidelines for recommender -systems research. Expand
Meta-analysis of evaluation methods and metrics used in context-aware scholarly recommender systems
TLDR
Meta-analyses of the evaluation methods and metrics of 67 studies related to context-aware scholarly recommender systems from the years 2000 to 2014 show that offline evaluation methods are more commonly used compared to online and user studies, with the maximum rate of success. Expand
Research-paper recommender systems: a literature survey
TLDR
Several actions could improve the research landscape: developing a common evaluation framework, agreement on the information to include in research papers, a stronger focus on non-accuracy aspects and user modeling, a platform for researchers to exchange information, and an open-source framework that bundles the available recommendation approaches. Expand
A Comparison of Offline Evaluations, Online Evaluations, and User Studies in the Context of Research-Paper Recommender Systems
TLDR
It is concluded that in practice, offline evaluations are probably not suitable to evaluate recommender systems, particularly in the domain of research paper recommendations. Expand
Predicting Online Performance of News Recommender Systems Through Richer Evaluation Metrics
TLDR
This work investigates how metrics that can be measured offline can be used to predict the online performance of recommender systems, thus avoiding costly A-B testing and proposing a self-adjusting algorithm blend that optimizes a recommender's parameters over time. Expand
SeRenA: a semantic recommender for all
TLDR
This study presents SeRenA (Semantic Recommender for All), an unsupervised recommending strategy based on extraction of initial interests through online data and mapped onto a number of Wikipedia documents. Expand
Towards effective research-paper recommender systems and user modeling based on mind maps
TLDR
The findings let us to conclude that user modeling based on mind maps is a promising research field, and that developers of mind-mapping applications should integrate recommender systems into their applications. Expand
Using an Exponential Random Graph Model to Recommend Academic Collaborators
TLDR
This paper proposes a weighting method that can be used to combine two or more social context factors in a recommendation engine that leverages an exponential random graph model (ERGM) based on historical network data. Expand
Deep Latent Factor Models for Recommender Systems
Recommendation systems help users discover relevant items. One of the types of models used to generate the recommendations are latent factor models. We survey the state of the art neural networkExpand
...
1
2
...

References

SHOWING 1-10 OF 114 REFERENCES
Feature-Weighted User Model for Recommender Systems
TLDR
This paper constructs a feature-weighted user profile to disclose the duality between users and features and performs experimental comparison of the proposed method against the well-known CF, CB and a hybrid algorithm with a real data set. Expand
Evaluating Recommendation Systems
TLDR
This paper discusses how to compare recommenders based on a set of properties that are relevant for the application, and focuses on comparative studies, where a few algorithms are compared using some evaluation metric, rather than absolute benchmarking of algorithms. Expand
Beyond Algorithms: An HCI Perspective on Recommender Systems
TLDR
From a user’s perspective, an effective recommender system inspires trust in the system; has system logic that is at least somewhat transparent; points users towards new, not-yet-experienced items; provides details about recommended items, including pictures and community ratings; and finally, provides ways to refine recommendations by including or excluding particular genres. Expand
Item-based collaborative filtering recommendation algorithms
TLDR
This paper analyzes item-based collaborative ltering techniques and suggests that item- based algorithms provide dramatically better performance than user-based algorithms, while at the same time providing better quality than the best available userbased algorithms. Expand
Rank and relevance in novelty and diversity metrics for recommender systems
TLDR
A formal framework for the definition of novelty and diversity metrics is presented that unifies and generalizes several state of the art metrics and identifies three essential ground concepts at the roots of noveltyand diversity: choice, discovery and relevance, upon which the framework is built. Expand
Temporal diversity in recommender systems
TLDR
It is shown that temporal diversity is an important facet of recommender systems, by showing how CF data changes over time and performing a user survey, and proposed and evaluated set methods that maximise temporal recommendation diversity without extensively penalising accuracy. Expand
Recommender systems by means of information retrieval
TLDR
This paper presents a method for reformulating the Recommender Systems problem in an Information Retrieval one, and uses the ratings of users, weighted according to the rank, to predict the rating of the active user. Expand
Users and noise: the magic barrier of recommender systems
TLDR
This work investigates the inconsistencies of the user ratings and estimates the magic barrier in order to assess the actual quality of the recommender system, and presents a mathematical characterization of themagic barrier based on the assumption that user ratings are afflicted with inconsistencies - noise. Expand
Recommender systems
TLDR
This special section includes descriptions of five recommender systems, which provide recommendations as inputs, which the system then aggregates and directs to appropriate recipients, and which combine evaluations with content analysis. Expand
Being accurate is not enough: how accuracy metrics have hurt recommender systems
TLDR
This paper proposes informal arguments that the recommender community should move beyond the conventional accuracy metrics and their associated experimental methodologies, and proposes new user-centric directions for evaluating recommender systems. Expand
...
1
2
3
4
5
...