Comparative recommender system evaluation: benchmarking recommendation frameworks

  title={Comparative recommender system evaluation: benchmarking recommendation frameworks},
  author={A. Said and Alejandro Bellog{\'i}n},
  booktitle={RecSys '14},
  • A. Said, Alejandro Bellogín
  • Published in RecSys '14 2014
  • Computer Science
  • Recommender systems research is often based on comparisons of predictive accuracy: the better the evaluation scores, the better the recommender. [...] Key Method To provide a fair comparison, we have complete control of the evaluation dimensions being benchmarked: dataset, data splitting, evaluation strategies, and metrics. We also include results using the internal evaluation mechanisms of these frameworks. Our analysis points to large differences in recommendation accuracy across frameworks and strategies, i…Expand Abstract
    136 Citations

    Figures, Tables, and Topics from this paper.

    Explore Further: Topics Discussed in This Paper

    Research Paper Recommender System Evaluation Using Coverage
    • 3
    Mix and Rank: A Framework for Benchmarking Recommender Systems
    Reproducibility of Experiments in Recommender Systems Evaluation
    • 4
    • Highly Influenced
    • PDF
    Rival: a toolkit to foster reproducibility in recommender system evaluation
    • 44
    • PDF
    Non-transparent recommender system evaluation leads to misleading results
    Replicable Evaluation of Recommender Systems
    • 6
    • PDF
    Exploring Data Splitting Strategies for the Evaluation of Recommendation Models
    • 1
    • PDF
    Survey on Evaluation of Recommender Systems
    • 6


    Item-based collaborative filtering recommendation algorithms
    • 7,135
    • Highly Influential
    • PDF
    Toward identification and adoption of best practices in algorithmic recommender systems research
    • 17
    • Highly Influential
    Factorization meets the neighborhood: a multifaceted collaborative filtering model
    • 2,846
    • Highly Influential
    • PDF