Towards reproducibility in recommender-systems research

  title={Towards reproducibility in recommender-systems research},
  author={J. Beel and Corinna Breitinger and S. Langer and A. Lommatzsch and Bela Gipp},
  journal={User Modeling and User-Adapted Interaction},
  • J. Beel, Corinna Breitinger, +2 authors Bela Gipp
  • Published 2016
  • Computer Science
  • User Modeling and User-Adapted Interaction
  • Numerous recommendation approaches are in use today. However, comparing their effectiveness is a challenging task because evaluation results are rarely reproducible. In this article, we examine the challenge of reproducibility in recommender-system research. We conduct experiments using Plista’s news recommender system, and Docear’s research-paper recommender system. The experiments show that there are large discrepancies in the effectiveness of identical recommendation approaches in only… CONTINUE READING
    57 Citations

    Figures, Tables, and Topics from this paper.

    Collaborative Filtering: Matrix Completion and Session-Based Recommendation Tasks
    • 1
    • PDF
    User Experience and Recommender Systems
    • 5
    It's Time to Consider "Time" when Evaluating Recommender-System Algorithms [Proposal]
    • 2
    • PDF
    A Novel Approach to Recommendation Algorithm Selection using Meta-Learning
    • 11
    • PDF
    Document Embeddings vs. Keyphrases vs. Terms for Recommender Systems: A Large-Scale Online Evaluation
    • A. Collins, J. Beel
    • Computer Science
    • 2019 ACM/IEEE Joint Conference on Digital Libraries (JCDL)
    • 2019
    • 3
    One-at-a-time: A Meta-Learning Recommender-System for Recommendation-Algorithm Selection on Micro Level
    • 9
    • PDF
    Real World Evaluation of Approaches to Research Paper Recommendation
    • 2
    • PDF


    Evaluating Recommendation Systems
    • 1,026
    • PDF
    A Survey of Accuracy Evaluation Metrics of Recommendation Tasks
    • 461
    • PDF
    Research-paper recommender systems: a literature survey
    • 411
    • PDF
    The Comparability of Recommender System Evaluations and Characteristics of Docear ’ s Users
    • Langer Docear
    • 2014
    • 6
    • PDF
    Rival: a toolkit to foster reproducibility in recommender system evaluation
    • 44
    • PDF
    Evaluating the Accuracy and Utility of Recommender Systems
    • 11
    • Highly Influential
    • PDF
    What Recommenders Recommend - An Analysis of Accuracy, Popularity, and Sales Diversity Effects
    • 90
    • PDF
    Recommender systems: from algorithms to user experience
    • 567
    • Highly Influential
    • PDF
    Layered Evaluation of Multi-Criteria Collaborative Filtering for Scientific Paper Recommendation
    • 20
    • PDF
    Do clicks measure recommendation relevancy?: an empirical user study
    • 21