A comparative analysis of offline and online evaluations and discussion of research paper recommender system evaluation

@inproceedings{Beel2013ACA,
  title={A comparative analysis of offline and online evaluations and discussion of research paper recommender system evaluation},
  author={J. Beel and Marcel Genzmehr and S. Langer and A. N{\"u}rnberger and Bela Gipp},
  booktitle={RepSys '13},
  year={2013}
}
  • J. Beel, Marcel Genzmehr, +2 authors Bela Gipp
  • Published in RepSys '13 2013
  • Computer Science
  • Offline evaluations are the most common evaluation method for research paper recommender systems. However, no thorough discussion on the appropriateness of offline evaluations has taken place, despite some voiced criticism. We conducted a study in which we evaluated various recommendation approaches with both offline and online evaluations. We found that results of offline and online evaluations often contradict each other. We discuss this finding in detail and conclude that offline evaluations… CONTINUE READING
    95 Citations

    Figures and Topics from this paper.

    Comparing Offline and Online Recommender System Evaluations on Long-tail Distributions
    • 3
    • PDF
    Research paper recommender system evaluation: a quantitative literature survey
    • 96
    • PDF
    Random Performance Differences Between Online Recommender System Algorithms
    • 5
    • PDF
    Meta-analysis of evaluation methods and metrics used in context-aware scholarly recommender systems
    • 7
    Revisiting offline evaluation for implicit-feedback recommender systems
    • 5
    • PDF

    References

    SHOWING 1-2 OF 2 REFERENCES
    Recommender Systems Handbook
    • 1,689
    • Highly Influential
    • PDF
    CiteSeer: an autonomous Web agent for automatic retrieval and identification of interesting publications
    • 344
    • Highly Influential
    • PDF