From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442)

@article{Ferro2018FromET,
  title={From Evaluating to Forecasting Performance: How to Turn Information Retrieval, Natural Language Processing and Recommender Systems into Predictive Sciences (Dagstuhl Perspectives Workshop 17442)},
  author={N. Ferro and N. Fuhr and G. Grefenstette and J. Konstan and P. Castells and E. Daly and Thierry Declerck and Michael D. Ekstrand and W. Geyer and J. Gonzalo and T. Kuflik and Krister Lind{\'e}n and B. Magnini and Jian-Yun Nie and R. Perego and Bracha Shapira and I. Soboroff and N. Tintarev and Karin M. Verspoor and M. C. Willemsen and J. Zobel},
  journal={Dagstuhl Manifestos},
  year={2018},
  volume={7},
  pages={96-139}
}
  • N. Ferro, N. Fuhr, +18 authors J. Zobel
  • Published 2018
  • Computer Science
  • Dagstuhl Manifestos
  • We describe the state-of-the-art in performance modeling and prediction for Information Retrieval (IR), Natural Language Processing (NLP) and Recommender Systems (RecSys) along with its shortcomings and strengths. We present a framework for further research, identifying five major problem areas: understanding measures, performance analysis, making underlying assumptions explicit, identifying application features determining performance, and the development of prediction models describing the… CONTINUE READING
    Offline evaluation options for recommender systems
    • 4
    Limits to Surprise in Recommender Systems
    • 2
    • Open Access
    The Information Retrieval Group at the University of Duisburg-Essen
    • 1
    Evaluating Multimedia and Language Tasks
    ‘Algorithm-Performance Personas’ for Siamese Meta-Learning and Automated Algorithm Selection
    • Bryan Tyrrell
    • 2020

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 119 REFERENCES
    Item-based top-N recommendation algorithms
    • 1,890
    • Open Access
    The Netflix Prize
    • 1,675
    • Open Access
    Predicting query performance
    • 689
    • Open Access
    The MovieLens Datasets: History and Context
    • 1,595
    • Open Access