• Corpus ID: 199064536

Sudden Death: A New Way to Compare Recommendation Diversification

@article{Bridge2019SuddenDA,
  title={Sudden Death: A New Way to Compare Recommendation Diversification},
  author={Derek G. Bridge and Mesut Kaya and Pablo Castells},
  journal={ArXiv},
  year={2019},
  volume={abs/1908.00419}
}
This paper describes problems with the current way we compare the diversity of different recommendation lists in offline experiments. We illustrate the problems with a case study. We propose the Sudden Death score as a new and better way of making these comparisons. 

Figures from this paper

References

SHOWING 1-9 OF 9 REFERENCES
Improving recommendation lists through topic diversification
TLDR
This work presents topic diversification, a novel method designed to balance and diversify personalized recommendation lists in order to reflect the user's complete spectrum of interests, and introduces the intra-list similarity metric to assess the topical diversity of recommendation lists.
Novelty and Diversity in Recommender Systems
TLDR
An overview of the main contributions to this area in the field of recommender systems, and seeks to relate them together in a unified view, analyzing the common elements underneath the different forms under which novelty and diversity have been addressed, and identifying connections to closely related work on diversity in other fields.
A comparison of calibrated and intent-aware recommendations
TLDR
It is found that defining interests in terms of subprofiles results in highest precision and the best relevance/diversity trade-off.
An on-line evaluation framework for recommender systems
TLDR
It is argued that while off-line analysis is useful, user satisfaction with a recommendation strategy can only be measured in an on-line context and a new evaluation framework is proposed which involves a paired test of two recommender systems which simultaneously compete to give the best recommendations to the same user at the same time.
Novelty and diversity in information retrieval evaluation
TLDR
This paper develops a framework for evaluation that systematically rewards novelty and diversity into a specific evaluation measure, based on cumulative gain, and demonstrates the feasibility of this approach using a test collection based on the TREC question answering track.
The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries
TLDR
A method for combining query-relevance with information-novelty in the context of text retrieval and summarization and preliminary results indicate some benefits for MMR diversity ranking in document retrieval and in single document summarization.
The use of MMR, diversity-based reranking for reordering documents and producing summaries
TLDR
This paper presents a method for combining query-relevance with information-novelty in the context of text retrieval and summarization, and preliminary results indicate some benefits for MMR diversity ranking in document retrieval and in single document summarization.
Less is more: probabilistic models for retrieving fewer relevant documents
TLDR
This work considers a number of information retrieval metrics from the literature, including the rank of the first relevant result, the %no metric that penalizes a system only for retrieving no relevant results near the top, and the diversity of retrieved results when queries have multiple interpretations.
Subprofile-aware diversification of recommendations
TLDR
A new form of intent-aware diversification, which is called SPAD (Subprofile-Aware Diversification), and a variant called RSPAD (Relevance-based SPAD), where the aspects are not item features; they are subprofiles of the user’s profile.