Rating support interfaces to improve user experience and recommender accuracy

@article{Nguyen2013RatingSI,
  title={Rating support interfaces to improve user experience and recommender accuracy},
  author={Tien T. Nguyen and Daniel Kluver and Ting-Yu Wang and Pik-Mai Hui and Michael D. Ekstrand and Martijn C. Willemsen and John Riedl},
  journal={Proceedings of the 7th ACM conference on Recommender systems},
  year={2013}
}
One of the challenges for recommender systems is that users struggle to accurately map their internal preferences to external measures of quality such as ratings. We study two methods for supporting the mapping process: (i) reminding the user of characteristics of items by providing personalized tags and (ii) relating rating decisions to prior rating decisions using exemplars. In our study, we introduce interfaces that provide these methods of support. We also present a set of methodologies to… 

Figures and Tables from this paper

Pairwise Preferences and Recommender Systems
TLDR
This research work focuses on pairwise preferences as an alternative way for modeling user preferences and compute recommendations, and aims at identifying specific conditions/situations where pairwise preference elicitation is meaningful and beneficial.
Leveraging prior ratings for recommender systems in e-commerce
Evaluating Recommender Systems with User Experiments
TLDR
This chapter provides a detailed practical description of how to conduct user experiments, covering the following topics: formulating hypotheses, sampling participants, creating experimental manipulations, measuring subjective constructs with questionnaires, and statistically evaluating the results.
Interacting with Recommenders—Overview and Research Directions
TLDR
This work provides a comprehensive overview on the existing literature on user interaction aspects in recommender systems, covering existing approaches for preference elicitation and result presentation, as well as proposals that consider recommendation as an interactive process.
Coherence and inconsistencies in rating behavior: estimating the magic barrier of recommender systems
TLDR
This work presents a mathematical characterization of the magic barrier based on the assumption that user ratings are afflicted with inconsistencies—noise, and proposes a measure of the consistency of user ratings (rating coherence) that predicts the performance of recommendation methods.
Towards Recommender Engineering: tools and experiments for identifying recommender differences
TLDR
The LensKit toolkit for conducting experiments on a wide variety of recommender algorithms and data sets under different experimental conditions, along with new developments in object-oriented software configuration to support this toolkit, and experiments on the configuration options of widely-used algorithms to provide guidance on tuning and configuring them are made.
Pairwise Preferences Elicitation and Exploitation for Conversational Collaborative Filtering
TLDR
This work has developed a recommendation technology that, combining ratings and pairwise preferences, can generate better recommendations than a state of the art solution uniquely based on ratings.
rScholar: An Interactive Contextual User Interface to Enhance UX of Scholarly Recommender Systems
Scholarly recommender systems attempt to reduce the number of research resources or papers presented to scholars and predict the utility of resources for their scholarly tasks. Industry practitioners
Learning from Sets of Items in Recommender Systems
TLDR
This article investigates two questions related to using set- level ratings in recommender systems, how users’ item-level ratings relate to their set-level rating prediction, and how collaborative filtering-based models for item- level rating prediction can take advantage of such set-levels ratings.
Improving recommender systems: user roles and lifecycles
TLDR
This dissertation investigates how the data collection methods and the life cycles of users affect the prediction accuracies and the performance of recommendation algorithms.
...
...

References

SHOWING 1-10 OF 34 REFERENCES
Users and noise: the magic barrier of recommender systems
TLDR
This work investigates the inconsistencies of the user ratings and estimates the magic barrier in order to assess the actual quality of the recommender system, and presents a mathematical characterization of themagic barrier based on the assumption that user ratings are afflicted with inconsistencies - noise.
Interfaces for Eliciting New User Preferences in Recommender Systems
TLDR
It is found that the two pure interfaces both produced accurate user models, but that directly asking users for items to rate increases user loyalty in the system.
Evaluating collaborative filtering recommender systems
TLDR
The key decisions in evaluating collaborative filtering recommender systems are reviewed: the user tasks being evaluated, the types of analysis and datasets being used, the ways in which prediction quality is measured, the evaluation of prediction attributes other than quality, and the user-based evaluation of the system as a whole.
Predicting Users' Preference from Tag Relevance
TLDR
A novel approach to infer user preference from tag relevance, an indication of how strong each tag applies to each item in recommender systems, and a methodology to choose tags that tell most about each user’s preference.
Explaining the user experience of recommender systems
TLDR
This paper proposes a framework that takes a user-centric approach to recommender system evaluation that links objective system aspects to objective user behavior through a series of perceptual and evaluative constructs (called subjective system aspects and experience, respectively).
Recommender systems, consumer preferences, and anchoring effects
TLDR
Results provide strong evidence viewers’ preferences are malleable and can be significantly influenced by ratings provided by recommender systems.
Rate it again: increasing recommendation accuracy by user re-rating
TLDR
A novel approach to improve RS accuracy by reducing the natural noise in the input data via a preprocessing step and proposing a novel algorithm to denoise existing datasets by means of re-rating: i.e. by asking users to rate previously rated items again.
The design space of opinion measurement interfaces: exploring recall support for rating and ranking
TLDR
It is found that designing ranking systems is challenging; there may be a mismatch between a ranking interface that forces people to specify a total ordering for a set of items, and their mental model that some items are not directly comparable to each other.
An Economic Model of User Rating in an Online Recommender System
TLDR
It is found that while economic modeling in this domain requires an initial understanding of user behavior and access to an uncommonly broad set of user survey and behavioral data, it returns significant formal understanding of the activity being modeled.
Remembering the stars?: effect of time on preference retrieval from memory
TLDR
A psychological process, the "positivity effect", that influences the retrieval of quality judgments from the authors' memory by which pleasant items are being processed and recalled from memory more effectively than unpleasant items.
...
...