Optimizing Interactive Systems with Data-Driven Objectives

  title={Optimizing Interactive Systems with Data-Driven Objectives},
  author={Ziming Li and Artem Grotov and Julia Kiseleva and M. de Rijke and Harrie Oosterhuis},
  booktitle={International Joint Conference on Artificial Intelligence},
Effective optimization is essential for interactive systems to provide a satisfactory user experience. However, it is often challenging to find an objective to optimize for. Generally, such objectives are manually crafted and rarely capture complex user needs in an accurate manner. We propose to infer the objective directly from observed user interactions. These inferences can be made regardless of prior knowledge and across different types of user behavior. It is promising if we model the… 
2 Citations

Figures and Tables from this paper

Optimizing web search engines with interactions

This thesis studies how to use observed interactions between the user and the web search engine to optimize search engines using large datasets annotated by human judges.

Learning from User Interactions with Rankings: A Unification of the Field

The second part of this thesis proposes a framework that bridges many gaps between areas of online, counterfactual, and supervised learning to rank that has taken approaches, previously considered independent, and unified them into a single methodology for widely applicable and effective learning toRank from user clicks.



UvA-DARE Optimizing Interactive Systems via Data-Driven Objectives

This work proposes a new general principled approach to optimizing interactive systems using data-driven objectives that infers the objective directly from observed user interactions and can be made regardless of prior knowledge and across different types of user behavior.

A I ] 8 M ay 2 01 8 Optimizing Interactive Systems with Data-Driven Objectives

Interactive System Optimizer (ISO) is introduced, a novel algorithm that uses inferred objectives directly from observed user interactions for optimization of interactive systems using data-driven objectives.

Understanding User Satisfaction with Intelligent Assistants

A user study designed to measure user satisfaction over a range of typical scenarios of use is described, finding that the notion of satisfaction varies across different scenarios, and that overall task-level satisfaction cannot be reduced to query- level satisfaction alone.

Cohort modeling for enhanced personalized search

It is shown via extensive experimentation with large-scale logs from a commercial search engine that leveraging cohort behavior can yield significant relevance gains when combined with a production search engine ranking algorithm that uses similar classes of personalization signal but at the individual searcher level.

Predicting User Satisfaction with Intelligent Assistants

This paper proposes an automatic method to predict user satisfaction with intelligent assistants that exploits all the interaction signals, including voice commands and physical touch gestures on the device, and finds that interaction signals that capture the user reading patterns have a high impact.

Shaping Feedback Data in Recommender Systems with Interventions Based on Information Foraging Theory

This work proposes shaping the feedback generation process as an alternate and complementary route to improving accuracy inRecommender systems and explores how changes to the user interface can impact the quality and quantity of feedback data -- and therefore the learning accuracy.

Reusing historical interaction data for faster online learning to rank for IR

This paper investigates whether and how previously collected (historical) interaction data can be used to speed up learning in online learning to rank for IR and shows that historical data canSpeed up learning, leading to substantially and significantly higher online performance.

A User Simulator for Task-Completion Dialogues

A new, publicly available simulation framework, where the simulator, designed for the movie-booking domain, leverages both rules and collected data, and several agents are demonstrated and the procedure to add and test your own agent is detailed.

Evaluating implicit measures to improve web search

There was an association between implicit measures of user activity and the user's explicit satisfaction ratings, and the best models for individual pages combined clickthrough, time spent on the search result page, and how a user exited a result or ended a search session.

Evaluating implicit feedback models using searcher simulations

Six different models that base their decisions on the interactions of searchers and use different approaches to rank query modification terms are introduced, to determine which of these models should be used to assist searchers in the systems the authors develop.