Analyzing and Learning from User Interactions for Search Clarification

  title={Analyzing and Learning from User Interactions for Search Clarification},
  author={Hamed Zamani and Bhaskar Mitra and Everest Chen and Gord Lueck and Fernando Diaz and Paul N. Bennett and Nick Craswell and Susan T. Dumais},
  journal={Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval},
  • Hamed Zamani, Bhaskar Mitra, S. Dumais
  • Published 30 May 2020
  • Computer Science
  • Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval
Asking clarifying questions in response to search queries has been recognized as a useful technique for revealing the underlying intent of the query. Clarification has applications in retrieval systems with different interfaces, from the traditional web search interfaces to the limited bandwidth interfaces as in speech-only and small screen devices. Generation and evaluation of clarifying questions have been recently studied in the literature. However, user interaction with clarifying questions… 
Generating Clarifying Questions with Web Search Results
This work designs a rule-based algorithm to extract description candidates from search results and rank them by various human-designed features and applies an learning-to-rank model and another generative model for generalization to improve the quality of clarifying questions.
Users Meet Clarifying Questions: Toward a Better Understanding of User Interactions for Search Clarification
A large user study is conducted to understand the interaction of users with CQs in various quality categories, and the effect of CQ quality on user search performance in terms of finding relevant information, search behavior, and user satisfaction.
A User Study on Clarifying Comparative Questions
In an experiment that reflects a conversational search interface with a clarification component, 70% of the study participants find clarifications useful to retrieve relevant results for questions with unclear comparison aspects.
User Engagement Prediction for Clarification in Search
This work model search clarification prediction as user engagement problem and proposes a Transformerbased model to tackle the task, assuming that the better a clarification is, the higher user engagement with it would be.
Template-guided Clarifying Question Generation for Web Search Clarification
This paper tackles the problem of generating clarifying questions for Web search by jointly learning to select question templates and fill question slots, using Transformer-based networks, and demonstrates to achieve significant improvements over various competitive baselines.
MIMICS: A Large-Scale Data Collection for Search Clarification
MIMICS, a collection of search clarification datasets for real web search queries sampled from the Bing query logs, is introduced and the results returned by the Bing's web search API for all the queries in MIMICS are released to allow researchers to utilize search results for the tasks related to search clarification.
Analysing the Effect of Clarifying Questions on Document Ranking in Conversational Search
There needs to be some fine-grained treatment of the entire conversational round of clarification, based on the explicit feedback which is present in such mixed-initiative settings, and a simple heuristic-based lexical baseline is introduced, that significantly outperforms the existing naive baselines.
Simulating and Modeling the Risk of Conversational Search
A simulation framework to simulate the risk of asking questions in Conversational search and a risk-aware conversational search model to control the risk is proposed and shown to be robustness and effectiveness.
Ranking Clarifying Questions Based on Predicted User Engagement
This research aims to predict the user engagement with the clarification pane as an indicator of relevance based on the lexical information: query, question, and answers and can serve as a first neural baseline for future research to improve on.
How to Approach Ambiguous Queries in Conversational Search? A Survey of Techniques, Approaches, Tools and Challenges
This work provides a divine overview of characteristics of ambiguous queries and contributes to better understanding of the existing technologies and challenges in CSS focus on disambiguation of unclear queries from various dimensions.


Generating Clarifying Questions for Information Retrieval
A taxonomy of clarification for open-domain search queries is identified by analyzing large-scale query reformulation data sampled from Bing search logs, and supervised and reinforcement learning models for generating clarifying questions learned from weak supervision data are proposed.
Asking Clarifying Questions in Open-Domain Information-Seeking Conversations
This paper formulate the task of asking clarifying questions in open-domain information-seeking conversational systems, propose an offline evaluation methodology for the task, and collect a dataset, called Qulac, through crowdsourcing, which significantly outperforms competitive baselines.
What Do You Mean Exactly?: Analyzing Clarification Questions in CQA
The dialogues between the users on a community question answering (CQA) website is explored as a rich repository of information-seeking interactions and the problem of predicting the specific subject of a clarification question is explored, a first step towards automatic generation of clarification questions.
On user interactions with query auto-completion
This paper presents the first large-scale study of user interactions with auto-completion based on query logs of Bing, a commercial search engine, and confirms that lower-ranked auto- completion suggestions receive substantially lower engagement than those ranked higher.
Towards Conversational Search and Recommendation: System Ask, User Respond
This paper proposes a System Ask -- User Respond (SAUR) paradigm for conversational search, defines the major components of the paradigm, and designs a unified implementation of the framework for product search and recommendation in e-commerce.
Toward Voice Query Clarification
It is found that user experience depends on language proficiency levels, users are not dissatisfied when prompted for clarifications, and the most effective way of query clarification depends on the number and lengths of the possible answers.
Beyond clicks: query reformulation as a predictor of search satisfaction
It is shown that a query-based model (with no click information) can indicate satisfaction more accurately than click-based models, and that search success is an incremental process for successful tasks with multiple queries.
Information Retrieval with Verbose Queries
This tutorial aims to put together various research pieces of the puzzle, provide a comprehensive and structured overview of various proposed methods, and also list various application scenarios where effective verbose query processing can make a significant difference.
HARD Track Overview in TREC 2003: High Accuracy Retrieval from Documents
Abstract : The High Accuracy Retrieval from Documents (HARD) track explores methods for improving the accuracy of document retrieval systems. It does so by considering three questions. Can additional
Predicting Search Intent Based on Pre-Search Context
A mixture generative model is developed to learn without any supervision how queries are triggered by different types of pre-search context and is shown to accurately predict user search intent with pre- search context and improve upon the state-of-the-art methods significantly.