An Analysis of Mixed Initiative and Collaboration in Information-Seeking Dialogues

@article{Vakulenko2020AnAO,
  title={An Analysis of Mixed Initiative and Collaboration in Information-Seeking Dialogues},
  author={Svitlana Vakulenko and E. Kanoulas and M. de Rijke},
  journal={Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year={2020}
}
The ability to engage in mixed-initiative interaction is one of the core requirements for a conversational search system. How to achieve this is poorly understood. We propose a set of unsupervised metrics, termed ConversationShape, that highlights the role each of the conversation participants plays by comparing the distribution of vocabulary and utterance types. Using ConversationShape as a lens, we take a closer look at several conversational search datasets and compare them with other… 

Figures and Tables from this paper

A Large-scale Analysis of Mixed Initiative in Information-Seeking Dialogues for Conversational Search

A large-scale dialogue analysis of more than 150K transcripts from 16 publicly available dialogue datasets is performed to establish close relations between conversational search and other conversational AI tasks and uncover limitations of existing conversational datasets to inform future data collection tasks.

Helpfulness and Fairness of Task-Oriented Dialogue Systems

It is found that dialogue systems tend to be more helpful for highly- developed countries than less-developed countries, uncovering a fairness issue underlying these dialogue systems.

Report on the future conversations workshop at CHIIR 2021

This paper summarises the submitted statements and the discussions the authors had during the two sessions of the workshop on conversational interactions and solicited position statements on opportunities, problems, and solutions in conversational search in all modalities.

On Helpfulness Of Task-Oriented Dialogue Systems And Its Fairness Implication

Computational measure-ments of helpfulness are studied and it is proposed to use the helpfulness level of a Dialogue system towards different user queries to measure the fairness of a dialogue system.

Initiative-Aware Self-Supervised Learning for Knowledge-Grounded Conversations

Experimental results on two datasets show that MIKe significantly outperforms state-of-the-art methods in terms of both automatic and human evaluations, indicating that it can select more appropriate knowledge and generate more informative and engaging responses.

KuaiRec: A Fully-observed Dataset for Recommender Systems

The experimental results show that two factors in traditional partially-observed data — the data density and the exposure bias — greatly affect the evaluation results, and the significance of the KuaiRec data in researching many directions in recommender systems, e.g., the unbiased recommendation, interactive/conversational recommendation, and evaluation.

Semi-Supervised Variational Reasoning for Medical Dialogue Generation

This work proposes an end-to-end variational reasoning approach to medical dialogue generation that outperforms state-of-the-art baselines in terms of objective and subjective evaluation metrics and uses an efficient stochastic gradient variational Bayes estimator to optimize the derived evidence lower bound.

References

SHOWING 1-10 OF 28 REFERENCES

An Evidential Model for Tracking Initiative in Collaborative Dialogue Interactions

An evidential model for tracking shifts in both types of initiatives in collaborative dialogue interactions is presented and how this initiative tracking model may be employed by a dialogue system to enable the system to tailor its responses to user utterances based on application domain, system's role in the domain, dialogue history, and user characteristics is shown.

Introducing MANtIS: a novel Multi-Domain Information Seeking Dialogues Dataset

Conversational search is an approach to information retrieval (IR), where users engage in a dialogue with an agent in order to satisfy their information needs. Previous conceptual work described

Mixed Initiative in Dialogue: An Investigation into Discourse Segmentation

A set of rules for the transfer of control is applied to 4 sets of dialogues and it is found that both allocation of control and the manner in which control is transferred is radically different for the two dialogue types.

Analyzing and Characterizing User Intent in Information-seeking Conversations

A new dataset designed for this purpose is introduced and used to analyze information-seeking conversations by user intent distribution, co-occurrence, and flow patterns and finds some highly recurring patterns in user intent during an information- seeking process.

Wizard of Wikipedia: Knowledge-Powered Conversational agents

The best performing dialogue models are able to conduct knowledgeable discussions on open-domain topics as evaluated by automatic metrics and human evaluations, while a new benchmark allows for measuring further improvements in this important research direction.

Informing the Design of Spoken Conversational Search: Perspective Paper

This work enhances the understanding of different search behaviours and proposes research opportunities for an audio-only search system for Spoken Conversational Search.

A Theoretical Framework for Conversational Search

This paper studies conversational approaches to information retrieval, presenting a theory and model of information interaction in a chat setting, and shows that while theoretical, the model could be practically implemented to satisfy the desirable properties presented.

What makes a good conversation? How controllable attributes affect human judgments

This work examines two controllable neural text generation methods, conditional training and weighted decoding, in order to control four important attributes for chit-chat dialogue: repetition, specificity, response-relatedness and question-asking, and shows that by controlling combinations of these variables their models obtain clear improvements in human quality judgments.

MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling

The Multi-Domain Wizard-of-Oz dataset (MultiWOZ), a fully-labeled collection of human-human written conversations spanning over multiple domains and topics is introduced, at a size of 10k dialogues, at least one order of magnitude larger than all previous annotated task-oriented corpora.

Coached Conversational Preference Elicitation: A Case Study in Understanding Movie Preferences

A new approach to obtaining user preferences in dialogue: Coached Conversational Preference Elicitation that allows collection of natural yet structured conversational preferences.