GODEL: Large-Scale Pre-Training for Goal-Directed Dialog

  title={GODEL: Large-Scale Pre-Training for Goal-Directed Dialog},
  author={Baolin Peng and Michel Galley and Pengcheng He and Chris Brockett and Lars Lid{\'e}n and Elnaz Nouri and Zhou Yu and Bill Dolan and Jianfeng Gao},
We introduce G ODEL ( G rounded O pen D ialogu e L anguage Model), a large pretrained language model for dialog. In contrast with earlier models such as DialoGPT, G ODEL leverages a new phase of grounded pre-training designed to better support adapt-ing G ODEL to a wide range of downstream dialog tasks that require information external to the current conversation ( e.g., a database or document) to produce good responses. Experiments against an array of benchmarks that encompass task-oriented… 



LaMDA: Language Models for Dialog Applications

It is demonstrated that fine-tuning with annotated data and enabling the model to consult external knowledge sources can lead to significant improvements towards the two key challenges of safety and factual grounding.

DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation

It is shown that conversational systems that leverage DialoGPT generate more relevant, contentful and context-consistent responses than strong baseline systems.

Soloist: Building Task Bots at Scale with Transfer Learning and Machine Teaching

A new method that uses transfer learning and machine teaching to build task bots at scale, Soloist, is presented, which parameterize classical modular task-oriented dialog systems using a Transformer-based auto-regressive language model, which subsumes different dialog modules into a single neural model.

Wizard of Wikipedia: Knowledge-Powered Conversational agents

The best performing dialogue models are able to conduct knowledgeable discussions on open-domain topics as evaluated by automatic metrics and human evaluations, while a new benchmark allows for measuring further improvements in this important research direction.

MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling

The Multi-Domain Wizard-of-Oz dataset (MultiWOZ), a fully-labeled collection of human-human written conversations spanning over multiple domains and topics is introduced, at a size of 10k dialogues, at least one order of magnitude larger than all previous annotated task-oriented corpora.

Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access

An augmented version of MultiWOZ 2.1 is introduced, which includes new out-of-API-coverage turns and responses grounded on external knowledge sources, and defines three sub-tasks: knowledge-seeking turn detection, knowledge selection, and knowledge-grounded response generation, which can be modeled individually or jointly.

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

This systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks and achieves state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more.

MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines

This work identifies and fixes dialogue state annotation errors across 17.3% of the utterances on top of MultiWOZ 2.1, and redefines the ontology by disallowing vocabularies of slots with a large number of possible values to help avoid annotation errors.

ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons

A novel procedure involving comparing two full dialogues, where a human judge is asked to pay attention to only one speaker within each, and make a pairwise judgment, resulting in better tests.

Proactive Human-Machine Conversation with Explicit Conversation Goal

Experimental results show that dialogue models that plan over the knowledge graph can make full use of related knowledge to generate more diverse multi-turn conversations.