GODEL: Large-Scale Pre-Training for Goal-Directed Dialog

@article{Peng2022GODELLP,
  title={GODEL: Large-Scale Pre-Training for Goal-Directed Dialog},
  author={Baolin Peng and Michel Galley and Pengcheng He and Chris Brockett and Lars Lid{\'e}n and Elnaz Nouri and Zhou Yu and Bill Dolan and Jianfeng Gao},
  journal={ArXiv},
  year={2022},
  volume={abs/2206.11309}
}
We introduce G ODEL ( G rounded O pen D ialogu e L anguage Model), a large pretrained language model for dialog. In contrast with earlier models such as DialoGPT, G ODEL leverages a new phase of grounded pre-training designed to better support adapt-ing G ODEL to a wide range of downstream dialog tasks that require information external to the current conversation ( e.g., a database or document) to produce good responses. Experiments against an array of benchmarks that encompass task-oriented… 

References

SHOWING 1-10 OF 39 REFERENCES

LaMDA: Language Models for Dialog Applications

TLDR
It is demonstrated that fine-tuning with annotated data and enabling the model to consult external knowledge sources can lead to significant improvements towards the two key challenges of safety and factual grounding.

DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation

TLDR
It is shown that conversational systems that leverage DialoGPT generate more relevant, contentful and context-consistent responses than strong baseline systems.

Soloist: Building Task Bots at Scale with Transfer Learning and Machine Teaching

TLDR
A new method that uses transfer learning and machine teaching to build task bots at scale, Soloist, is presented, which parameterize classical modular task-oriented dialog systems using a Transformer-based auto-regressive language model, which subsumes different dialog modules into a single neural model.

Wizard of Wikipedia: Knowledge-Powered Conversational agents

TLDR
The best performing dialogue models are able to conduct knowledgeable discussions on open-domain topics as evaluated by automatic metrics and human evaluations, while a new benchmark allows for measuring further improvements in this important research direction.

MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling

TLDR
The Multi-Domain Wizard-of-Oz dataset (MultiWOZ), a fully-labeled collection of human-human written conversations spanning over multiple domains and topics is introduced, at a size of 10k dialogues, at least one order of magnitude larger than all previous annotated task-oriented corpora.

Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access

TLDR
An augmented version of MultiWOZ 2.1 is introduced, which includes new out-of-API-coverage turns and responses grounded on external knowledge sources, and defines three sub-tasks: knowledge-seeking turn detection, knowledge selection, and knowledge-grounded response generation, which can be modeled individually or jointly.

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

TLDR
This systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks and achieves state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more.

MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines

TLDR
This work identifies and fixes dialogue state annotation errors across 17.3% of the utterances on top of MultiWOZ 2.1, and redefines the ontology by disallowing vocabularies of slots with a large number of possible values to help avoid annotation errors.

ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons

TLDR
A novel procedure involving comparing two full dialogues, where a human judge is asked to pay attention to only one speaker within each, and make a pairwise judgment, resulting in better tests.

Proactive Human-Machine Conversation with Explicit Conversation Goal

TLDR
Experimental results show that dialogue models that plan over the knowledge graph can make full use of related knowledge to generate more diverse multi-turn conversations.