Neural Response Generation with Dynamic Vocabularies

@article{Wu2017NeuralRG,
  title={Neural Response Generation with Dynamic Vocabularies},
  author={Yu Wu and Wei Yu Wu and Dejian Yang and Can Xu and Zhoujun Li and Ming Zhou},
  journal={ArXiv},
  year={2017},
  volume={abs/1711.11191}
}
We study response generation for open domain conversation in chatbots. [] Key Method In training, vocabulary construction and response generation are jointly learned by maximizing a lower bound of the true objective with a Monte Carlo sampling method. In inference, the model dynamically allocates a small vocabulary for an input with the word prediction model, and conducts decoding only with the small vocabulary.

Figures and Tables from this paper

DVKCM: Knowledge-guided Conversation Generation with Dynamic Vocabulary

This paper proposes a Dynamic Vocabulary based Knowledge-guided Conversation Model (DVKCM), which adopts the vocabulary construction module to allocate the sentence-level vocabulary which relates to the input sentence and background knowledge, and then only uses the small vocabulary to execute the decoding part.

Cue-word Driven Neural Response Generation with a Shrinking Vocabulary

A novel but natural approach that can produce multiple cue-words during decoding, and then uses the produced cue- words to drive decoding and shrinks the decoding vocabulary is proposed, which can converge to concrete semantics more efficiently during decoding.

GLSE: Global-Local Selective Encoding for Response Generation in Neural Conversation Model

  • Hongli WangJiangtao Ren
  • Computer Science
    2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI)
  • 2019
A Global-Local Selective Encoding model (GLSE) is proposed to extend the seq2seq framework to generate more relevant and informative responses and empirical studies indicate the advantage of the model over several classical and strong baselines.

HL-EncDec: A Hybrid-Level Encoder-Decoder for Neural Response Generation

Experimental results show the proposed hybrid-level Encoder-Decoder model (HL-EncDec) significantly outperforms other non-word-level models in automatic metrics and human annotations and is able to generate more informative responses.

Content Word-based Sentence Decoding and Evaluating for Open-domain Neural Response Generation

Motivated by Broca's aphasia, a content word sequence is proposed to use as an intermediate representation for open-domain response generation and conventional metrics on content word sequences are proposed, which are a better indicator of content relevance.

Vocabulary Pyramid Network: Multi-Pass Encoding and Decoding with Multi-Level Vocabularies for Response Generation

A Vocabulary Pyramid Network (VPN) is presented which is able to incorporate multi-pass encoding and decoding with multi-level vocabularies into response generation and remarkably outperforms strong baselines.

LSDSCC: a Large Scale Domain-Specific Conversational Corpus for Response Generation with Diversity Oriented Evaluation Metrics

The experimental results have shown that the proposed corpus can be taken as a new benchmark dataset for the NRG task, and the presented metrics are promising to guide the optimization of NRG models by quantifying the diversity of the generated responses reasonably.

ReBoost: a retrieval-boosted sequence-to-sequence model for neural response generation

This work proposes to utilize retrieved responses to boost the Seq2seq model for generating more informative replies and demonstrates that the model is able to generate more informative responses in both automatic and human evaluations and outperforms the state-of-the-art response generation models.

ReBoost: a retrieval-boosted sequence-to-sequence model for neural response generation

This work proposes to utilize retrieved responses to boost the Seq2seq model for generating more informative replies and demonstrates that the model is able to generate more informative responses in both automatic and human evaluations and outperforms the state-of-the-art response generation models.

Improving Matching Models with Contextualized Word Representations for Multi-turn Response Selection in Retrieval-based Chatbots

Experimental results on two benchmark datasets indicate that the proposed contextualized word vectors can significantly and consistently improve the performance of existing matching models for response selection.
...

References

SHOWING 1-10 OF 38 REFERENCES

Generating Long and Diverse Responses with Neural Conversation Models

This work presents new training and decoding methods that improve the quality, coherence, and diversity of long responses generated using sequence-to-sequence models, and introduces a stochastic beam-search algorithm with segment-by-segment reranking which lets us inject diversity earlier in the generation process.

Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models

This work focuses on the single turn setting, introduces a stochastic beam-search algorithm with segment-by-segment reranking which lets us inject diversity earlier in the generation process, and proposes a practical approach, called the glimpse-model, for scaling to large datasets.

A Diversity-Promoting Objective Function for Neural Conversation Models

This work proposes using Maximum Mutual Information (MMI) as the objective function in neural models, and demonstrates that the proposed MMI models produce more diverse, interesting, and appropriate responses, yielding substantive gains in BLEU scores on two conversational datasets and in human evaluations.

Topic Aware Neural Response Generation

A topic aware sequence-to-sequence (TA-Seq2Seq) model that utilizes topics to simulate prior human knowledge that guides them to form informative and interesting responses in conversation, and leverages topic information in generation by a joint attention mechanism and a biased generation probability.

A Neural Conversational Model

A simple approach to conversational modeling which uses the recently proposed sequence to sequence framework, and is able to extract knowledge from both a domain specific dataset, and from a large, noisy, and general domain dataset of movie subtitles.

A Dataset for Research on Short-Text Conversations

This paper introduces a dataset of short-text conversation based on the real-world instances from Sina Weibo, which provides rich collection of instances for the research on finding natural and relevant short responses to a given short text, and useful for both training and testing of conversation models.

Data-Driven Response Generation in Social Media

It is found that mapping conversational stimuli onto responses is more difficult than translating between languages, due to the wider range of possible responses, the larger fraction of unaligned words/phrases, and the presence of large phrase pairs whose alignment cannot be further decomposed.

Vocabulary Manipulation for Neural Machine Translation

This paper introduces a sentence-level or batch-level vocabulary, which is only a very small sub-set of the full output vocabulary for each sentence or batch, which reduces both the computing time and the memory usage of neural machine translation models.

Neural Responding Machine for Short-Text Conversation

Empirical study shows that NRM can generate grammatically correct and content-wise appropriate responses to over 75% of the input text, outperforming state-of-the-arts in the same setting, including retrieval-based and SMT-based models.

Learning to Respond with Deep Neural Networks for Retrieval-Based Human-Computer Conversation System

This paper proposes a retrieval-based conversation system with the deep learning-to-respond schema through a deep neural network framework driven by web data and demonstrates significant performance improvement against a series of standard and state-of-art baselines for conversational purposes.