• Corpus ID: 12300158

A Neural Conversational Model

@article{Vinyals2015ANC,
  title={A Neural Conversational Model},
  author={Oriol Vinyals and Quoc V. Le},
  journal={ArXiv},
  year={2015},
  volume={abs/1506.05869}
}
Conversational modeling is an important task in natural language understanding and machine intelligence. [] Key Method Our model converses by predicting the next sentence given the previous sentence or sentences in a conversation. The strength of our model is that it can be trained end-to-end and thus requires much fewer hand-crafted rules. We find that this straightforward model can generate simple conversations given a large conversational training dataset. Our preliminary results suggest that, despite…

Figures from this paper

Conversation Modeling with Neural Network
TLDR
This paper proposes a simple approach based on use of neural networks’ recently proposed sequence to sequence framework that generates reply by predicting sentence using chained probability for given sentence(s) in conversation.
A Neural Chatbot with Personality
TLDR
This paper built chatbots that imitate characters in popular TV shows: Barney from How I Met Your Mother, Sheldon from The Big Bang Theory, Michael from The Office, and Joey from Friends and designed an evaluation metric with a human judgment element, for which the chatbots performed well.
Keyword-Guided Neural Conversational Model
TLDR
This paper proposes a keyword-guided neural conversational model that can leverage external commonsense knowledge graphs (CKG) for both keyword transition and response retrieval and suggests that commonsense improves the performance of both next-turn keyword prediction and keyword-augmented response retrieval.
Generating Long and Diverse Responses with Neural Conversation Models
TLDR
This work presents new training and decoding methods that improve the quality, coherence, and diversity of long responses generated using sequence-to-sequence models, and introduces a stochastic beam-search algorithm with segment-by-segment reranking which lets us inject diversity earlier in the generation process.
Deep Learning Based Chatbot Models
TLDR
This work conducts an in-depth survey of recent literature, examining over 70 publications related to chatbots published in the last 3 years and makes the argument that the very nature of the general conversation domain demands approaches that are different from current state-of-the-art architectures.
A Neural Network Approach for Knowledge-Driven Response Generation
TLDR
An end-to-end learnable model that is able to generate context-sensitive and knowledge-driven responses by leveraging the alignment of two different data sources is presented.
Exploring Cells and Context Approaches for RNN Based Conversational Agents
TLDR
An introduction on how to create conversational agents based on the current state-of-the-art, using Recurrent Neural Networks (RNN), delve into di↵erent RNN architectures and compare the quality of the outputs from nine distinct agents.
A Neural Conversation Generation Model via Equivalent Shared Memory Investigation
TLDR
A novel reading and memory framework called Deep Reading Memory Network (DRMN) which is capable of remembering useful information of similar conversations for improving utterance generation is proposed and applied to two large-scale conversation datasets of justice and e-commerce fields.
Machine Reading Comprehension for Answer Re-Ranking in Customer Support Chatbots
TLDR
A method that adapts a deep neural architecture from the domain of machine reading comprehension to re-rank the suggested answers from different models using the question as a context and results show that this re-ranking framework can improve the performance in terms of word overlap and semantics both for individual models as well as for model combinations.
Towards Exploiting Background Knowledge for Building Conversation Systems
TLDR
This work creates a new dataset containing movie chats wherein each response is explicitly generated by copying and/or modifying sentences from unstructured background knowledge such as plots, comments and reviews about the movie.
...
...

References

SHOWING 1-10 OF 24 REFERENCES
Sequence to Sequence Learning with Neural Networks
TLDR
This paper presents a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure, and finds that reversing the order of the words in all source sentences improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier.
Show and tell: A neural image caption generator
TLDR
This paper presents a generative model based on a deep recurrent architecture that combines recent advances in computer vision and machine translation and that can be used to generate natural sentences describing an image.
Addressing the Rare Word Problem in Neural Machine Translation
TLDR
This paper proposes and implements an effective technique to address the problem of end-to-end neural machine translation's inability to correctly translate very rare words, and is the first to surpass the best result achieved on a WMT’14 contest task.
A Neural Probabilistic Language Model
TLDR
This work proposes to fight the curse of dimensionality by learning a distributed representation for words which allows each training sentence to inform the model about an exponential number of semantically neighboring sentences.
Neural Responding Machine for Short-Text Conversation
TLDR
Empirical study shows that NRM can generate grammatically correct and content-wise appropriate responses to over 75% of the input text, outperforming state-of-the-arts in the same setting, including retrieval-based and SMT-based models.
On Using Very Large Target Vocabulary for Neural Machine Translation
TLDR
It is shown that decoding can be efficiently done even with the model having a very large target vocabulary by selecting only a small subset of the whole target vocabulary.
Recurrent Continuous Translation Models
We introduce a class of probabilistic continuous translation models called Recurrent Continuous Translation Models that are purely based on continuous representations for words, phrases and sentences
Grammar as a Foreign Language
TLDR
The domain agnostic attention-enhanced sequence-to-sequence model achieves state-of-the-art results on the most widely used syntactic constituency parsing dataset, when trained on a large synthetic corpus that was annotated using existing parsers.
Neural Machine Translation by Jointly Learning to Align and Translate
TLDR
It is conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and it is proposed to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly.
A Neural Network Approach to Context-Sensitive Generation of Conversational Responses
TLDR
A neural network architecture is used to address sparsity issues that arise when integrating contextual information into classic statistical models, allowing the system to take into account previous dialog utterances.
...
...