Multi-Domain Joint Semantic Frame Parsing Using Bi-Directional RNN-LSTM

@inproceedings{HakkaniTr2016MultiDomainJS,
  title={Multi-Domain Joint Semantic Frame Parsing Using Bi-Directional RNN-LSTM},
  author={Dilek Z. Hakkani-T{\"u}r and G{\"o}khan T{\"u}r and Asli Celikyilmaz and Yun-Nung (Vivian) Chen and Jianfeng Gao and Li Deng and Ye-Yi Wang},
  booktitle={INTERSPEECH},
  year={2016}
}
Sequence-to-sequence deep learning has recently emerged as a new paradigm in supervised learning for spoken language understanding. However, most of the previous studies explored this framework for building single domain models for each task, such as slot filling or domain classification, comparing deep learning based approaches with conventional ones like conditional random fields. This paper proposes a holistic multi-domain, multi-task (i.e. slot filling, domain and intent detection) modeling… 

Figures and Tables from this paper

A Deep Learning Based Multi-task Ensemble Model for Intent Detection and Slot Filling in Spoken Language Understanding
TLDR
A deep learning based multi-task ensemble model that can perform both intent detection and slot filling tasks together and achieves better results than the individual models and the existing state-of-the-art systems is proposed.
A Deep Multi-task Model for Dialogue Act Classification, Intent Detection and Slot Filling
TLDR
Experiments on three datasets show that the proposed deep learning-based multi-task model performs better than the individual models as well as all the pipeline models and proves that the approach is better than existing methods.
A Bi-Model Based RNN Semantic Frame Parsing Model for Intent Detection and Slot Filling
TLDR
New Bi-model based RNN semantic frame parsing network structures are designed to perform the intent detection and slot filling tasks jointly, by considering their cross-impact to each other using two correlated bidirectional LSTMs (BLSTM).
Context Memory Networks for Multi-Objective Semantic Parsing in Conversational Understanding
TLDR
The Context Memory Network (CMN), a neural network architecture which specifically focuses on learning better representations as attention vectors from past memory to be reasoned with for the end task of jointly learning the intent class and slot tags, is introduced.
Deep Neural Architecture with Character Embedding for Semantic Frame Detection
TLDR
A deep neural network architecture that uses long-short term memory (RNN-LSTM) with character embedding for joint modeling of intent detection and slot filling and results show significant improvement in slot-filling and intent detection at the sentence level over state-of-the-art semantic frame detection methods.
A Joint Learning Framework With BERT for Spoken Language Understanding
TLDR
A novel encoder-decoder framework based multi-task learning model, which conducts joint training for intent classification and slot filling tasks and outperforms the state-of-the-art approaches.
Towards Zero-Shot Frame Semantic Parsing for Domain Scaling
TLDR
This paper proposes a deep learning based approach that can utilize only the slot description in context without the need for any labeled or unlabeled in-domain examples, to quickly bootstrap a new domain.
Joint Slot Filling and Intent Detection in Spoken Language Understanding by Hybrid CNN-LSTM Model
TLDR
A novel model is proposed that combines between convolutional neural networks, for their ability to detect complex features in the input sequences by applying filters to frames of these inputs, and recurrent neural networks taking in account the fact, that they can keep track of the long- and short- term dependencies in theinput sequences.
User Information Augmented Semantic Frame Parsing Using Progressive Neural Networks
TLDR
A novel progressive deep neural network model is designed to incorporate prior knowledge of user information intermediately to better and quickly train a semantic frame parser by utilizing the ubiquitous user information.
Graph LSTM with Context-Gated Mechanism for Spoken Language Understanding
TLDR
This paper proposes to tackle spoken language understanding with Graph LSTM, which first converts text into a graph and then utilizes the message passing mechanism to learn the node representation, and proposes a context-gated mechanism to make better use of context information for slot filling.
...
...

References

SHOWING 1-10 OF 41 REFERENCES
Joint semantic utterance classification and slot filling with recursive neural networks
TLDR
Recursive neural networks can be used to perform the core spoken language understanding (SLU) tasks in a spoken dialog system, more specifically domain and intent determination, concurrently with slot filling, in one jointly trained model.
Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding
TLDR
This paper implemented and compared several important RNN architectures, including Elman, Jordan, and hybrid variants, and implemented these networks with the publicly available Theano neural network toolkit and completed experiments on the well-known airline travel information system (ATIS) benchmark.
Sequence to Sequence Learning with Neural Networks
TLDR
This paper presents a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure, and finds that reversing the order of the words in all source sentences improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier.
Generative and discriminative algorithms for spoken language understanding
TLDR
Generative and discriminative approaches to modeling the sentence segmentation and concept labeling are studied and it is shown how non-local non-lexical features (e.g. a-priori knowledge) can be modeled with CRF which is the best performing algorithm across tasks.
Deep belief network based semantic taggers for spoken language understanding
TLDR
It is observed that when discrete features are projected onto a continuous space during neural network training, the model learns to cluster these features leading to its improved generalization capability, relative to a CRF model, especially in cases where some features are either missing or noisy.
Zero-shot learning of intent embeddings for expansion by convolutional deep structured semantic models
TLDR
This paper focuses on the task of intent expansion, which helps remove the domain limit and make an intent schema flexible, and a con-volutional deep structured semantic model (CDSSM) is applied to jointly learn the representations for human intents and associated utterances.
Spoken language understanding using long short-term memory neural networks
TLDR
This paper investigates using long short-term memory (LSTM) neural networks, which contain input, output and forgetting gates and are more advanced than simple RNN, for the word labeling task and proposes a regression model on top of the LSTM un-normalized scores to explicitly model output-label dependence.
Matrix Factorization with Knowledge Graph Propagation for Unsupervised Spoken Language Understanding
TLDR
A novel matrix factorization (MF) approach to learn latent feature vectors for utterances and semantic elements without the need of corpus annotations is introduced to learn semantic slots for a domain-specific SDS in an unsupervised fashion and carries out semantic parsing using latent MF techniques.
Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding
TLDR
The results show that on this task, both types of recurrent networks outperform the CRF baseline substantially, and a bi-directional Jordantype network that takes into account both past and future dependencies among slots works best, outperforming a CRFbased baseline by 14% in relative error reduction.
Recurrent neural network and LSTM models for lexical utterance classification
TLDR
This work proposes RNN and LSTM models for utterance intent classification and finds that RNNs work best when utterances are short, while LSTMs are best when uttered words are longer.
...
...