An Effective Non-Autoregressive Model for Spoken Language Understanding

@article{Cheng2021AnEN,
  title={An Effective Non-Autoregressive Model for Spoken Language Understanding},
  author={Lizhi Cheng and Weijia Jia and Wenmian Yang},
  journal={Proceedings of the 30th ACM International Conference on Information \& Knowledge Management},
  year={2021}
}
  • Lizhi Cheng, Weijia Jia, Wenmian Yang
  • Published 16 August 2021
  • Computer Science
  • Proceedings of the 30th ACM International Conference on Information & Knowledge Management
Spoken Language Understanding (SLU), a core component of the task-oriented dialogue system, expects a shorter inference latency due to the impatience of humans. Non-autoregressive SLU models clearly increase the inference speed but suffer uncoordinated-slot problems caused by the lack of sequential dependency information among each slot chunk. To gap this shortcoming, in this paper, we propose a novel non-autoregressive SLU model named Layered-Refine Transformer, which contains a Slot Label… 

Capture Salient Historical Information: A Fast and Accurate Non-Autoregressive Model for Multi-turn Spoken Language Understanding

TLDR
A novel model for multi-turn SLU named Salient History Attention with Layer-Refined Transformer (SHA-LRT) is proposed, which composes of an SHA module, a Layer- refined Mechanism (LRM), and a Slot Label Generation (SLG) task.

A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond

TLDR
This survey conducts a systematic survey with comparisons and discussions of various non-autoregressive translation (NAT) models from different aspects, and categorizes the efforts of NAT into several groups, including data manipulation, modeling methods, training criterion, decoding algorithms, and the benefit from pre-trained models.

References

SHOWING 1-10 OF 34 REFERENCES

Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding

TLDR
This paper implemented and compared several important RNN architectures, including Elman, Jordan, and hybrid variants, and implemented these networks with the publicly available Theano neural network toolkit and completed experiments on the well-known airline travel information system (ATIS) benchmark.

A Result Based Portable Framework for Spoken Language Understanding

TLDR
A novel Result-based Portable Framework for SLU (RPFSLU) is proposed, which allows most existing single-turn SLU models to obtain the contextual information from multi-turn dialogues and takes full advantage of predicted results in the dialogue history during the current prediction.

A Joint Model of Intent Determination and Slot Filling for Spoken Language Understanding

TLDR
A joint model is proposed based on the idea that the intent and semantic slots of a sentence are correlative, and it outperforms the state-of-the-art approaches on both tasks.

Syntax or semantics? knowledge-guided joint semantic frame parsing

TLDR
This paper proposes to apply knowledge-guided structural attention networks (K-SAN), which additionally incorporate non-flat network topologies guided by prior knowledge, to a language understanding task, and shows that the proposed K-SAN models with syntax or semantics outperform the state-of-the-art neural network based results.

Memory Consolidation for Contextual Spoken Language Understanding with Dialogue Logistic Inference

TLDR
A new dialogue logistic inference (DLI) task to consolidate the context memory jointly with SLU in the multi-task framework and improvements are quite impressive, especially in slot filling.

A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding

TLDR
A novel framework for SLU to better incorporate the intent information, which further guiding the slot filling is proposed, which achieves the state-of-the-art performance and outperforms other previous methods by a large margin.

Multi-Domain Joint Semantic Frame Parsing Using Bi-Directional RNN-LSTM

TLDR
Experimental results show the power of a holistic multi-domain, multi-task modeling approach to estimate complete semantic frames for all user utterances addressed to a conversational system over alternative methods based on single domain/task deep learning.

SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Filling

TLDR
In-depth analyses show that 1) pretraining schemes could further enhance the model; 2) two-pass mechanism indeed remedy the uncoordinated slots problem caused by conditional independence of non-autoregressive model.

CM-Net: A Novel Collaborative Memory Network for Spoken Language Understanding

TLDR
A novel Collaborative Memory Network (CM-Net) based on the well-designed block, named CM-block, which achieves the state-of-the-art results on the ATIS and SNIPS in most of criteria, and significantly outperforms the baseline models on the CAIS.

A Study of Non-autoregressive Model for Sequence Generation

TLDR
An analysis model called CoMMA is proposed to characterize the difficulty of different NAR sequence generation tasks and has several interesting findings, including that among the NMT, ASR and TTS tasks,ASR has the most target-token dependency while TTS has the least.