Energy-based Unknown Intent Detection with Data Manipulation

  title={Energy-based Unknown Intent Detection with Data Manipulation},
  author={Yawen Ouyang and Jiasheng Ye and Yu Chen and Xinyu Dai and Shujian Huang and Jiajun Chen},
Unknown intent detection aims to identify the out-of-distribution (OOD) utterance whose intent has never appeared in the training set. In this paper, we propose using energy scores for this task as the energy score is theoretically aligned with the density of the input and can be derived from any classifier. However, highquality OOD utterances are required during the training stage in order to shape the energy gap between OOD and in-distribution (IND), and these utterances are difficult to… Expand

Figures and Tables from this paper


Deep Unknown Intent Detection with Margin Loss
This paper uses bidirectional long short-term memory network with the margin loss as the feature extractor, and feeds the feature vectors to the density-based novelty detection algorithm, local outlier factor (LOF), to detect unknown intents. Expand
Unknown Intent Detection Using Gaussian Mixture Model with an Application to Zero-shot Intent Classification
A semantic-enhanced Gaussian mixture model (SEG) for unknown intent detection is proposed, which model utterance embeddings with aGaussian mixture distribution and inject dynamic class semantic information into Gaussian means, which enables learning more class-concentratedembeddings that help to facilitate downstream outlier detection. Expand
Out-of-Domain Utterance Detection Using Classification Confidences of Multiple Topics
A novel OOD detection framework is proposed, which makes use of the classification confidence scores of multiple topics and applies a linear discriminant model to perform in-domain verification and introduces topic clustering which enables reliable topic confidence scores to be generated even for indistinct utterances. Expand
Out-of-Domain Detection for Natural Language Understanding in Dialog Systems
A novel model is proposed to generate high-quality pseudo OOD samples that are akin to IN-Domain (IND) input utterances and thereby improves the performance of OOD detection and is demonstrated to be effective in NLU. Expand
Likelihood Ratios and Generative Classifiers for Unsupervised Out-of-Domain Detection In Task Oriented Dialog
The ablations validate that specifically using likelihood ratios rather than plain likelihood is necessary to discriminate well between OOD and in-domain data and propose learning a generative classifier and computing a marginal likelihood (ratio) for OOD detection. Expand
An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction
A new dataset is introduced that includes queries that are out-of-scope—i.e., queries that do not fall into any of the system’s supported intents, posing a new challenge because models cannot assume that every query at inference time belongs to a system-supported intent class. Expand
Breaking the Closed World Assumption in Text Classification
A new learning strategy, called center-based similarity (CBS) space learning (or CBS learning), is proposed to provide a novel solution to the open world classification problem by reducing the open space risk while balancing the empirical risk. Expand
Detecting out-of-domain utterances addressed to a virtual personal assistant
S syntactic and semantic parse “structure” features are extracted in addition to lexical features to train a binary SVM classifier using a large number of random web search queries and VPA utterances from multiple domains and results indicate that such structured features result in higher precision especially when the test domain bears little resemblance to the existing domains. Expand
Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples
A novel training method for classifiers so that such inference algorithms can work better, and it is demonstrated its effectiveness using deep convolutional neural networks on various popular image datasets. Expand
Neural sentence embedding using only in-domain sentences for out-of-domain sentence detection in dialog systems
A novel neural sentence embedding method that represents sentences in a low-dimensional continuous vector space that emphasizes aspects that distinguish ID cases from OOD cases is proposed. Expand