Zero-shot User Intent Detection via Capsule Neural Networks

  title={Zero-shot User Intent Detection via Capsule Neural Networks},
  author={Congying Xia and Chenwei Zhang and Xiaohui Yan and Yi Chang and Philip S. Yu},
  booktitle={Conference on Empirical Methods in Natural Language Processing},
User intent detection plays a critical role in question-answering and dialog systems. [] Key Method We propose two capsule-based architectures: INTENT-CAPSNET that extracts semantic features from utterances and aggregates them to discriminate existing intents, and INTENTCAPSNET-ZSL which gives INTENTCAPSNET the zero-shot learning ability to discriminate emerging intents via knowledge transfer from existing intents. Experiments on two real-world datasets show that our model not only can better discriminate…

Figures and Tables from this paper

Zero-shot learning of user intent understanding by Convolutional Neural Networks

A zero-shot learning approach for intent understanding is presented, it can predict intents at runtime that did not exist at training time, and performs better to discriminate emerging intents when no labeled utterances are available.

Zero-Shot User Intent Detection via Augmented Conditional Variational Autoencoders

Experimental results show that the proposed cvae-based architecture for the user intent detection can not only discriminate existing intents, but also be able to discriminate emerging intents effectively when no labeled utterances exist.

Generalized Zero-shot Intent Detection via Commonsense Knowledge

This work proposes RIDE: an intent detection model that leverages commonsense knowledge in an unsupervised fashion to overcome the issue of training data scarcity and shows that RIDE outperforms the state-of-the-art models.

Learning Class-Transductive Intent Representations for Zero-shot Intent Detection

A novel framework that utilizes unseen class labels to learn Class-Transductive Intent Representations (CTIR) is proposed, which allows the model to predict unseen intents during training, with the corresponding label names serving as input utterances.

Reconstructing Capsule Networks for Zero-shot Intent Classification

This work proposes to reconstruct capsule networks for zero-shot intent classification by introducing a dimensional attention mechanism to fight against polysemy and reconstruct the transformation matrices for unseen intents by utilizing abundant latent information of the labeled utterances, which significantly improves the model generalization ability.

Learning Disentangled Intent Representations for Zero-shot Intent Detection

A class-transductive framework that utilizes unseen class labels to learn Disentangled Intent Representations (DIR), which allows the model to predict unseen intents in the training stage, with the corresponding label names serving as input utterances.

Unknown Intent Detection Using Multi-Objective Optimization on Deep Learning Classifiers

This paper proposes an effective post-processing method using multi-objective optimization to tune an existing neural network based intent classifier and make it capable of detecting unknown intents and yields significant improvements compared with the state-of-the-art methods for unknown intent detection.

Unknown Intent Detection Using Gaussian Mixture Model with an Application to Zero-shot Intent Classification

A semantic-enhanced Gaussian mixture model (SEG) for unknown intent detection is proposed, which model utterance embeddings with aGaussian mixture distribution and inject dynamic class semantic information into Gaussian means, which enables learning more class-concentratedembeddings that help to facilitate downstream outlier detection.

Template-based Approach to Zero-shot Intent Recognition

This paper treats the generalized zero-shot setup for intent recognition with a sentence pair modeling approach, and outperforms previous state-of-the-art f1-measure by up to 16% for unseen intents, using intent labels and user utterances and without accessing external sources.



Zero-shot learning of intent embeddings for expansion by convolutional deep structured semantic models

This paper focuses on the task of intent expansion, which helps remove the domain limit and make an intent schema flexible, and a con-volutional deep structured semantic model (CDSSM) is applied to jointly learn the representations for human intents and associated utterances.

Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling

This work proposes an attention-based neural network model for joint intent detection and slot filling, both of which are critical steps for many speech understanding and dialog systems.

Zero-Shot Learning Across Heterogeneous Overlapping Domains

This work presents a zero-shot learning approach for text classification, predicting which natural language understanding domain can handle a given utterance, and compares to generative baselines and shows that this approach requires less storage and performs better on new domains.

Understanding user's query intent with wikipedia

This paper proposes a general methodology to the problem of query intent classification that can achieve much better coverage to classify queries in an intent domain even through the number of seed intent examples is very small and can be easily applied to various intent domains.

Zero-Shot Learning Through Cross-Modal Transfer

This work introduces a model that can recognize objects in images even if no training data is available for the object class, and uses novelty detection methods to differentiate unseen classes from seen classes.

Zero-shot semantic parser for spoken language understanding

This work presents a novel zero-shot learning method, based on word embeddings, allowing to derive a full semantic parser for spoken language understanding, and shows that this model can reach instantly performance comparable to those obtained by either state-of-the-art carefully handcrafted rule-based or trained statistical models for extraction of dialog acts on the Dialog State Tracking test datasets.

Mining User Intentions from Medical Queries: A Neural Network Based Heterogeneous Jointly Modeling Approach

A neural network based jointly modeling approach to model and capture user intentions in medical related text queries and Sentence rephrasing is finally introduced as a data augmentation technique to improve model generalization ability during model training.

DeViSE: A Deep Visual-Semantic Embedding Model

This paper presents a new deep visual-semantic embedding model trained to identify visual objects using both labeled image data as well as semantic information gleaned from unannotated text and shows that the semantic information can be exploited to make predictions about tens of thousands of image labels not observed during training.

Online adaptative zero-shot learning spoken language understanding using word-embedding

This proposition can significantly improve the performance of the spoken language understanding module on the second Dialog State Tracking Challenge (DSTC2) datasets and an online adaptative strategy allowing to refine progressively the initial model with only a light and adjustable supervision is extended.

Attribute-Based Classification for Zero-Shot Visual Object Categorization

We study the problem of object recognition for categories for which we have no training examples, a task also called zero--data or zero-shot learning. This situation has hardly been studied in