• Corpus ID: 24462966

Learning to Compose Task-Specific Tree Structures

@inproceedings{Choi2018LearningTC,
  title={Learning to Compose Task-Specific Tree Structures},
  author={Jihun Choi and Kang Min Yoo and Sang-goo Lee},
  booktitle={AAAI},
  year={2018}
}
For years, recursive neural networks (RvNNs) have been shown to be suitable for representing text into fixed-length vectors and achieved good performance on several natural language processing tasks. [] Key Method Our model uses Straight-Through Gumbel-Softmax estimator to decide the parent node among candidates dynamically and to calculate gradients of the discrete decision. We evaluate the proposed model on natural language inference and sentiment analysis, and show that our model outperforms or is at…

Figures and Tables from this paper

Modeling Hierarchical Structures with Continuous Recursive Neural Networks
TLDR
This work proposes Continuous Recursive Neural Network (CRvNN) as a backpropagation-friendly alternative to address the aforementioned limitations of traditional RvNNs by incorporating a continuous relaxation to the induced structure.
Dynamic Compositionality in Recursive Neural Networks with Structure-aware Tag Representations
TLDR
This work presents a novel RvNN architecture that can provide dynamic compositionality by considering comprehensive syntactic information derived from both the structure and linguistic tags and introduces a structure-aware tag representation constructed by a separate tag-level tree-LSTM.
Jointly learning sentence embeddings and syntax with unsupervised Tree-LSTMs
TLDR
This paper introduces a model based on the CKY chart parser, and evaluates its downstream performance on a natural language inference task and a reverse dictionary task, and finds that its approach is competitive against similar models of comparable size and outperforms Tree-LSTMs that use trees produced by a parser.
Modularized Syntactic Neural Networks for Sentence Classification
TLDR
Experimental results on four benchmark datasets show that the proposed Modularized Syntactic Neural Network significantly outperforms previous state-of-the-art tree-based methods on the sentence classification task.
Learning to Embed Sentences Using Attentive Recursive Trees
TLDR
An end-to-end reinforced training strategy for AR-Tree is proposed, which is demonstrated to consistently outperform, or be at least comparable to, the state-of-the-art sentence embedding methods on three sentence understanding tasks.
FastTrees: Parallel Latent Tree-Induction for Faster Sequence Encoding
TLDR
It is shown that the FASTTREES module can be applied to enhance Transformer models, achieving performance gains on three sequence transduction tasks (machine translation, subject-verb agreement and mathematical language understanding), paving the way for modular tree-induction modules.
Latent Tree Learning with Ordered Neurons: What Parses Does It Produce?
TLDR
ON-LSTM has reasonably consistent parsing behaviors across different restarts, but the model struggles with the internal structures of complex noun phrases and has a tendency to overestimate the height of the split points right before verbs.
Ordered Memory
TLDR
A new attention-based mechanism is introduced and use its cumulative probability to control the writing and erasing operation of the memory and a new Gated Recursive Cell is introduced to compose lower-level representations into higher-level representation.
On Tree-Based Neural Sentence Modeling
TLDR
Replacing syntactic parsing trees with trivial trees in the encoders indicates that explicit syntax guidance may not be the main contributor to the superior performances of tree-based neural sentence modeling and shows that tree modeling gives better results when crucial words are closer to the final representation.
ListOps: A Diagnostic Dataset for Latent Tree Learning
TLDR
It is shown that the current leading latent tree models are unable to learn to parse and succeed at ListOps, a toy dataset created to study the parsing ability of latentTree models.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 58 REFERENCES
Jointly learning sentence embeddings and syntax with unsupervised Tree-LSTMs
TLDR
This paper introduces a model based on the CKY chart parser, and evaluates its downstream performance on a natural language inference task and a reverse dictionary task, and finds that its approach is competitive against similar models of comparable size and outperforms Tree-LSTMs that use trees produced by a parser.
Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks
TLDR
The Tree-LSTM is introduced, a generalization of LSTMs to tree-structured network topologies that outperform all existing systems and strong LSTM baselines on two tasks: predicting the semantic relatedness of two sentences and sentiment classification.
Deep Recursive Neural Networks for Compositionality in Language
TLDR
The results show that deep RNNs outperform associated shallow counterparts that employ the same number of parameters and outperforms previous baselines on the sentiment analysis task, including a multiplicative RNN variant as well as the recently introduced paragraph vectors.
Learning to Compose Words into Sentences with Reinforcement Learning
TLDR
Reinforcement learning is used to learn tree-structured neural networks for computing representations of natural language sentences and it is shown that while they discover some linguistically intuitive structures, they are different than conventional English syntactic structures.
Neural Tree Indexers for Text Understanding
TLDR
A robust syntactic parsing-independent tree structured model, Neural Tree Indexers (NTI) is introduced that provides a middle ground between the sequential RNNs and the syntactic treebased recursive models, outperforming state-of-the-art recurrent and recursive neural networks.
A Convolutional Neural Network for Modelling Sentences
TLDR
A convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) is described that is adopted for the semantic modelling of sentences and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations.
Long Short-Term Memory Over Recursive Structures
TLDR
This paper proposes to extend chain-structured long short-term memory to tree structures, in which a memory cell can reflect the history memories of multiple child cells or multiple descendant cells in a recursive process, and calls the model S-LSTM, which provides a principled way of considering long-distance interaction over hierarchies.
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
TLDR
A Sentiment Treebank that includes fine grained sentiment labels for 215,154 phrases in the parse trees of 11,855 sentences and presents new challenges for sentiment compositionality, and introduces the Recursive Neural Tensor Network.
Recurrent Neural Network-Based Sentence Encoder with Gated Attention for Natural Language Inference
TLDR
This paper describes a model (alpha) that is ranked among the top in the Shared Task, on both the in- domain test set and on the cross-domain test set, demonstrating that the model generalizes well to theCross-domain data.
Sentence Modeling with Gated Recursive Neural Network
TLDR
A gated recursive neural network (GRNN) to model sentences, which employs a full binary tree (FBT) structure to control the combinations in recursive structure by introducing two kinds of gates.
...
1
2
3
4
5
...