• Corpus ID: 5693201

Bidirectional Recursive Neural Networks for Token-Level Labeling with Structure

@article{Irsoy2013BidirectionalRN,
  title={Bidirectional Recursive Neural Networks for Token-Level Labeling with Structure},
  author={Ozan Irsoy and Claire Cardie},
  journal={ArXiv},
  year={2013},
  volume={abs/1312.0493}
}
Recently, deep architectures, such as recurrent and recursive neural networks have been successfully applied to various natural language processing tasks. Inspired by bidirectional recurrent neural networks which use representations that summarize the past and future around an instance, we propose a novel architecture that aims to capture the structural information around an input, and use it to label instances. We apply our method to the task of opinion expression extraction, where we employ… 
View PDF on arXiv
27 Citations
Highly Influential Citations
1
Background Citations
12
Methods Citations
9
Results Citations
2

Figures and Tables from this paper

27 Citations

Citation Type

Citation Type

When Are Tree Structures Necessary for Deep Learning of Representations?
TLDR
This paper benchmarks recursive neural models against sequential recurrent neural models, enforcing applesto-apples comparison as much as possible, and introduces a method for allowing recurrent models to achieve similar performance: breaking long sentences into clause-like units at punctuation and processing them separately before combining.
Learning vector representations for sentences: The recursive deep learning approach
TLDR
This dissertation aims at extending the RNN model by allowing information to flow in a parse tree not only bottom-up but also top-down such that both the content and context of a constituent can be recursively encoded in vectors.
Predictive Property of Hidden Representations in Recurrent Neural Network Language Models
TLDR
This study experimentally discovers that, even for an unmodified RNNLM, the farther sequences can also be predicted given the activation of the hidden neurons.
Inside-Outside Semantics : A Framework for Neural Models of Semantic Composition
TLDR
This paper argues that the RNN model is restricted to a subset of NLP tasks where semantic compositionality plays a role, and proposes an extension called Inside-Outside Semantics, which performs on par with or better than the state-of-the-art (neural) models in word prediction, phrase-similarity judgements and semantic role labelling.
Structural Attention Neural Networks for improved sentiment analysis
TLDR
A tree-structured attention neural network for sentences and small phrases is introduced and applied to the problem of sentiment classification by incorporating structural information around a node of a syntactic tree using both bottom-up and top-down information propagation.
Global Belief Recursive Neural Networks
TLDR
Global belief recursive neural networks (GB-RNNs) are introduced which are based on the idea of extending purely feedforward neural networks to include one feedbackward step during inference, which allows phrase level predictions and representations to give feedback to words.
Improving Aspect Term Extraction With Bidirectional Dependency Tree Representation
TLDR
A novel bidirectional dependency tree network is proposed to extract dependency structure features from the given sentences to solve the aspect term extraction problem and outperforms state-of-the-art baseline models on four benchmark SemEval datasets.
Gated Neural Networks for Targeted Sentiment Analysis
TLDR
A sentence-level neural model is proposed to address the limitation of pooling functions, which do not explicitly model tweet-level semantics and gives significantly higher accuracies compared to the current best method for targeted sentiment analysis.
Feature Weight Tuning for Recursive Neural Networks
This paper addresses how a recursive neural network model can automatically leave out useless information and emphasize important evidence, in other words, to perform "weight tuning" for higher-level…
Combine HowNet lexicon to train phrase recursive autoencoder for sentence-level sentiment analysis
...
1
2
3
...

References

SHOWING 1-10 OF 22 REFERENCES
Natural Language Processing (Almost) from Scratch
We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including part-of-speech tagging, chunking, named entity…
A unified architecture for natural language processing: deep neural networks with multitask learning
We describe a single convolutional neural network architecture that, given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, semantic…
Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding
TLDR
The results show that on this task, both types of recurrent networks outperform the CRF baseline substantially, and a bi-directional Jordantype network that takes into account both past and future dependencies among slots works best, outperforming a CRFbased baseline by 14% in relative error reduction.
Syntactic and Semantic Structure for Opinion Expression Detection
We demonstrate that relational features derived from dependency-syntactic and semantic role structures are useful for the task of detecting opinionated expressions in natural-language text,…
Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection
TLDR
This work introduces a method for paraphrase detection based on recursive autoencoders (RAE) and unsupervised RAEs based on a novel unfolding objective and learns feature vectors for phrases in syntactic trees to measure word- and phrase-wise similarity between two sentences.
Word Representations: A Simple and General Method for Semi-Supervised Learning
TLDR
This work evaluates Brown clusters, Collobert and Weston (2008) embeddings, and HLBL (Mnih & Hinton, 2009) embeds of words on both NER and chunking, and finds that each of the three word representations improves the accuracy of these baselines.
Parsing Natural Scenes and Natural Language with Recursive Neural Networks
TLDR
A max-margin structure prediction architecture based on recursive neural networks that can successfully recover such structure both in complex scene images as well as sentences is introduced.
Learning task-dependent distributed representations by backpropagation through structure
  • C. Goller, A. Küchler
  • Computer Science
    Proceedings of International Conference on Neural Networks (ICNN'96)
  • 1996
TLDR
A connectionist architecture together with a novel supervised learning scheme which is capable of solving inductive inference tasks on complex symbolic structures of arbitrary size is presented.
Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions
TLDR
A novel machine learning framework based on recursive autoencoders for sentence-level prediction of sentiment label distributions that outperform other state-of-the-art approaches on commonly used datasets, without using any pre-defined sentiment lexica or polarity shifting rules.
Efficient Estimation of Word Representations in Vector Space
TLDR
Two novel model architectures for computing continuous vector representations of words from very large data sets are proposed and it is shown that these vectors provide state-of-the-art performance on the authors' test set for measuring syntactic and semantic word similarities.
...
1
2
3
...