Improving Neural Parsing by Disentangling Model Combination and Reranking Effects

@article{Fried2017ImprovingNP,
  title={Improving Neural Parsing by Disentangling Model Combination and Reranking Effects},
  author={Daniel Fried and Mitchell Stern and Dan Klein},
  journal={ArXiv},
  year={2017},
  volume={abs/1707.03058}
}
Recent work has proposed several generative neural models for constituency parsing that achieve state-of-the-art results. Since direct search in these generative models is difficult, they have primarily been used to rescore candidate outputs from base parsers in which decoding is more straightforward. We first present an algorithm for direct search in these generative models. We then demonstrate that the rescoring results are at least partly due to implicit model combination rather than… Expand
Effective Inference for Generative Neural Parsing
TLDR
An alternative to the conventional action-level beam search used for discriminative neural models that enables us to decode directly in these generative models is described and it is shown that by improving the basic candidate selection strategy and using a coarse pruning function, one can improve accuracy while exploring significantly less of the search space. Expand
Two Local Models for Neural Constituent Parsing
TLDR
Two conceptually simple local neural models for constituent parsing are investigated, which make local decisions to constituent spans and CFG rules, respectively, which give highly competitive results. Expand
Comparing Top-Down and Bottom-Up Neural Generative Dependency Models
TLDR
While both generative models improve parsing performance over a discriminative baseline, they are significantly less effective than non-syntactic LSTM language models. Expand
Constituency Parsing with a Self-Attentive Encoder
TLDR
It is demonstrated that replacing an LSTM encoder with a self-attentive architecture can lead to improvements to a state-of-the-art discriminative constituency parser, and it is found that separating positional and content information in the encoder canlead to improved parsing accuracy. Expand
Scalable Syntax-Aware Language Models Using Knowledge Distillation
TLDR
An efficient knowledge distillation (KD) technique is introduced that transfers knowledge from a syntactic language model trained on a small corpus to an LSTM language model, hence enabling the L STM to develop a more structurally sensitive representation of the larger training data it learns from. Expand
Neural Generative Rhetorical Structure Parsing
TLDR
This paper presents the first generative model for RST parsing, a document-level RNN grammar (RNNG) with a bottom-up traversal order and develops a novel beam search algorithm that keeps track of both structure-and word-generating actions without exhibit-ing this branching bias. Expand
Sequential Parsing with In-order Tree Traversals
  • 2018
We apply in-order tree traversals to the Seq2Seq parser (Vinyals et al., 2015), simplifying symbol prediction. Our simple greedy parser achieves 91.4 F1, which is the best among greedy parsers in theExpand
Multilingual Constituency Parsing with Self-Attention and Pre-Training
TLDR
It is shown that constituency parsing benefits from unsupervised pre-training across a variety of languages and a range of pre- training conditions, and the idea of joint fine-tuning is explored and shows that it gives low-resource languages a way to benefit from the larger datasets of other languages. Expand
A Span-based Linearization for Constituent Trees
TLDR
This work proposes a novel linearization of a constituent tree that is tied on the spans directly and considers more local features when performing span prediction, which is more interpretable and effective. Expand
Max-Margin Incremental CCG Parsing
TLDR
An improved version of beam search optimisation that minimises all beam search violations instead of minimising only the biggest violation, which gives better results than all previously published incremental CCG parsers, and outperforms even some widely used non-incremental CCG Parsers. Expand
...
1
2
3
4
...

References

SHOWING 1-10 OF 19 REFERENCES
Span-Based Constituency Parsing with a Structure-Label System and Provably Optimal Dynamic Oracles
TLDR
A new shift-reduce system whose stack contains merely sentence spans, represented by a bare minimum of LSTM features, which is the first provably optimal dynamic oracle for constituency parsing, which runs in amortized O(1) time, compared to O(n^3) oracles for standard dependency parsing. Expand
Shift-Reduce Constituent Parsing with Neural Lookahead Features
TLDR
A bidirectional LSTM model is built, which leverages full sentence information to predict the hierarchy of constituents that each word starts and ends and gives the highest reported accuracies for fully-supervised parsing. Expand
Efficient Stacked Dependency Parsing by Forest Reranking
TLDR
A discriminative forest reranking algorithm for dependency parsing that can be seen as a form of efficient stacked parsing and a novel shift-reduce parser that eliminates the spurious ambiguity of arc-standard transition systems. Expand
Generative Incremental Dependency Parsing with Neural Networks
TLDR
A neural network model for scalable generative transition-based dependency parsing that surpasses the accuracy and speed of previous generative dependency parsers and shows a strong improvement over n-gram language models, opening the way to the efficient integration of syntax into neural models for language generation. Expand
Inducing History Representations for Broad Coverage Statistical Parsing
TLDR
A neural network method for inducing representations of parse histories and using these history representations to estimate the probabilities needed by a statistical left-corner parser achieves performance on the Penn Treebank which is only 0.6% below the best current parser for this task. Expand
Grammar as a Foreign Language
TLDR
The domain agnostic attention-enhanced sequence-to-sequence model achieves state-of-the-art results on the most widely used syntactic constituency parsing dataset, when trained on a large synthetic corpus that was annotated using existing parsers. Expand
Parsing as Language Modeling
We recast syntactic parsing as a language modeling problem and use recent advances in neural network language modeling to achieve a new state of the art for constituency Penn Treebank parsing — 93.8Expand
Products of Random Latent Variable Grammars
TLDR
This work combines multiple automatically learned grammars into an unweighted product model, which gives significantly improved performance over state-of-the-art individual Grammars. Expand
Two/Too Simple Adaptations of Word2Vec for Syntax Problems
We present two simple modifications to the models in the popular Word2Vec tool, in order to generate embeddings more suited to tasks involving syntax. The main issue with the original models is theExpand
A Latent Variable Model for Generative Dependency Parsing
TLDR
This work proposes a generative dependency parsing model which uses binary latent variables to induce conditioning features and uses a recently proposed class of Bayesian Networks for structured prediction, Incremental Sigmoid Belief Networks to define this model. Expand
...
1
2
...