Unsupervised Parsing via Constituency Tests

  title={Unsupervised Parsing via Constituency Tests},
  author={Steven Cao and Nikita Kitaev and Dan Klein},
We propose a method for unsupervised parsing based on the linguistic notion of a constituency test. One type of constituency test involves modifying the sentence via some transformation (e.g. replacing the span with a pronoun) and then judging the result (e.g. checking if it is grammatical). Motivated by this idea, we design an unsupervised parser by specifying a set of transformations and using an unsupervised neural acceptability model to make grammaticality decisions. To produce a tree given… Expand

Figures and Tables from this paper

Focused Contrastive Training for Test-based Constituency Analysis
It is shown that consistent gains can be achieved if only certain positive instances are chosen for training, depending on whether they could be the result of a test transformation. Expand
Improved Latent Tree Induction with Distant Supervision via Span Constraints
This work presents a technique that uses distant supervision in the form of span constraints (i.e. phrase bracketing) to improve performance in unsupervised constituency parsing and demonstrates its effectiveness by parsing biomedical text from the CRAFT dataset. Expand
Learning Syntax from Naturally-Occurring Bracketings
Experiments demonstrate that the distantly-supervised models trained on naturally-occurring bracketing data are more accurate in inducing syntactic structures than competing unsupervised systems. Expand
Neural Bi-Lexicalized PCFG Induction
This paper proposes an approach to parameterize L-PCFGs without making implausible independence assumptions, which directly models bilexical dependencies and meanwhile reduces both learning and representation complexities of L PCFGs. Expand
PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols
A new parameterization form of PCFGs based on tensor decomposition is presented, which has at most quadratic computational complexity in the symbol number and therefore allows us to use a much larger number of symbols. Expand
Inducing Semantic Roles Without Syntax
It is shown it is possible to automatically induce semantic roles from QA-SRL, a scalable and ontology-free semantic annotation scheme that uses question-answer pairs to represent predicate-argument structure, and this method outperforms all previous models as well as a new state-of-the-art baseline over gold syntax. Expand
LM-Critic: Language Models for Unsupervised Grammatical Error Correction
This work shows how to leverage a pretrained language model (LM) in defining an LM-Critic, which judges a sentence to be grammatical if the LM assigns it a higher probability than its local perturbations. Expand


A Minimal Span-Based Neural Constituency Parser
This work presents a minimal neural model for constituency parsing based on independent scoring of labels and spans that is compatible with classical dynamic programming techniques, but also admits a novel greedy top-down inference algorithm based on recursive partitioning of the input. Expand
A Generative Constituent-Context Model for Improved Grammar Induction
A generative distributional model for the unsupervised induction of natural language syntax which explicitly models constituent yields and contexts is presented, giving the best published un-supervised parsing results on the ATIS corpus. Expand
Constituency Parsing with a Self-Attentive Encoder
It is demonstrated that replacing an LSTM encoder with a self-attentive architecture can lead to improvements to a state-of-the-art discriminative constituency parser, and it is found that separating positional and content information in the encoder canlead to improved parsing accuracy. Expand
Profiting from Mark-Up: Hyper-Text Annotations for Guided Parsing
It is demonstrated that derived constraints aid grammar induction by training Klein and Manning's Dependency Model with Valence (DMV) on this data set: parsing accuracy on Section 23 (all sentences) of the Wall Street Journal corpus jumps to 50.4%, beating previous state-of-the-art by more than 5%. Expand
Neural Network Acceptability Judgments
This paper introduces the Corpus of Linguistic Acceptability (CoLA), a set of 10,657 English sentences labeled as grammatical or ungrammatical from published linguistics literature, and trains several recurrent neural network models on acceptability classification, and finds that the authors' models outperform unsupervised models by Lau et al. (2016) on CoLA. Expand
Two Local Models for Neural Constituent Parsing
Two conceptually simple local neural models for constituent parsing are investigated, which make local decisions to constituent spans and CFG rules, respectively, which give highly competitive results. Expand
Grammaticality, Acceptability, and Probability: A Probabilistic View of Linguistic Knowledge
It is argued that the results of a set of large-scale experiments using crowd-sourced acceptability judgments that demonstrate gradience to be a pervasive feature inacceptability judgments support the view that linguistic knowledge can be intrinsically probabilistic. Expand
Are Pre-trained Language Models Aware of Phrases? Simple but Strong Baselines for Grammar Induction
The proposed method provides an effective way of extracting constituency trees from the pre-trained LMs without training, and reports intriguing findings in the induced trees, including the fact that pre- trained LMs outperform other approaches in correctly demarcating adverb phrases in sentences. Expand
Neural Language Modeling by Jointly Learning Syntax and Lexicon
A novel neural language model, called the Parsing-Reading-Predict Networks (PRPN), that can simultaneously induce the syntactic structure from unannotated sentences and leverage the inferred structure to learn a better language model is proposed. Expand
Unsupervised Recurrent Neural Network Grammars
An inference network parameterized as a neural CRF constituency parser is developed to maximize the evidence lower bound and apply amortized variational inference to unsupervised learning of RNNGs. Expand