Towards Incremental Parsing of Natural Language Using Recursive Neural Networks

  title={Towards Incremental Parsing of Natural Language Using Recursive Neural Networks},
  author={Fabrizio Costa and Paolo Frasconi and Vincenzo Lombardo and Giovanni Soda},
  journal={Applied Intelligence},
In this paper we develop novel algorithmic ideas for building a natural language parser grounded upon the hypothesis of incrementality. Although widely accepted and experimentally supported under a cognitive perspective as a model of the human parser, the incrementality assumption has never been exploited for building automatic parsers of unconstrained real texts. The essentials of the hypothesis are that words are processed in a left-to-right fashion, and the syntactic structure is kept… 

Ambiguity resolution in incremental parsing of natural language

A systematic analysis of the behavior of the network allows to gain important insights about the kind of information that is exploited to resolve different forms of ambiguity, and it is found that learning from examples allows to predict the location of the attachment point with high accuracy, while the discrimination amongst alternative syntactic structures with the same anchor point is slightly better than making a decision purely based on frequencies.

Ambiguity resolution analysis in incremental parsing of natural language

A systematic analysis of the behavior of the network allows for important insights about the kind of information that is exploited to resolve different forms of ambiguity in incremental parsing.

Enhancing First-Pass Attachment Prediction

This paper explores the convergence between cognitive modeling and engineering solutions to the parsing problem in NLP, and analyzes and refine a connectionist paradigm capable of processing acyclic graphs to perform supervised learning on syntactic trees extracted from a large corpus of parsed sentences.

Strongly Incremental Constituency Parsing with Graph Neural Networks

This paper proposes a novel transition system called attach-juxtapose, which represents a partial sentence using a single tree; each action adds exactly one token into the partial tree, and develops a strongly incremental parser.

Parsing with Compositional Vector Grammars

A Compositional Vector Grammar (CVG), which combines PCFGs with a syntactically untied recursive neural network that learns syntactico-semantic, compositional vector representations and improves performance on the types of ambiguities that require semantic information such as PP attachments.

Transition-based Neural Constituent Parsing

This work proposes a neural network structure that explicitly models the unbounded history of actions performed on the stack and queue employed in transition-based parsing, in addition to the representations of partially parsed tree structure.

What is Needed for a Robot to Acquire Grammar? Some Underlying Primitive Mechanisms for the Synthesis of Linguistic Ability

The authors can deconstruct grammar to derive underlying primitive mechanisms, including serial processing, segmentation, categorization, compositionality, and forward planning that are necessary preparatory steps to reconstruct a working syntactic/semantic/pragmatic processor which can handle real language.

Proposal Encoding and Decoding Graph Representations of Natural Language

This thesis develops novel algorithms and models for decoding—automatically extracting graphs from raw text; and encoding—transforming graphs into tensors so that they can be used as input to a downstream neural network.

Improvements to Training an RNN parser

This work examines an RNN model by comparing it with an abstract generative probabilistic model using a Deep Belief Network (DBN), and develops two methods which are able to improve the parsing F-Score by 0.83% on longer sentences.



Two Principles of Parse Preference

A compendium of many of the heuristics devised for choosing the preferred parses in the DIALOGIC system is presented and two principles that seem to underlie them are proposed.

Monotonic Syntactic Processing : A Cross-linguistic Study of Attachment and Reanalysis

It is demonstrated how the definition of two simple parsing operations, simple attachment and tree lowering, which are related to the grammatical composition operations of substitution and adjunction in the Tree Adjoining Grammar formalism, yields a parser which is more constrained than previous Description Theory based models.

Coping with Ambiguity and Unknown Words through Probabilistic Models

A new natural language system (PLUM) is constructed for extracting data from text, e.g., newswire text, based on results of experiments in predicting parts of speech of highly ambiguous words, predicting the intended interpretation of an utterance when more than one interpretation satisfies all known syntactic and semantic constraints.

A New Statistical Parser Based on Bigram Lexical Dependencies

A new statistical parser which is based on probabilities of dependencies between head-words in the parse tree, which trains on 40,000 sentences in under 15 minutes and can be improved to over 200 sentences a minute with negligible loss in accuracy.

Learning Parse and Translation Decisions from Examples with Rich Context

A knowledge and context-based system for parsing and translating natural language and evaluate it on sentences from the Wall Street Journal, which relies heavily on context, as encoded in features which describe the morphological, syntactic, semantic and other aspects of a given parse state.

Supertagging: An Approach to Almost Parsing

Novel methods for robust parsing that integrate the flexibility of linguistically motivated lexical descriptions with the robustness of statistical techniques are proposed.

Building a Large Annotated Corpus of English: The Penn Treebank

As a result of this grant, the researchers have now published on CDROM a corpus of over 4 million words of running text annotated with part-of- speech (POS) tags, which includes a fully hand-parsed version of the classic Brown corpus.

Recursive Distributed Representations

Reanalysis in sentence processing

This chapter discusses prosodic Influences on Reading Syntactically Ambiguous Sentences and Parsing as Incremental Restructuring, and generalized Monotonicity for Reanalysis Models.