Supertagging the Long Tail with Tree-Structured Decoding of Complex Categories

  title={Supertagging the Long Tail with Tree-Structured Decoding of Complex Categories},
  author={Jakob Prange and Nathan Schneider and Vivek Srikumar},
  journal={Transactions of the Association for Computational Linguistics},
Abstract Although current CCG supertaggers achieve high accuracy on the standard WSJ test set, few systems make use of the categories’ internal structure that will drive the syntactic derivation during parsing. The tagset is traditionally truncated, discarding the many rare and complex category types in the long tail. However, supertags are themselves trees. Rather than give up on rare tags, we investigate constructive models that account for their internal structure, including novel methods… 
Geometry-Aware Supertagging with Heterogeneous Dynamic Convolutions
This work revisits constructive supertagging from a graph-theoretic perspective, and proposes a framework based on heterogeneous dynamic graph convolutions, aimed at exploiting the distinctive structure of a supertagger’s output space.
Morphology Without Borders: Clause-Level Morphological Annotation
Morphological tasks use large multi-lingual datasets that organize words into inflection tables, which then serve as training and evaluation data for various tasks. However, a closer inspection of
A New Representation for Span-based CCG Parsing
The proposed representation decomposes CCG derivations into several independent pieces and prevents the span-based parsing models from violating the schemata.
CCG Supertagging as Top-down Tree Generation
Combinatory Categorial Grammar (CCG; Steedman, 2000) is a strongly-lexicalized grammar formalism, whose syntax-semantics interface has been attractive for downstream tasks such as semantic parsing
Generating CCG Categories
This work proposes to generate categories rather than classify them: each category is decomposed into a sequence of smaller atomic tags, and the tagger aims to generate the correct sequence.
Making Heads and Tails of Models with Marginal Calibration for Sparse Tagsets
This work addresses the open problem of calibration for tagging models with sparse tagsets, and proposes tag frequency grouping (TFG) as a way to measure calibration error in different frequency bands.
Oracle Linguistic Graphs Complement a Pretrained Transformer Language Model: A Cross-formalism Comparison
It is found that, overall, semantic constituency structures are most useful to language modeling performance—outpacing syntactic constituency structures as well as syntactic and semantic dependency structures.
Proof Net Structure for Neural Lambek Categorial Parsing
This paper presents the first statistical parser for Lambek categorial grammar (LCG), a grammatical formalism for which the graphical proof method known as *proof nets* is applicable, and derives novel loss functions by expressing proof net constraints as differentiable functions of the model output.
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey
A survey of recent work that uses large, pre-trained transformer-based language models to solve NLP tasks via pre-training then fine-tuning, prompting, or text generation approaches.


pertagging Combinatory Categorial Grammar with Attentive Graph Convolutional Networks
  • 2020
The Java Version of the C & C Parser Version 0 . 95
The first Java version of the C&C parser was created by Stephen Clark, based heavily on the C++ implementation of C &C, and this new version is based on that version, with some changes.
Supertagging for Combinatory Categorial Grammar
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages
This work introduces Stanza, an open-source Python natural language processing toolkit supporting 66 human languages that features a language-agnostic fully neural pipeline for text analysis, including tokenization, multi-word token expansion, lemmatization, part-of-speech and morphological feature tagging, dependency parsing, and named entity recognition.
Supertagging Combinatory Categorial Grammar with Attentive Graph Convolutional Networks
Attentive graph convolutional networks are proposed to enhance neural CCG supertagging through a novel solution of leveraging contextual information.
Supertagging with CCG primitives
An L STM-based model is presented that replaces standard word-level classification with prediction of a sequence of primitives, similarly to LSTM decoders, and obtains state-of-the-art word accuracy for single-task English CCG supertagging, increases parser coverage and F1, and is able to produce novel categories.
pertagging Combinatory Categorial Grammar with Attentive Graph Convolutional Networks
  • 2020
A Logic-Driven Framework for Consistency of Neural Models
This paper proposes a learning framework for constraining models using logic rules to regularize them away from inconsistency, and instantiate it on natural language inference, where experiments show that enforcing invariants stated in logic can help make the predictions of neural models both accurate and consistent.
A Structural Probe for Finding Syntax in Word Representations
A structural probe is proposed, which evaluates whether syntax trees are embedded in a linear transformation of a neural network’s word representation space, and shows that such transformations exist for both ELMo and BERT but not in baselines, providing evidence that entire syntax Trees are embedded implicitly in deep models’ vector geometry.
Augmenting Neural Networks with First-order Logic
This paper presents a novel framework for introducing declarative knowledge to neural network architectures in order to guide training and prediction, and shows that knowledge-augmented networks can strongly improve over baselines, especially in low-data regimes.