Verb Argument Structure Alternations in Word and Sentence Embeddings

  title={Verb Argument Structure Alternations in Word and Sentence Embeddings},
  author={Katharina Kann and Alex Warstadt and Adina Williams and Samuel R. Bowman},
Verbs occur in different syntactic environments, or frames. We investigate whether artificial neural networks encode grammatical distinctions necessary for inferring the idiosyncratic frame-selectional properties of verbs. We introduce five datasets, collectively called FAVA, containing in aggregate nearly 10k sentences labeled for grammatical acceptability, illustrating different verbal argument structure alternations. We then test whether models can distinguish acceptable English verb-frame… 

Figures and Tables from this paper

Investigating Representations of Verb Bias in Neural Language Models

DAIS, a large benchmark dataset containing 50K human judgments for 5K distinct sentence pairs in the English dative alternation, is introduced, showing that larger models perform better than smaller models, and transformer architectures tend to out-perform recurrent architectures even under comparable parameter and training settings.

Probing for Understanding of English Verb Classes and Alternations in Large Pre-trained Language Models

Investigation of the extent to which verb alternation classes are encoded in the embeddings of Large Pretrained Language Models such as BERT, RoBERTa, ELECTRA, and DeBERTa using selectively constructed diagnostic classifiers for word and sentence-level prediction tasks finds that contextualembeddings from PLMs not only outperform non-contextual embedDings, but achieve astonishingly high accuracies on tasks across most alternation Classes.

Investigating Novel Verb Learning in BERT: Selectional Preference Classes and Alternation-Based Syntactic Generalization

A novel word-learning paradigm is deployed to test BERT’s few-shot learning capabilities for two aspects of English verbs: alternations and classes of selectional preferences and finds that BERT makes robust grammatical generalizations after just one or two instances of a novel word in fine-tuning.

Grammatical Analysis of Pretrained Sentence Encoders with Acceptability Judgments

A grammatically annotated development set for the Corpus of Linguistic Acceptability (CoLA; Warstadt et al., 2018) is introduced, which is used to investigate the grammatical knowledge of three pretrained encoders, including the popular OpenAI Transformer and BERT.

Learning Argument Structures with Recurrent Neural Network Grammars

In targeted syntactic evaluations, the syntactic competence of language models (LMs) has been investigated through various syntactic phenomena, among which one of the important domains has been

BLiMP: A Benchmark of Linguistic Minimal Pairs for English

The Benchmark of Linguistic Minimal Pairs, a challenge set for evaluating the linguistic knowledge of language models (LMs) on major grammatical phenomena in English, finds that state-of-the-art models identify morphological contrasts related to agreement reliably, but they struggle with some subtle semantic and syntactic phenomena.

Cross-Linguistic Syntactic Evaluation of Word Prediction Models

ClAMS (Cross-Linguistic Assessment of Models on Syntax), a syntactic evaluation suite for monolingual and multilingual models, is introduced, which uses subject-verb agreement challenge sets for English, French, German, Hebrew and Russian, generated from grammars developed.

Corpus-based Identification of Verbs Participating in Verb Alternations Using Classification and Manual Annotation

This work uses ENCOW and VerbNet data to train classifiers to predict the instrument subject alternation and the causative-inchoative alternation, relying on count-based and vector-based features as well as perplexity-based language model features, which are intended to reflect each alternation’s felicity by simulating it.

Sentence Analogies: Linguistic Regularities in Sentence Embeddings

This paper investigates to what extent commonly used sentence vector representation spaces as well reflect certain kinds of regularities, and proposes a number of schemes to induce evaluation data, based on lexical analogy data as well as semantic relationships between sentences.

Using Integrated Gradients and Constituency Parse Trees to explain Linguistic Acceptability learnt by BERT

The decision-making process of BERT in distinguishing between Linguistic Acceptable sentences (LA) and Linguistically Unacceptable sentences (LUA) is understood and Layer Integrated Gradients Attribution Scores (LIG) are leveraged to explain the L linguistic Acceptability criteria that are learnt by BERT on the Corpus of Linguists Acceptability (CoLA) (Warstadt et al., 2018).



Verbnet: a broad-coverage, comprehensive verb lexicon

VerbNet is created, a verb lexicon compatible with Word-Net but with explicitly stated syntactic and semantic information, using Levin verb classes to systematically construct lexical entries, to address the gap in coverage of syntactic frames and predicate argument structures associated with individual verb senses.

English Verb Classes and Alternations: A Preliminary Investigation

Beth Levin shows how identifying verbs with similar syntactic behavior provides an effective means of distinguishing semantically coherent verb classes, and isolates these classes by examining verb behavior with respect to a wide range of syntactic alternations that reflect verb meaning.

Neural Network Acceptability Judgments

This paper introduces the Corpus of Linguistic Acceptability (CoLA), a set of 10,657 English sentences labeled as grammatical or ungrammatical from published linguistics literature, and trains several recurrent neural network models on acceptability classification, and finds that the authors' models outperform unsupervised models by Lau et al. (2016) on CoLA.

Linguistic Regularities in Continuous Space Word Representations

The vector-space word representations that are implicitly learned by the input-layer weights are found to be surprisingly good at capturing syntactic and semantic regularities in language, and that each relationship is characterized by a relation-specific vector offset.

Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies

It is concluded that LSTMs can capture a non-trivial amount of grammatical structure given targeted supervision, but stronger architectures may be required to further reduce errors; furthermore, the language modeling signal is insufficient for capturing syntax-sensitive dependencies, and should be supplemented with more direct supervision if such dependencies need to be captured.

Deep Contextualized Word Representations

A new type of deep contextualized word representation is introduced that models both complex characteristics of word use and how these uses vary across linguistic contexts, allowing downstream models to mix different types of semi-supervision signals.

VerbNet Class Assignment as a WSD Task

This work describes a VerbNet token classifier, which uses rich syntactic and semantic features to label verb instances with their appropriate VerbNet class, and achieves an accuracy of 88.67% with multiclass verbs, which is a 49% error reduction over the most frequent class baseline.

Extending VerbNet with Novel Verb Classes

The integration of 57 novel classes for verbs not covered (comprehensively) by Levin into VerbNet is described, which is the most extensive Levin-style classification for English verbs which can be highly useful for practical applications.

From Phonology to Syntax: Unsupervised Linguistic Typology at Different Levels with Language Embeddings

A core part of linguistic typology is the classification of languages according to linguistic properties, such as those detailed in the World Atlas of Language Structure (WALS). Doing this manually

Unaccusativity: At the Syntax-Lexical Semantics Interface

Unaccusativity is an extended investigation into a set of linguistic phenomena that have received much attention over the last fifteen years. Besides providing extensive support for David