Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information

@inproceedings{Giulianelli2018UnderTH,
  title={Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Information},
  author={Mario Giulianelli and John Harding and Florian Mohnert and Dieuwke Hupkes and Willem H. Zuidema},
  booktitle={BlackboxNLP@EMNLP},
  year={2018}
}
How do neural language models keep track of number agreement between subject and verb? We show that ‘diagnostic classifiers’, trained to predict number from the internal states of a language model, provide a detailed understanding of how, when, and where this information is represented. Moreover, they give us insight into when and where number information is corrupted in cases where the language model ends up making agreement errors. To demonstrate the causal role played by the representations… 
Analysing Neural Language Models: Contextual Decomposition Reveals Default Reasoning in Number and Gender Assignment
Extensive research has recently shown that recurrent neural language models are able to process a wide range of grammatical phenomena. How these models are able to perform these remarkable feats so
Analysing Neural Language Models: Contextual Decomposition Reveals Default Reasoning in Number and Gender Assignment
TLDR
A generalisation of Contextual Decomposition (GCD) is proposed that enables us to accurately distil which part of a prediction stems from semantic heuristics, which part truly emanates from syntactic cues and which part arise from the model biases themselves instead.
Understanding Learning Dynamics Of Language Models with SVCCA
TLDR
This first study on the learning dynamics of neural language models is presented, using a simple and flexible analysis method called Singular Vector Canonical Correlation Analysis (SVCCA), which enables to compare learned representations across time and across models, without the need to evaluate directly on annotated data.
What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models
TLDR
A suite of diagnostics drawn from human language experiments are introduced, which allow us to ask targeted questions about information used by language models for generating predictions in context, and the popular BERT model is applied.
Attribution Analysis of Grammatical Dependencies in LSTMs
TLDR
Using layer-wise relevance propagation, it is shown that LSTM performance on number agreement is directly correlated with the model's ability to distinguish subjects from other nouns, suggesting that L STM language models are able to infer robust representations of syntactic dependencies.
Language Models Use Monotonicity to Assess NPI Licensing
TLDR
This work introduces a series of experiments consisting of probing with diagnostic classifiers (DCs), linguistic acceptability tasks, as well as a novel DC ranking method that tightly connects the probing results to the inner workings of the LM.
Influence Paths for Characterizing Subject-Verb Number Agreement in LSTM Language Models
TLDR
A causal account of structural properties as carried by paths across gates and neurons of a recurrent neural network is introduced, offering a finer and a more complete view of an LSTM’s handling of this structural aspect of the English language than prior results based on diagnostic classifiers and ablation.
Are Transformers a Modern Version of ELIZA? Observations on French Object Verb Agreement
TLDR
This work takes a critical look at this line of research by showing that it is possible to achieve high accuracy on this agreement task with simple surface heuristics, indicating a possible flaw in the assessment of neural networks’ syntactic ability.
Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languages
TLDR
A paradigm is proposed that creates synthetic versions of English, which differ from English in one or more typological parameters, and generates corpora for those languages based on a parsed English corpus, and finds overt morphological case makes agreement prediction significantly easier, regardless of word order.
Investigating BERT’s Knowledge of Language: Five Analysis Methods with NPIs
TLDR
It is concluded that a variety of methods is necessary to reveal all relevant aspects of a model’s grammatical knowledge in a given domain.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 14 REFERENCES
Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies
TLDR
It is concluded that LSTMs can capture a non-trivial amount of grammatical structure given targeted supervision, but stronger architectures may be required to further reduce errors; furthermore, the language modeling signal is insufficient for capturing syntax-sensitive dependencies, and should be supplemented with more direct supervision if such dependencies need to be captured.
Visualisation and 'diagnostic classifiers' reveal how recurrent and recursive neural networks process hierarchical structure
TLDR
The results indicate that the networks follow a strategy similar to the hypothesised ‘cumulative strategy’, which explains the high accuracy of the network on novel expressions, the generalisation to longer expressions than seen in training, and the mild deterioration with increasing length.
Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks
TLDR
This work proposes a framework that facilitates better understanding of the encoded representations of sentence vectors and demonstrates the potential contribution of the approach by analyzing different sentence representation mechanisms.
Representation of Linguistic Form and Function in Recurrent Neural Networks
TLDR
A method for estimating the amount of contribution of individual tokens in the input to the final prediction of the networks is proposed and shows that the Visual pathway pays selective attention to lexical categories and grammatical functions that carry semantic information, and learns to treat word types differently depending on their grammatical function and their position in the sequential structure of the sentence.
Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context
TLDR
This paper investigates the role of context in an LSTM LM, through ablation studies, and analyzes the increase in perplexity when prior context words are shuffled, replaced, or dropped to provide a better understanding of how neural LMs use their context.
What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties
TLDR
10 probing tasks designed to capture simple linguistic features of sentences are introduced and used to study embeddings generated by three different encoders trained in eight distinct ways, uncovering intriguing properties of bothencoders and training methods.
Colorless green recurrent networks dream hierarchically
TLDR
Support is brought to the hypothesis that RNNs are not just shallow-pattern extractors, but they also acquire deeper grammatical competence by making reliable predictions about long-distance agreement and do not lag much behind human performance.
An Analysis of Neural Language Modeling at Multiple Scales
TLDR
This work takes existing state-of-the-art word level language models based on LSTMs and QRNNs and extend them to both larger vocabularies as well as character-level granularity, achieving state- of- the-art results on character- level and word-level datasets.
The Semantics of Adjective Noun Phrases in the Human Brain
TLDR
This work explores semantic composition using magnetoencephalography recordings of people reading adjective-noun phrases presented one word at a time, and tracks the neural representation of semantic information over time, through different brain regions.
Characterizing the dynamics of mental representations: the temporal generalization method
...
1
2
...