“You Are Grounded!”: Latent Name Artifacts in Pre-trained Language Models

@article{Shwartz2020YouAG,
  title={“You Are Grounded!”: Latent Name Artifacts in Pre-trained Language Models},
  author={Vered Shwartz and Rachel Rudinger and Oyvind Tafjord},
  journal={ArXiv},
  year={2020},
  volume={abs/2004.03012}
}
Pre-trained language models (LMs) may perpetuate biases originating in their training corpus to downstream models. We focus on artifacts associated with the representation of given names (e.g., Donald), which, depending on the corpus, may be associated with specific entities, as indicated by next token prediction (e.g., Trump). While helpful in some contexts, grounding happens also in under-specified or inappropriate contexts. For example, endings generated for `Donald is a' substantially… Expand
Societal Biases in Language Generation: Progress and Challenges
TLDR
A survey on societal biases in language generation is presented, focusing on how data and techniques contribute to biases and progress towards reducing biases, and the effects of decoding techniques. Expand
Do Neural Language Models Overcome Reporting Bias?
TLDR
It is found that while pre-trained language models' generalization capacity allows them to better estimate the plausibility of frequent but unspoken of actions, outcomes, and properties, they also tend to overestimate that of the very rare, amplifying the bias that already exists in their training corpus. Expand
Entity-Based Knowledge Conflicts in Question Answering
Knowledge-dependent tasks typically use two sources of knowledge: parametric, learned at training time, and contextual, given as a passage at inference time. To understand how models use theseExpand
Evaluating Entity Disambiguation and the Role of Popularity in Retrieval-Based NLP
TLDR
It is found that the retrievers exhibit popularity bias, significantly under-performing on rarer entities that share a name, e.g., they are twice as likely to retrieve erroneous documents on queries for the less popular entity under the same name. Expand
Gender and Representation Bias in GPT-3 Generated Stories
Using topic modeling and lexicon-based word similarity, we find that stories generated by GPT-3 exhibit many known gender stereotypes. Generated stories depict different topics and descriptionsExpand
Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases
TLDR
It is found that previous decent performance mainly owes to the biased prompts which overfit dataset artifacts, and incorporating illustrative cases and external contexts improve knowledge prediction mainly due to entity type guidance and golden answer leakage. Expand
Quantifying Gender Bias Towards Politicians in Cross-Lingual Language Models
TLDR
A simple method to probe pre-trained language models for gender bias, which is used to effect a multi-lingual study of gender bias towards politicians, and suggests that larger language models do not tend to be significantly more gender-biased than smaller ones. Expand
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets
Auditing NLP systems for computational harms like surfacing stereotypes is an elusive goal. Several recent efforts have focused on benchmark datasets consisting of pairs of contrastive sentences,Expand
Uncovering Implicit Gender Bias in Narratives through Commonsense Inference
Pre-trained language models learn socially harmful biases from their training corpora, and may repeat these biases when used for generation. We study gender biases associated with the protagonist inExpand
What do Bias Measures Measure?
TLDR
This work presents a comprehensive survey of existing bias measures in NLP as a function of the associated NLP tasks, metrics, datasets, and social biases and corresponding harms and proposes a documentation standard for bias measures to aid their development, categorization, and appropriate usage. Expand
...
1
2
...

References

SHOWING 1-10 OF 44 REFERENCES
Perturbation Sensitivity Analysis to Detect Unintended Model Biases
TLDR
A generic evaluation framework, Perturbation Sensitivity Analysis, is proposed, which detects unintended model biases related to named entities, and requires no new annotations or corpora to be employed. Expand
Language Models are Unsupervised Multitask Learners
TLDR
It is demonstrated that language models begin to learn these tasks without any explicit supervision when trained on a new dataset of millions of webpages called WebText, suggesting a promising path towards building language processing systems which learn to perform tasks from their naturally occurring demonstrations. Expand
Language Models as Knowledge Bases?
TLDR
An in-depth analysis of the relational knowledge already present (without fine-tuning) in a wide range of state-of-the-art pretrained language models finds that BERT contains relational knowledge competitive with traditional NLP methods that have some access to oracle knowledge. Expand
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
TLDR
This systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks and achieves state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. Expand
Improving Language Understanding by Generative Pre-Training
TLDR
The general task-agnostic model outperforms discriminatively trained models that use architectures specifically crafted for each task, significantly improving upon the state of the art in 9 out of the 12 tasks studied. Expand
Barack’s Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling
TLDR
This work introduces the knowledge graph language model (KGLM), a neural language model with mechanisms for selecting and copying facts from a knowledge graph that are relevant to the context that enable the model to render information it has never seen before, as well as generate out-of-vocabulary tokens. Expand
The Woman Worked as a Babysitter: On Biases in Language Generation
TLDR
The notion of the regard towards a demographic is introduced, the varying levels of regard towards different demographics are used as a defining metric for bias in NLG, and the extent to which sentiment scores are a relevant proxy metric for regard is analyzed. Expand
On Measuring Social Biases in Sentence Encoders
TLDR
The Word Embedding Association Test is extended to measure bias in sentence encoders and mixed results including suspicious patterns of sensitivity that suggest the test’s assumptions may not hold in general. Expand
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TLDR
A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks. Expand
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
TLDR
A Sentiment Treebank that includes fine grained sentiment labels for 215,154 phrases in the parse trees of 11,855 sentences and presents new challenges for sentiment compositionality, and introduces the Recursive Neural Tensor Network. Expand
...
1
2
3
4
5
...