ValNorm Quantifies Semantics to Reveal Consistent Valence Biases Across Languages and Over Centuries

@inproceedings{Toney2021ValNormQS,
  title={ValNorm Quantifies Semantics to Reveal Consistent Valence Biases Across Languages and Over Centuries},
  author={Autumn Toney and Aylin Caliskan},
  booktitle={EMNLP},
  year={2021}
}
Word embeddings learn implicit biases from linguistic regularities captured by word co-occurrence statistics. By extending methods that quantify human-like biases in word embeddings, we introduce ValNorm, a novel intrinsic evaluation task and method to quantify the valence dimension of affect in human-rated word sets from social psychology. We apply ValNorm on static word embeddings from seven languages (Chinese, English, German, Polish, Portuguese, Spanish, and Turkish) and from historical… 
Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Models
TLDR
It is found that infrequent names are more self-similar across contexts, indicating that models rely on less context-informed representations of uncommon and minority names which are overfit to a lower number of observed contexts.
Measuring Gender Bias in Word Embeddings of Gendered Languages Requires Disentangling Grammatical Gender Signals
Does the grammatical gender of a language interfere when measuring the semantic gender information captured by its word embeddings? A number of anomalous gender bias measurements in the embeddings of
Contrastive Visual Semantic Pretraining Magnifies the Semantics of Natural Language Representations
TLDR
The results indicate that high anisotropy is not an inevitable consequence of contextualization, and that visual semantic pretraining is beneficial not only for ordering visual representations, but also for encoding useful semantic representations of language, both on the word level and the sentence level.
Gender Bias in Word Embeddings: A Comprehensive Analysis of Frequency, Syntax, and Semantics
Word embeddings are numeric representations of meaning derived from word co-occurrence statistics in corpora of human-produced texts. The statistical regularities in language corpora encode
American == White in Multimodal Language-and-Image AI
Three state-of-the-art language-and-image AI models, CLIP, SLIP, and BLIP, are evaluated for evidence of a bias previously observed in social and experimental psychology: equating American identity
Evidence for Hypodescent in Visual Semantic AI
We examine the state-of-the-art multimodal ”visual semantic” model CLIP (”Contrastive Language Image Pretraining”) for the rule of hypodescent, or one-drop rule, whereby multiracial people are more
VAST: The Valence-Assessing Semantics Test for Contextualizing Language Models
TLDR
VAST uses valence, the association of a word with pleasantness, to measure the correspondence of word-level LM semantics with widely used human judgments, and examines the effects of contextualization, tokenization, and LM-specific geometry.

References

SHOWING 1-10 OF 78 REFERENCES
Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases
TLDR
The first algorithmic bias detection findings on how intersectional group members are strongly associated with unique emergent biases that do not overlap with the biases of their constituent minority identities are presented.
Word embeddings quantify 100 years of gender and ethnic stereotypes
TLDR
A framework to demonstrate how the temporal dynamics of the embedding helps to quantify changes in stereotypes and attitudes toward women and ethnic minorities in the 20th and 21st centuries in the United States is developed.
Understanding Undesirable Word Embedding Associations
TLDR
It is shown that for any embedding model that implicitly does matrix factorization, debiasing vectors post hoc using subspace projection is, under certain conditions, equivalent to training on an unbiased corpus, and that WEAT, the most common association test for word embeddings, systematically overestimates bias.
Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change
TLDR
A robust methodology for quantifying semantic change is developed by evaluating word embeddings against known historical changes and it is revealed that words that are more polysemous have higher rates of semantic change.
Inferring Affective Meanings of Words from Word Embedding
TLDR
This work proposes a regression-based method to automatically infer multi-dimensional affective representation of words via their word embedding based on a set of seed words that can make use of the rich semantic meanings obtained fromword embedding to extract meanings in some specific semantic space.
CogniVal: A Framework for Cognitive Word Embedding Evaluation
TLDR
This paper presents the first multi-modal framework for evaluating English word representations based on cognitive lexical semantics, and finds strong correlations in the results between cognitive datasets, across recording modalities and to their performance on extrinsic NLP tasks.
Semantics derived automatically from language corpora contain human-like biases
TLDR
It is shown that machines can learn word associations from written texts and that these associations mirror those learned by humans, as measured by the Implicit Association Test (IAT), and that applying machine learning to ordinary human language results in human-like semantic biases.
Predicting Emotional Word Ratings using Distributional Representations and Signed Clustering
TLDR
This work develops a method that automatically extends word-level ratings to unrated words using signed clustering of vector space word representations along with affect ratings, which achieves superior out-of-sample word rating prediction on both affective dimensions across three different languages when compared to state- of-the-art word similarity based methods.
subs2vec: Word embeddings from subtitles in 55 languages
TLDR
A novel collection of word embeddings, numerical representations of lexical semantics, in 55 languages, trained on a large corpus of pseudo-conversational speech transcriptions from television shows and movies, and a novel evaluation method of particular relevance to psycholinguists.
ANGST: Affective norms for German sentiment terms, derived from the affective norms for English words
TLDR
The classic ratings of valence, arousal, and dominance were extended with additional arousal ratings using a slightly different scale, and measures of several objective psycholinguistic variables for the words were also added, so as to further facilitate the use of this new database in psychol linguistic research.
...
...