Discovering Differences in the Representation of People using Contextualized Semantic Axes

@inproceedings{Lucy2022DiscoveringDI,
  title={Discovering Differences in the Representation of People using Contextualized Semantic Axes},
  author={Li Lucy and Divya Tadimeti and David Bamman},
  booktitle={Conference on Empirical Methods in Natural Language Processing},
  year={2022}
}
A common paradigm for identifying semantic differences across social and temporal contexts is the use of static word embeddings and their distances. In particular, past work has compared embeddings against “semantic axes” that represent two opposing concepts. We extend this paradigm to BERT embeddings, and construct contextualized axes that mitigate the pitfall where antonyms have neighboring representations. We validate and demonstrate these axes on two people-centric datasets: occupations… 
[PDF] Semantic Reader
1 Citations

One Citation

Beyond Fish and Bicycles: Exploring the Varieties of Online Women's Ideological Spaces

The Internet has been instrumental in connecting under-represented and vulnerable groups of people. Platforms built to foster social interaction and engagement have enabled historically…

References

SHOWING 1-10 OF 86 REFERENCES

Entity-Centric Contextual Affective Analysis

It is found that contextualized word representations do encode meaningful affect information, but they are heavily biased towards their training data, which limits their usefulness to in-domain analyses.

Diachronic Sense Modeling with Deep Contextualized Word Embeddings: An Ecological View

A sense representation and tracking framework based on deep contextualized embeddings aiming at answering not only what and when, but also how the word meaning changes is proposed, bringing a significant improvement in word change detection task.

Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift

The proposed method has performance comparable to the current state-of-the-art without requiring any time consuming domain adaptation on large corpora and shows promising results in a multilingual settings.

Historical representations of social groups across 200 years of word embeddings from Google Books

Significance How did societies of the past represent the various social groups of their world? Here, we address this question using word embeddings from 850 billion words of English-language books…

On the use of Jargon and Word Embeddings to Explore Subculture within the Reddit’s Manosphere

A combination of computational and socio-linguistic methods are proposed to automatically extract new terminology from large amounts of data, using word-embeddings to semantically contextualise their meaning.

WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations

A large-scale Word in Context dataset, called WiC, based on annotations curated by experts, for generic evaluation of context-sensitive representations, and shows that existing models have surpassed the performance ceiling of the standard evaluation dataset for the purpose.

The Geometry of Culture: Analyzing the Meanings of Class through Word Embeddings

It is argued word embedding models are a useful tool for the study of culture using a historical analysis of shared understandings of social class as an empirical case, specifying a relational model of meaning consistent with contemporary theories of culture.

How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings

It is found that in all layers of ELMo, BERT, and GPT-2, on average, less than 5% of the variance in a word’s contextualized representations can be explained by a static embedding for that word, providing some justification for the success of contextualization representations.

The POLAR Framework: Polar Opposites Enable Interpretability of Pre-Trained Word Embeddings

The introduction of ‘POLAR’ — a framework that adds interpretability to pre-trained word embeddings via the adoption of semantic differentials and shows that the interpretable dimensions selected by the framework align with human judgement.

Semantic projection recovers rich human knowledge of multiple object features from word embeddings

It is shown that context-dependent knowledge about objects, such as the similarities between animals in terms of size versus danger versus habitat, can be recovered from word embeddings via a simple, interpretable geometrical operation.
...