Investigating Memorization of Conspiracy Theories in Text Generation

  title={Investigating Memorization of Conspiracy Theories in Text Generation},
  author={Sharon Levy and Michael Stephen Saxon and William Yang Wang},
The adoption of natural language generation (NLG) models can leave individuals vulnerable to the generation of harmful information memorized by the models, such as conspiracy theories. While previous studies examine conspiracy theories in the context of social media, they have not evaluated their presence in the new space of generative language models. In this work, we investigate the capability of language models to generate conspiracy theory text. Specifically, we aim to answer: can we test… 

Figures and Tables from this paper

Do Language Models Plagiarize?
The findings support that language models, especially GPT-2, reuse particular pieces of texts from the training corpus with or without obfuscation, and implies that future research on neural language models should take precautions to avoid models plagiarizing their training datasets.


Extracting Training Data from Large Language Models
This paper demonstrates that in such settings, an adversary can perform a training data extraction attack to recover individual training examples by querying the language model, and finds that larger models are more vulnerable than smaller models.
Release Strategies and the Social Impacts of Language Models
This report discusses OpenAI's work related to the release of its GPT-2 language model and discusses staged release, which allows time between model releases to conduct risk and benefit analyses as model sizes increased.
A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk
The characteristics of tasks and working patterns that yield higher hourly wages are explored, and platform design and worker tools are informed to create a more positive future for crowd work.
Research in the crowdsourcing age: A case study
  • 2016
What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation
The experiments demonstrate the significant benefits of memorization for generalization on several standard benchmarks and provide quantitative and visually compelling evidence for the theory put forth in Feldman (2019), which proposes a theoretical explanation for this phenomenon.
Towards Understanding Sample Variance in Visually Grounded Language Generation: Evaluations and Observations
A major challenge in visually grounded language generation is to build robust benchmark datasets and models that can generalize well in real-world settings. To do this, it is critical to ensure that
Dats Wassup!!: Investigating African-American Vernacular English in Transformer-Based Text Generation
This work investigates the performance of GPT-2 on AAVE text by creating a dataset of intent-equivalent parallel AAVE/SAE tweet pairs and isolating syntactic structure and AAVE- or SAE-specific language for each pair, and conducts human evaluation of AAVE and SAE text generated with GPT -2 to compare contextual rigor and overall quality.
The Radicalization Risks of GPT-3 and Advanced Neural Language Models
GPT-3 demonstrates significant improvement over its predecessor, GPT-2, in generating extremist texts and its strength in generating text that accurately emulates interactive, informational, and influential content that could be utilized for radicalizing individuals into violent far-right extremist ideologies and behaviors.
Language Models are Few-Shot Learners
GPT-3 achieves strong performance on many NLP datasets, including translation, question-answering, and cloze tasks, as well as several tasks that require on-the-fly reasoning or domain adaptation, such as unscrambling words, using a novel word in a sentence, or performing 3-digit arithmetic.
COVID-19 and the 5G Conspiracy Theory: Social Network Analysis of Twitter Data
This study is the first to analyze the 5G conspiracy theory in the context of COVID-19 on Twitter offering practical guidance to health authorities in how, in thecontext of a pandemic, rumors may be combated in the future.