• Corpus ID: 237386009

Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond

  title={Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond},
  author={Amir Feder and Katherine A. Keith and Emaad A. Manzoor and Reid Pryzant and Dhanya Sridhar and Zach Wood-Doughty and Jacob Eisenstein and Justin Grimmer and Roi Reichart and Margaret E. Roberts and Brandon M Stewart and Victor Veitch and Diyi Yang},
A fundamental goal of scientific research is to learn about causal relationships. However, despite its critical role in the life and social sciences, causality has not had the same importance in Natural Language Processing (NLP), which has traditionally placed more emphasis on predictive tasks. This distinction is beginning to fade, with an emerging area of interdisciplinary research at the convergence of causal inference and language processing. Still, research on causality in NLP remains… 

Figures from this paper

Causally Estimating the Sensitivity of Neural NLP Models to Spurious Features
This work quantifies model sensitivity to spurious features with a causal estimand, dubbed CENT, which draws on the concept of average treatment effect from the causality literature, to hypothesize and validate that models that are more sensitive to a spurious feature will be less robust against perturbations with this feature during inference.
Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective
This work first introduces a causal formalism of motion forecasting, which casts the problem as a dynamic process with three groups of latent variables, namely invariant mechanisms, style confounders, and spurious features, and introduces a learning framework that treats each group separately.
Unsupervised Editing for Counterfactual Stories
EDUCAT, an editing-based unsupervised approach for counterfactual story rewriting that includes a target position detection strategy based on estimating causal effects of the what-if conditions, keeps the causal invariant parts of the story.
Beyond Ads: Sequential Decision-Making Algorithms in Public Policy
This work highlights several applications of sequential decision-making algorithms in regulation and governance, and discusses areas for further research which would enable them to be more widely applicable, fair, and effective.
Natural Language Processing for Smart Healthcare
This work focuses on feature extraction and modelling for various NLP tasks encountered in smart healthcare from a technical point of view and discusses the limitations of current works and identifies the directions for future works.
Using Sociolinguistic Variables to Reveal Changing Attitudes Towards Sexuality and Gender
It is demonstrated that attitudes are changing but that these changes are driven by specific demographics within the United States, and it is shown that passages of Marriage Equality Acts in different states are drivers of linguistic change.


CausaLM: Causal Model Explanation Through Counterfactual Language Models
CausaLM is proposed, a framework for producing causal model explanations using counterfactual language representation models based on fine-tuning of deep contextualized embedding models with auxiliary adversarial tasks derived from the causal graph of the problem.
Challenges of Using Text Classifiers for Causal Inference
It is demonstrated how to conduct causal analyses using text classifiers on simulated and Yelp data, and the opportunities and challenges of future work that uses text data in causal inference are discussed.
Deconfounded Lexicon Induction for Interpretable Social Science
Two deep learning algorithms are introduced that are more predictive and less confound-related than those of standard feature weighting and lexicon induction techniques like regression and log odds and used to induce lexicons that are predictive of timely responses to consumer complaints, enrollment from course descriptions, and sales from product descriptions.
Annotation Artifacts in Natural Language Inference Data
It is shown that a simple text categorization model can correctly classify the hypothesis alone in about 67% of SNLI and 53% of MultiNLI, and that specific linguistic phenomena such as negation and vagueness are highly correlated with certain inference classes.
Replicability Analysis for Natural Language Processing: Testing Significance with Multiple Datasets
This paper proposes a Replicability Analysis framework for a statistically sound analysis of multiple comparisons between algorithms for NLP tasks, and demonstrates its empirical value across four applications: multi-domain dependency parsing, multilingual POS tagging, cross-domain sentiment classification and word similarity prediction.
Stress Test Evaluation for Natural Language Inference
This work proposes an evaluation methodology consisting of automatically constructed “stress tests” that allow us to examine whether systems have the ability to make real inferential decisions, and reveals strengths and weaknesses of these models with respect to challenging linguistic phenomena.
Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimates
This review is the first to gather and categorize examples of potential confounders from observed text and provide a guide to data-processing and evaluation decisions.
How to Make Causal Inferences Using Texts
A conceptual framework for making causal inferences with discovered measures as a treatment or outcome is introduced and this framework enables researchers to discover high-dimensional textual interventions and estimate the ways that observed treatments affect text-based outcomes.
Robust Text Classification under Confounding Shift
Pearl's back-door adjustment is used as a predictive framework to develop a model robust to confounding shift under the condition that Z is observed at training time and it is shown that this approach is able to outperform baselines in controlled cases where confounding shift is manually injected between fitting time and prediction time.
Investigating Gender Bias in Language Models Using Causal Mediation Analysis
This work proposes a methodology grounded in the theory of causal mediation analysis for interpreting which parts of a model are causally implicated in its behavior, and applies it to analyzing gender bias in pre-trained Transformer language models.