• Corpus ID: 245219282

Reframing Human-AI Collaboration for Generating Free-Text Explanations

@article{Wiegreffe2021ReframingHC,
  title={Reframing Human-AI Collaboration for Generating Free-Text Explanations},
  author={Sarah Wiegreffe and Jack Hessel and Swabha Swayamdipta and Mark O. Riedl and Yejin Choi},
  journal={ArXiv},
  year={2021},
  volume={abs/2112.08674}
}
Large language models are increasingly capa-ble of generating fluent-appearing text with relatively little task-specific supervision. But can these models accurately explain classification decisions? We consider the task of generating free-text explanations using human-written examples in a few-shot manner. We find that (1) authoring higher quality prompts results in higher quality generations; and (2) surprisingly, in a head-to-head comparison, crowdworkers often prefer explanations generated by… 
Few-Shot Self-Rationalization with Natural Language Prompts
TLDR
This work identifies the right prompting approach by extensively exploring natural language prompts on FEB and demonstrates that making progress on few-shot self-rationalization is possible, and presents FEB—a stan-dardized collection of four existing English-language datasets and associated metrics.
Chain of Thought Prompting Elicits Reasoning in Large Language Models
TLDR
Experiments show that inducing a chain of thought via prompting can be enabled byently large language models to better perform reasoning tasks that otherwise have flat scaling curves.
What to Learn, and How: Toward Effective Learning from Rationales
TLDR
This work highlights the importance of understanding properties of human explanations and exploiting them accordingly in model training, and proposes several novel loss functions and learning strategies that demonstrate consistent improvements over baselines in both label and rationale accuracy.

References

SHOWING 1-10 OF 67 REFERENCES
Few-Shot Self-Rationalization with Natural Language Prompts
TLDR
This work identifies the right prompting approach by extensively exploring natural language prompts on FEB and demonstrates that making progress on few-shot self-rationalization is possible, and presents FEB—a stan-dardized collection of four existing English-language datasets and associated metrics.
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation
TLDR
This work introduces a novel paradigm for dataset creation based on human and machine collaboration, which brings together the generative strength of language models and the eval-uativestrength of humans in order to curate NLP datasets of enhanced quality and diversity.
Explaining Question Answering Models through Text Generation
TLDR
A model for multi-choice question answering, where a LM-based generator generates a textual hypothesis that is later used by a classifier to answer the question, and produces hypotheses that elucidate the knowledge used by the LM for answering the question.
e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks
TLDR
e-ViL is a benchmark for explainable vision-language tasks that establishes a unified evaluation framework and provides the first comprehensive comparison of existing approaches that generate NLEs for VL tasks.
Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing
TLDR
This review identifies 61 datasets with three predominant classes of textual expla6 nations (highlights, free-text, and structured), organize the literature on annotating each type, identify strengths and shortcomings of existing collection methodologies, and give recommendations for collecting EXNLP datasets in the future.
Rationale-Inspired Natural Language Explanations with Commonsense
TLDR
This paper introduces a selfrationalizing framework, called REXC, that extracts rationales as most responsible features for the predictions, expands the extractive rationales using commonsense resources, and selects the best-suited commonsense knowledge to generate NLEs and give the final prediction.
Annotation Artifacts in Natural Language Inference Data
TLDR
It is shown that a simple text categorization model can correctly classify the hypothesis alone in about 67% of SNLI and 53% of MultiNLI, and that specific linguistic phenomena such as negation and vagueness are highly correlated with certain inference classes.
Explanations for CommonsenseQA: New Dataset and Models
TLDR
This work human-annotates a first-of-its-kind dataset of positive and negative properties, as well as free-flow explanations, for 11K QA pairs taken from the CQA dataset, and proposes a latent representation based property retrieval model aswell as a GPT-2 based property generation model with a novel two step fine-tuning procedure.
ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning
TLDR
This work presents ExplaGraphs, a new generative and structured commonsense-reasoning task (and an associated dataset) of explanation graph generation for stance prediction, and proposes a multi-level evaluation framework that check for the structural and semantic correctness of the generated graphs and their degree of match with ground-truth graphs.
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
TLDR
This work uses the generative nature of the language models to construct an artificial development set and based on entropy statistics of the candidate permutations from this set the authors identify performant prompts and improves upon GPT-family models by on average 13% relative across eleven different established text classification tasks.
...
1
2
3
4
5
...