Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences

  title={Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences},
  author={Denis Emelin and Ronan Le Bras and Jena D. Hwang and Maxwell Forbes and Yejin Choi},
In social settings, much of human behavior is governed by unspoken rules of conduct rooted in societal norms. For artificial systems to be fully integrated into social environments, adherence to such norms is a central prerequisite. To investigate whether language generation models can serve as behavioral priors for systems deployed in social settings, we evaluate their ability to generate action descriptions that achieve predefined goals under normative constraints. Moreover, we examine if… 

Aligning to Social Norms and Values in Interactive Narratives

The GALAD (Game-value ALignment through Action Distillation) agent is introduced that uses the social commonsense knowledge present in specially trained language models to contextually restrict its action space to only those actions that are aligned with socially beneficial values.

Delphi: Towards Machine Ethics and Norms

The first major attempt to computationally explore the vast space of moral implications in real-world settings is conducted, with Delphi, a unified model of descriptive ethics empowered by diverse data of people’s moral judgment from COMMONSENSE NORM BANK.


The first major attempt to computationally explore the vast space of moral implications in real-world settings is conducted, with Delphi, a unified model of descriptive ethics empowered by diverse data of people’s moral judgment from COMMONSENSE NORM BANK.

The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems

The Moral Integrity Corpus, MIC, is a resource, which captures the moral assumptions of 38k prompt-reply pairs, using 99k distinct Rules of Thumb (RoTs), and is suggested that MIC will be a useful resource for understanding and language models’ implicit moral assumptions and flexibly benchmarking the integrity of conversational agents.

Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?

This work proposes a new language understanding task, Linguistic Ethical Interventions (LEI), where the goal is to amend a questionanswering (QA) model’s unethical behavior by communicating context-specific principles of ethics and equity to it.

Possible Stories: Evaluating Situated Commonsense Reasoning under Multiple Possible Scenarios

This study frames this task by asking multiple questions with the same set of possible endings as candidate answers, given a short story text, and discovers that even current strong pretrained language models struggle to answer the questions consistently.

Does Moral Code have a Moral Code? Probing Delphi’s Moral Philosophy

In an effort to guarantee that machine learning model outputs conform with human moral values, recent work has begun exploring the possibility of explicitly training models to learn the difference

A Corpus for Understanding and Generating Moral Stories

A retrieval-augmented algorithm that effectively exploits related concepts or events in training sets as additional guidance to improve performance on two understanding tasks and two generation tasks to assess the abilities of machines.

Assessing Cognitive Linguistic Influences in the Assignment of Blame

There are statistically significant differences in uses of first-person passive voice, as well as first- person agents and patients, between descriptions of situations that receive different blame judgments, and these features also aid performance in the task of predicting the eventual collective verdicts.

Aligning Generative Language Models with Human Values

S ENSEI is a new reinforcement learning based method that can embed human values judgements into each step of language generation and achieves higher alignment performance in terms of both automatic and human evaluations, but also shows improvements on robustness and transfer learning on unseen human values.



Social Chemistry 101: Learning to Reason about Social and Moral Norms

A new conceptual formalism to study people's everyday social norms and moral judgments over a rich spectrum of real life situations described in natural language and a model framework, Neural Norm Transformer, learns and generalizes Social-Chem-101 to successfully reason about previously unseen situations, generating relevant (and potentially novel) attribute-aware social rules-of-thumb.

Social Bias Frames: Reasoning about Social and Power Implications of Language

It is found that while state-of-the-art neural models are effective at high-level categorization of whether a given statement projects unwanted social bias, they are not effective at spelling out more detailed explanations in terms of Social Bias Frames.

Aligning AI With Shared Human Values

With the ETHICS dataset, it is found that current language models have a promising but incomplete understanding of basic ethical knowledge, and it provides a steppingstone toward AI that is aligned with human values.

Social IQA: Commonsense Reasoning about Social Interactions

It is established that Social IQa, the first large-scale benchmark for commonsense reasoning about social situations, is challenging for existing question-answering models based on pretrained language models, compared to human performance (>20% gap).

Learning Norms from Stories: A Prior for Value Aligned Agents

This work trains multiple machine learning models to classify natural language descriptions of situations found in the comic strip as normative or non-normative by identifying if they align with the main characters' behavior.

Scruples: A Corpus of Community Ethical Judgments on 32, 000 Real-Life Anecdotes

This work introduces SCRUPLES, the first large-scale dataset with 625,000 ethical judgments over 32,000 real-life anecdotes, and presents a new method to estimate the best possible performance on such tasks with inherently diverse label distributions, and explores likelihood functions that separate intrinsic from model uncertainty.

ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning

Experimental results demonstrate that multitask models that incorporate the hierarchical structure of if-then relation types lead to more accurate inference compared to models trained in isolation, as measured by both automatic and human evaluation.

Thinking Like a Skeptic: Defeasible Inference in Natural Language

From Defeasible NLI, both a classification and generation task for defeasible inference are developed, and it is demonstrated that the generation task is much more challenging.

Event2Mind: Commonsense Inference on Events, Intents, and Reactions

It is demonstrated how commonsense inference on people’s intents and reactions can help unveil the implicit gender inequality prevalent in modern movie scripts.

Abductive Commonsense Reasoning

This study introduces a challenge dataset, ART, that consists of over 20k commonsense narrative contexts and 200k explanations, and conceptualizes two new tasks -- Abductive NLI: a multiple-choice question answering task for choosing the more likely explanation, and Abduction NLG: a conditional generation task for explaining given observations in natural language.