Learning to Speak and Act in a Fantasy Text Adventure Game

@article{Urbanek2019LearningTS,
  title={Learning to Speak and Act in a Fantasy Text Adventure Game},
  author={Jack Urbanek and Angela Fan and Siddharth Karamcheti and Saachi Jain and Samuel Humeau and Emily Dinan and Tim Rockt{\"a}schel and Douwe Kiela and Arthur D. Szlam and Jason Weston},
  journal={ArXiv},
  year={2019},
  volume={abs/1903.03094}
}
We introduce a large-scale crowdsourced text adventure game as a research platform for studying grounded dialogue. In it, agents can perceive, emote, and act whilst conducting dialogue with other agents. Models and humans can both act as characters within the game. We describe the results of training state-of-the-art generative and retrieval models in this setting. We show that in addition to using past dialogue, these models are able to effectively use the state of the underlying world to… 
View on ACL
arxiv.org
131 Citations
Highly Influential Citations
17
Background Citations
84
Methods Citations
31
Results Citations
1

131 Citations

How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds

A reinforcement learning system is introduced that incorporates large-scale language modeling-based and commonsense reasoning-based pre-training to imbue the agent with relevant priors and leverages a factorized action space of action commands and dialogue, balancing between the two.

MAKING KNIGHTS SMILE IN A FANTASY GAME WORLD

  • Computer Science
  • 2019
This work trains a goal-oriented model with reinforcement learning via selfplay against an imitation-learned “chit-chat” model with two new approaches and shows that both models outperform a strong inverse model baseline and can converse naturally with their dialogue partner in order to achieve goals.

Situated language learning via interactive narratives

Situated Dialogue Learning through Procedural Environment Generation

An ablation study shows that this method of learning from the tail of a distribution results in significantly higher generalization abilities as measured by zero-shot performance on never-before-seen quests.

Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence

This paper frames D&D specifically as a dialogue system challenge, where the tasks are to both generate the next conversational turn in the game and predict the state of the game given the dialogue history, and creates a gameplay dataset consisting of nearly 900 games.

Dialogue in the Wild: Learning from a Deployed Role-Playing Game with Humans and Bots

This work builds and deploy a role-playing game, whereby human players converse with learning agents situated in an open-domain fantasy world and shows that by training models on the conversations they have with humans in the game the models progressively improve, as measured by automatic metrics and online engagement scores.

Toward Automated Quest Generation in Text-Adventure Games

This paper considers the problem of procedurally generating a quest, defined as a series of actions required to progress towards a goal, in a text-adventure game, and presents and evaluates two quest generation techniques: a Markov model, and a neural generative model.

Modeling Worlds in Text

A dataset that enables the creation of learning agents that can build knowledge graph-based world models of interactive narratives and baseline models using rules-based, question-answering, and sequence learning approaches are provided in addition to an analysis of the data and corresponding learning tasks.

Ontologically Faithful Generation of Non-Player Character Dialogues

KNUDGE (KNowledge Constrained User-NPC Dialogue GEneration) requires models to produce trees of dialogue between video game characters that accurately reflect quest and entity specifications stated in natural language.

BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage

The goal of this research program is to enable the community to study ever-improving responsible agents that learn through interaction and release both the model weights and code, and the plan for continual learning using the data collected from deployment, which will also be publicly released.
...

53 References

TextWorld: A Learning Environment for Text-based Games

TextWorld is a Python library that handles interactive play-through of text games, as well as backend functions like state tracking and reward assignment, and comes with a curated list of games whose features and challenges the authors have analyzed.

Spoken Conversational AI in Video Games: Emotional Dialogue Management Increases User Engagement

A spoken conversational interface is presented, built using a state-of-the-art open-domain social conversational AI developed for the Amazon Alexa Challenge, which was modified for use in a video game.

Engaging Image Chat: Modeling Personality in Grounded Dialogue

This work collects a large dataset of grounded human-human conversations, where humans are asked to play the role of a given personality, as the use of personality in conversation has also been shown to be engaging.

Talk the Walk: Navigating New York City through Grounded Dialogue

This work focuses on the task of tourist localization and develops the novel Masked Attention for Spatial Convolutions (MASC) mechanism that allows for grounding tourist utterances into the guide's map, and shows it yields significant improvements for both emergent and natural language communication.

Game-Based Video-Context Dialogue

This work introduces a new video-context, many-speaker dialogue dataset based on live-broadcast soccer game videos and chats from Twitch.tv, and presents visually-grounded dialogue models that should generate relevant temporal and spatial event language from the live video, while also being relevant to the chat history.

Wizard of Wikipedia: Knowledge-Powered Conversational agents

The best performing dialogue models are able to conduct knowledgeable discussions on open-domain topics as evaluated by automatic metrics and human evaluations, while a new benchmark allows for measuring further improvements in this important research direction.

Training Millions of Personalized Dialogue Agents

A new dataset providing 5 million personas and 700 million persona-based dialogues is introduced and it is shown that, at this scale, training using personas still improves the performance of end-to-end systems.

Language Understanding for Text-based Games using Deep Reinforcement Learning

This paper employs a deep reinforcement learning framework to jointly learn state representations and action policies using game rewards as feedback to map text descriptions into vector representations that capture the semantics of the game states.

Learning Language Games through Interaction

A new language learning setting relevant to building adaptive natural language interfaces inspired by Wittgenstein's language games is introduced, showing that using compositionality and avoiding synonyms correlates positively with task performance and that modeling pragmatics further accelerates learning for successful players.

Personalizing Dialogue Agents: I have a dog, do you have pets too?

This work collects data and train models tocondition on their given profile information; and information about the person they are talking to, resulting in improved dialogues, as measured by next utterance prediction.
...