Corpus ID: 49669712

Talk the Walk: Navigating New York City through Grounded Dialogue

  title={Talk the Walk: Navigating New York City through Grounded Dialogue},
  author={H. D. Vries and Kurt Shuster and Dhruv Batra and D. Parikh and J. Weston and Douwe Kiela},
  • H. D. Vries, Kurt Shuster, +3 authors Douwe Kiela
  • Published 2018
  • Computer Science
  • ArXiv
  • We introduce "Talk The Walk", the first large-scale dialogue dataset grounded in action and perception. The task involves two agents (a "guide" and a "tourist") that communicate via natural language in order to achieve a common goal: having the tourist navigate to a given target location. The task and dataset, which are described in detail, are challenging and their full solution is an open problem that we pose to the community. We (i) focus on the task of tourist localization and develop the… CONTINUE READING
    65 Citations

    Figures, Tables, and Topics from this paper

    Natural Language Grounded Multitask Navigation
    • PDF
    Learning to Speak and Act in a Fantasy Text Adventure Game
    • 45
    • PDF
    RMM: A Recursive Mental Model for Dialog Navigation
    • 3
    • PDF
    Vision-and-Dialog Navigation
    • 45
    • PDF
    AirDialogue: An Environment for Goal-Oriented Dialogue Research
    • 25
    • Highly Influenced
    • PDF
    Talk2Nav: Long-Range Vision-and-Language Navigation in Cities
    • 2
    Situated and Interactive Multimodal Conversations
    • 2
    • PDF
    Where Are You? Localization from Embodied Dialog
    • PDF
    TOUCHDOWN: Natural Language Navigation and Spatial Reasoning in Visual Street Environments
    • 100
    • PDF


    Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences
    • 146
    • PDF
    Visual Dialog
    • A. Das, S. Kottur, +5 authors Dhruv Batra
    • Computer Science
    • 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
    • 2017
    • 328
    • PDF
    GuessWhat?! Visual Object Discovery through Multi-modal Dialogue
    • 229
    • PDF
    Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments
    • 312
    • Highly Influential
    • PDF
    Walk the Talk: Connecting Language, Knowledge, and Action in Route Instructions
    • 315
    • PDF
    Embodied Question Answering
    • 22
    Multi-Agent Cooperation and the Emergence of (Natural) Language
    • 209
    • PDF
    Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
    • 283
    • PDF
    Emergent Language in a Multi-Modal, Multi-Step Referential Game
    • 29
    • PDF