Corpus ID: 12534863

Learning Interpretable Spatial Operations in a Rich 3D Blocks World

@inproceedings{Bisk2018LearningIS,
  title={Learning Interpretable Spatial Operations in a Rich 3D Blocks World},
  author={Yonatan Bisk and K. Shih and Yejin Choi and D. Marcu},
  booktitle={AAAI},
  year={2018}
}
  • Yonatan Bisk, K. Shih, +1 author D. Marcu
  • Published in AAAI 2018
  • Computer Science
  • In this paper, we study the problem of mapping natural language instructions to complex spatial actions in a 3D blocks world. We first introduce a new dataset that pairs complex 3D spatial operations to rich natural language descriptions that require complex spatial and pragmatic interpretations such as "mirroring", "twisting", and "balancing". This dataset, built on the simulation environment of Bisk, Yuret, and Marcu (2016), attains language that is significantly richer and more complex… CONTINUE READING
    33 Citations

    Figures, Tables, and Topics from this paper.

    Robust and Interpretable Grounding of Spatial References with Relation Networks
    Photo-Realistic Blocksworld Dataset
    • 1
    • PDF
    Points, Paths, and Playscapes: Large-scale Spatial Language Understanding Tasks Set in the Real World
    • 6
    • Highly Influenced
    • PDF
    SPARE3D: A Dataset for SPAtial REasoning on Three-View Line Drawings
    • 1
    • PDF
    Computational Models for Spatial Prepositions
    • 6
    • PDF
    Prospection: Interpretable plans from language by predicting the future
    • 9
    • PDF
    RAVEN: A Dataset for Relational and Analogical Visual REasoNing
    • 30
    • PDF

    References

    SHOWING 1-10 OF 39 REFERENCES
    Source-Target Inference Models for Spatial Instruction Understanding
    • 6
    • PDF
    Generation and Comprehension of Unambiguous Object Descriptions
    • 370
    • PDF
    Toward Interactive Grounded Language Acqusition
    • 34
    • PDF
    Natural Language Communication with Robots
    • 62
    • PDF
    ReferItGame: Referring to Objects in Photographs of Natural Scenes
    • 379
    • PDF
    Mapping Instructions and Visual Observations to Actions with Reinforcement Learning
    • 114
    • PDF
    Weakly Supervised Learning of Semantic Parsers for Mapping Instructions to Actions
    • 352
    • PDF
    Learning visually grounded words and syntax for a scene description task
    • D. Roy
    • Computer Science
    • Comput. Speech Lang.
    • 2002
    • 221
    • PDF
    Grounding spatial relations for human-robot interaction
    • 111
    • PDF