Robot Object Retrieval with Contextual Natural Language Queries

@article{Nguyen2020RobotOR,
  title={Robot Object Retrieval with Contextual Natural Language Queries},
  author={Thao Nguyen and Nakul Gopalan and R. Patel and Matt Corsaro and Ellie Pavlick and Stefanie Tellex},
  journal={ArXiv},
  year={2020},
  volume={abs/2006.13253}
}
  • Thao Nguyen, Nakul Gopalan, +3 authors Stefanie Tellex
  • Published 2020
  • Computer Science
  • ArXiv
  • Natural language object retrieval is a highly useful yet challenging task for robots in human-centric environments. Previous work has primarily focused on commands specifying the desired object’s type such as “scissors” and/or visual attributes such as “red,” thus limiting the robot to only known object classes. We develop a model to retrieve objects based on descriptions of their usage. The model takes in a language command containing a verb, for example “Hand me something to cut,” and RGB… CONTINUE READING

    Figures, Tables, and Topics from this paper.

    Practical Cross-modal Manifold Alignment for Grounded Language

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 26 REFERENCES
    Grounding Language Attributes to Objects using Bayesian Eigenobjects
    8
    Natural Language Object Retrieval
    299
    Interactive Visual Grounding of Referring Expressions for Human-Robot Interaction
    37
    Jointly Learning to Parse and Perceive: Connecting Natural Language to the Physical World
    147
    Mining semantic affordances of visual object categories
    50
    Reducing errors in object-fetching interactions through social feedback
    39
    Deep Learning based Object Recognition for Robot picking task
    2
    Interactively Picking Real-World Objects with Unconstrained Spoken Language Instructions
    37
    Affordance detection of tool parts from geometric features
    126
    Microsoft COCO: Common Objects in Context
    9863