Jointly Learning to Parse and Perceive: Connecting Natural Language to the Physical World

@article{Krishnamurthy2013JointlyLT,
  title={Jointly Learning to Parse and Perceive: Connecting Natural Language to the Physical World},
  author={Jayant Krishnamurthy and Thomas Kollar},
  journal={Transactions of the Association for Computational Linguistics},
  year={2013},
  volume={1},
  pages={193-206}
}
This paper introduces Logical Semantics with Perception (LSP), a model for grounded language acquisition that learns to map natural language statements to their referents in a physical environment. For example, given an image, LSP can map the statement “blue mug on the table” to the set of image segments showing blue mugs on tables. LSP learns physical representations for both categorical (“blue,” “mug”) and relational (“on”) language, and also learns to compose these representations to produce… CONTINUE READING
BETA

Similar Papers

Citations

Publications citing this paper.
SHOWING 1-10 OF 121 CITATIONS

Toward Interactive Grounded Language Acqusition

  • Robotics: Science and Systems
  • 2013
VIEW 9 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

Learning from Language

VIEW 4 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

A Bayesian Model of Grounded Color Semantics

  • Transactions of the Association for Computational Linguistics
  • 2015
VIEW 4 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

Neural Module Networks

  • 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2015
VIEW 4 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

Unsupervised Alignment of Natural Language Instructions with Video Segments

VIEW 5 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

FILTER CITATIONS BY YEAR

2012
2019

CITATION STATISTICS

  • 10 Highly Influenced Citations

  • Averaged 17 Citations per year from 2017 through 2019

References

Publications referenced by this paper.