Learning words from sights and sounds: a computational model

  title={Learning words from sights and sounds: a computational model},
  author={Deb Roy and Alex Pentland},
  journal={Cognitive Science},
This paper presents an implemented computational model of word acquisition which learns directly from raw multimodal sensory input. Set in an information theoretic framework, the model acquires a lexicon by finding and statistically modeling consistent cross-modal structure. The model has been implemented in a system using novel speech processing, computer vision, and machine learning algorithms. In evaluations the model successfully performed speech segmentation, word discovery and visual… CONTINUE READING
Highly Influential
This paper has highly influenced 29 other papers. REVIEW HIGHLY INFLUENTIAL CITATIONS
Highly Cited
This paper has 1,342 citations. REVIEW CITATIONS