Word Recognition, Competition, and Activation in a Model of Visually Grounded Speech

@inproceedings{Havard2019WordRC,
  title={Word Recognition, Competition, and Activation in a Model of Visually Grounded Speech},
  author={William N. Havard and Jean-Pierre Chevrot and L. Besacier},
  booktitle={CoNLL},
  year={2019}
}
In this paper, we study how word-like units are represented and activated in a recurrent neural model of visually grounded speech. The model used in our experiments is trained to project an image and its spoken description in a common representation space. We show that a recurrent model trained on spoken sentences implicitly segments its input into word-like units and reliably maps them to their correct visual referents. We introduce a methodology originating from linguistics to analyse the… Expand
Learning to Recognise Words Using Visually Grounded Speech
Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech
Text-Free Image-to-Speech Synthesis Using Learned Segmental Units
Semantic sentence similarity: size does not always matter
Grounding ‘Grounding’ in NLP
Talk, Don't Write: A Study of Direct Speech-Based Image Retrieval
Visually grounded models of spoken language: A survey of datasets, architectures and evaluation techniques

References

SHOWING 1-10 OF 28 REFERENCES
Encoding of phonology in a recurrent neural model of grounded speech
Representation of Linguistic Form and Function in Recurrent Neural Networks
Representations of language in a model of visually grounded speech signal
Models of Visually Grounded Speech Signal Pay Attention to Nouns: A Bilingual Experiment on English and Japanese
Semantic Speech Retrieval With a Visually Grounded Model of Untranscribed Speech
Language learning using Speech to Image retrieval
Spoken Word Recognition
Spoken word recognition processes and the gating paradigm
  • F. Grosjean
  • Computer Science, Medicine
  • Perception & psychophysics
  • 1980
Processing interactions and lexical access during word recognition in continuous speech
...
1
2
3
...