Encoding of phonology in a recurrent neural model of grounded speech

@inproceedings{Alishahi2017EncodingOP,
  title={Encoding of phonology in a recurrent neural model of grounded speech},
  author={Afra Alishahi and Marie Barking and Grzegorz Chrupala},
  booktitle={CoNLL},
  year={2017}
}
We study the representation and encoding of phonemes in a recurrent neural network model of grounded speech. We use a model which processes images and their spoken descriptions, and projects the visual and auditory representations into the same semantic space. We perform a number of analyses on how information about individual phonemes is encoded in the MFCC features extracted from the speech signal, and the activations of the layers of the model. Via experiments with phoneme decoding and… CONTINUE READING

Citations

Publications citing this paper.
SHOWING 1-10 OF 23 CITATIONS

Analysis Methods in Neural Language Processing: A Survey

  • Transactions of the Association for Computational Linguistics
  • 2018
VIEW 5 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

Models of Visually Grounded Speech Signal Pay Attention to Nouns: A Bilingual Experiment on English and Japanese

  • ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • 2019
VIEW 4 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

References

Publications referenced by this paper.
SHOWING 1-10 OF 45 REFERENCES

Dynamic Encoding of Acoustic Features in Neural Responses to Continuous Speech

  • The Journal of neuroscience : the official journal of the Society for Neuroscience
  • 2017
VIEW 5 EXCERPTS
HIGHLY INFLUENTIAL

Synthetically spoken COCO

Grzegorz Chrupała, Lieke Gelderloos, Afra Alishahi.
  • https://doi.org/10.5281/zenodo.400926.
  • 2017
VIEW 8 EXCERPTS
HIGHLY INFLUENTIAL

The Kaldi Speech Recognition Toolkit

VIEW 2 EXCERPTS
HIGHLY INFLUENTIAL

NLTK: The Natural Language Toolkit

VIEW 3 EXCERPTS
HIGHLY INFLUENTIAL

Working memory

Alan D Baddeley, Graham Hitch.
  • Psychology of learning and motivation 8:47–89.
  • 1974
VIEW 3 EXCERPTS
HIGHLY INFLUENTIAL

Similar Papers

Loading similar papers…