Skip to search formSkip to main contentSkip to account menu
End-to-End Audiovisual Speech Recognition
TLDR
This is the first audiovisual fusion model which simultaneously learns to extract features directly from the image pixels and audio waveforms and performs within-context word recognition on a large publicly available dataset (LRW).
View on IEEE
Lipreading Using Temporal Convolutional Networks
TLDR
It is shown that the current state-of-the-art methodology produces models that do not generalize well to variations on the sequence length, and this work addresses this issue by proposing a variable-length augmentation.
View on IEEE
Realistic Speech-Driven Facial Animation with GANs
TLDR
This work presents an end-to-end system that generates videos of a talking head, using only a still image of a person and an audio clip containing speech, without relying on handcrafted intermediate features.
View on Springer
End-to-End Speech-Driven Facial Animation with Temporal GANs
TLDR
This work presents a system for generating videos of a talking head, using a still image of a person and an audio clip containing speech, that doesn't rely on any handcrafted intermediate features and is the first method capable of generating subject independent realistic videos directly from raw audio.
View PDF on arXiv
The MAHNOB Laughter database
View via Publisher
Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection
TLDR
Extensive experiments show that this simple approach significantly surpasses the state-of-the-art in terms of generalisation to unseen manipulations and robustness to perturbations, as well as shed light on the factors responsible for its performance.
View on IEEE
End-to-End Speech-Driven Realistic Facial Animation with Temporal GANs
TLDR
This work presents a system for generating videos of a talking head, using a still image of a person and an audio clip containing speech, that does not rely on any handcrafted intermediate features and is the first method capable of generating subject independent realistic videos directly from raw audio.
View Paper
Video-Driven Speech Reconstruction using Generative Adversarial Networks
TLDR
This paper presents an end-to-end temporal model capable of directly synthesising audio from silent video, without needing to transform to-and-from intermediate features, based on GANs.
View PDF on arXiv
End-to-End Multi-View Lipreading
TLDR
This work presents an end-to-end multi-view lipreading system based on Bidirectional Long-Short Memory (BLSTM) networks, believed to be the first model which simultaneously learns to extract features directly from the pixels and performs visual speech classification from multiple views and also achieves state-of-the-art performance.
View PDF on arXiv
Audiovisual Discrimination Between Speech and Laughter: Why and When Visual Information Might Help
TLDR
An audiovisual approach to distinguishing laughter from speech is presented, and it is shown that integrating the information from audio and video channels may lead to improved performance over single-modal approaches.
View on IEEE
...
...
By clicking accept or continuing to use the site, you agree to the terms outlined in our Privacy Policy, Terms of Service, and Dataset License