Correlation based speech-video synchronization

@article{ElSallam2011CorrelationBS,
  title={Correlation based speech-video synchronization},
  author={Amar A. El-Sallam and Ajmal S. Mian},
  journal={Pattern Recognition Letters},
  year={2011},
  volume={32},
  pages={780-786}
}
This paper presents a novel Lip synchronization technique which investigates the correlation between the speech and lips movements. First, the speech signal is represented as a nonlinear time-varying model which involves a sum of AM–FM signals. Each of these signals is employed to model a single Formant frequency. The model is realized using Taylor series expansion in a way which provides the relationship between the lip shape (width and height) w.r.t. the speech amplitude and instantaneous… CONTINUE READING