• Corpus ID: 231749743

SPEAK WITH YOUR HANDS - Using Continuous Hand Gestures to control Articulatory Speech Synthesizer

@article{Saha2021SPEAKWY,
  title={SPEAK WITH YOUR HANDS - Using Continuous Hand Gestures to control Articulatory Speech Synthesizer},
  author={Pramit Saha and Debasis Mohapatra and Sidney S. Fels},
  journal={ArXiv},
  year={2021},
  volume={abs/2102.01640}
}
This work presents our advancements in controlling an articulatory speech synthesis engine, viz., Pink Trombone, with hand gestures. Our interface translates continuous finger movements and wrist flexion into continuous speech using vocal tract area-function based articulatory speech synthesis. We use Cyberglove II with 18 sensors to capture the kinematic information of the wrist and the individual fingers, in order to control a virtual tongue. The coordinates and the bending values of the… 

Figures from this paper

References

SHOWING 1-4 OF 4 REFERENCES

Glove-TalkII-a neural-network interface which maps gestures to parallel formant speech synthesizer controls

One subject has trained to speak intelligibly with Glove-TalkII, a system which translates hand gestures to speech through an adaptive interface that uses several input devices, a parallel formant speech synthesizer, and three neural networks.

Sound-Stream II: Towards Real-Time Gesture Controlled Articulatory Sound Synthesis

We present an interface involving four degrees-of-freedom (DOF) mechanical control of a two dimensional, mid-sagittal tongue through a biomechanical toolkit called ArtiSynth and a sound synthesis

Vocal tract modeling: implementation of continuous length variations in a half-sample delay Kelly-Lochbaum model

  • S. MathurB. Story
  • Geology
    Proceedings of the 3rd IEEE International Symposium on Signal Processing and Information Technology (IEEE Cat. No.03EX795)
  • 2003
Algorithms for modeling static vowel shapes using a half-sample delay Kelly-Lochbaum model are described, so that the total tract length is no longer limited to an integer multiple of the segment

Pink Trombone

  • https : / / dood . al / pinktrombone/. version 1.1.
  • 2017