Corpus ID: 236318121

Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives

  title={Mixed SIGNals: Sign Language Production via a Mixture of Motion Primitives},
  author={Ben Saunders and Necati Cihan Camgoz and R. Bowden},
It is common practice to represent spoken languages at their phonetic level. However, for sign languages, this implies breaking motion into its constituent motion primitives. Avatar based Sign Language Production (SLP) has traditionally done just this, building up animation from sequences of hand motions, shapes and facial expressions. However, more recent deep learning based solutions to SLP have tackled the problem using a single network that estimates the full skeletal structure. We propose… Expand

Figures and Tables from this paper


Neural Sign Language Translation
This work formalizes SLT in the framework of Neural Machine Translation (NMT) for both end-to-end and pretrained settings (using expert knowledge) and allows to jointly learn the spatial representations, the underlying language model, and the mapping between sign and spoken language. Expand
Sign Language Production using Neural Machine Translation and Generative Adversarial Networks
A novel approach to automatic Sign Language Production using state-of-the-art Neural Machine Translation (NMT) and Image Generation techniques, capable of producing sign videos from spoken language sentences. Expand
Everybody Sign Now: Translating Spoken Language to Photo Realistic Sign Language Video
This paper proposes SignGAN, the first SLP model to produce photo-realistic continuous sign language videos directly from spoken language, and introduces a method for controllable video generation, enabling training on large, diverse sign language datasets and providing the ability to control the signer appearance at inference. Expand
Text2Sign: Towards Sign Language Production Using Neural Machine Translation and Generative Adversarial Networks
This work presents a novel approach to automatic Sign Language Production using recent developments in Neural Machine Translation, Generative Adversarial Networks, and motion generation to produce sign videos from spoken language sentences. Expand
Deep Gesture Video Generation With Learning on Regions of Interest
Experimental results show that the method can produce gesture videos with more realistic appearance and precise meaning than the state-of-the-art video generation approaches. Expand
Adversarial Training for Multi-Channel Sign Language Production
An adversarial discriminator evaluates the realism of sign production conditioned on the source text, pushing the generator towards a realistic and articulate output, and set new benchmarks for the production of multi-channel sign to underpin future research into realistic SLP. Expand
Bridging the gap between sign language machine translation and sign language animation using sequence classification
This paper presents work that bridges the gap between the output of a sign language translation system and the input of a signing language animation system by incorporating non-manual information into the final output of the translation system. Expand
Sign Language Transformers: Joint End-to-End Sign Language Recognition and Translation
A novel transformer based architecture that jointly learns Continuous Sign Language Recognition and Translation while being trainable in an end-to-end manner is introduced by using a Connectionist Temporal Classification (CTC) loss to bind the recognition and translation problems into a single unified architecture. Expand
Weakly Supervised Learning with Multi-Stream CNN-LSTM-HMMs to Discover Sequential Parallelism in Sign Language Videos
This work applies the approach to the domain of sign language recognition exploiting the sequential parallelism to learn sign language, mouth shape and hand shape classifiers and clearly outperform the state-of-the-art on all data sets and observe significantly faster convergence using the parallel alignment approach. Expand
Neural Sign Language Synthesis: Words Are Our Glosses
  • J. Zelinka, J. Kanis
  • Computer Science
  • 2020 IEEE Winter Conference on Applications of Computer Vision (WACV)
  • 2020
The main goal in this paper was to design a fully end-to-end automatic sign language synthesis system trained only on available free data (daily TV broadcasting) and excluded any manual video annotation. Expand