Lightweight and Efficient End-to-End Speech Recognition Using Low-Rank Transformer

@article{Winata2019LightweightAE,
  title={Lightweight and Efficient End-to-End Speech Recognition Using Low-Rank Transformer},
  author={Genta Indra Winata and Samuel Cahyawijaya and Zhaojiang Lin and Zihan Liu and Pascale Fung},
  journal={ArXiv},
  year={2019},
  volume={abs/1910.13923}
}
  • Genta Indra Winata, Samuel Cahyawijaya, +2 authors Pascale Fung
  • Published 2019
  • Computer Science, Engineering
  • ArXiv
  • Highly performing deep neural networks come at the cost of computational complexity that limits their practicality for deployment on portable devices. We propose the low-rank transformer (LRT), a memory-efficient and fast neural architecture that significantly reduces the parameters and boosts the speed of training and inference for end-to-end speech recognition. Our approach reduces the number of parameters of the network by more than 50% and speeds up the inference time by around 1.35x… CONTINUE READING

    Citations

    Publications citing this paper.

    Learning Fast Adaptation on Cross-Accented Speech Recognition

    VIEW 5 EXCERPTS
    CITES METHODS

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 20 REFERENCES

    Low-rank matrix factorization for Deep Neural Network training with high-dimensional output targets

    VIEW 1 EXCERPT

    Joint CTC-attention based end-to-end speech recognition using multi-task learning

    VIEW 1 EXCERPT

    End-to-end Speech Recognition with Adaptive Computation Steps

    • Mohan Li, Min Liu, Masanori Hattori
    • Computer Science, Engineering
    • ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
    • 2019

    Speech-Transformer: A No-Recurrence Sequence-to-Sequence Model for Speech Recognition

    • Linhao Dong, Shuang Xu, Bo Xu
    • Computer Science
    • 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
    • 2018
    VIEW 2 EXCERPTS

    An empirical exploration of CTC acoustic models

    The Speechtransformer for Large-scale Mandarin Chinese Speech Recognition

    VIEW 2 EXCERPTS

    Building Competitive Direct Acoustics-to-Word Models for English Conversational Speech Recognition

    VIEW 1 EXCERPT