Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition

  title={Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition},
  author={G. Dahl and Dong Yu and L. Deng and A. Acero},
  journal={IEEE Transactions on Audio, Speech, and Language Processing},
  • G. Dahl, Dong Yu, +1 author A. Acero
  • Published 2012
  • Computer Science
  • IEEE Transactions on Audio, Speech, and Language Processing
  • We propose a novel context-dependent (CD) model for large-vocabulary speech recognition (LVSR) that leverages recent advances in using deep belief networks for phone recognition. We describe a pre-trained deep neural network hidden Markov model (DNN-HMM) hybrid architecture that trains the DNN to produce a distribution over senones (tied triphone states) as its output. The deep belief network pre-training algorithm is a robust and often helpful way to initialize deep neural networks… CONTINUE READING
    2,464 Citations
    Pipelined Back-Propagation for Context-Dependent Deep Neural Networks
    • 70
    • PDF
    Standalone training of context-dependent deep neural network acoustic models
    • C. Zhang, P. Woodland
    • Computer Science
    • 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
    • 2014
    • 30
    • PDF
    Context-dependent deep neural networks for commercial Mandarin speech recognition applications
    • J. Niu, L. Xie, Lei Jia, Na Hu
    • Computer Science
    • 2013 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference
    • 2013
    • 4
    • PDF
    Deep neural network acoustic modeling for native and non-native Mandarin speech recognition
    • X. Chen, J. Cheng
    • Computer Science
    • The 9th International Symposium on Chinese Spoken Language Processing
    • 2012
    • 11
    • 28
    • PDF
    KL-divergence regularized deep neural network adaptation for improved large vocabulary speech recognition
    • 364
    • PDF
    Hybrid context dependent CD-DNN-HMM Keyword Spotting (KWS) in speech conversations
    • V. Tyagi
    • Computer Science
    • 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP)
    • 2016
    • 6
    Application of Pretrained Deep Neural Networks to Large Vocabulary Speech Recognition
    • 229
    • PDF


    Roles of Pre-Training and Fine-Tuning in Context-Dependent DBN-HMMs for Real-World Speech Recognition
    • 194
    • PDF
    Context-dependent connectionist probability estimation in a hybrid hidden Markov model-neural net speech recognition system
    • 56
    • Highly Influential
    Large vocabulary continuous speech recognition with context-dependent DBN-HMMS
    • 182
    • PDF
    CDNN: a context dependent neural network for continuous speech recognition
    • 82
    • PDF
    Investigation of full-sequence training of deep belief networks for speech recognition
    • 210
    • PDF
    Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine
    • 293
    • PDF
    A segmental CRF approach to large vocabulary continuous speech recognition
    • G. Zweig, P. Nguyen
    • Computer Science
    • 2009 IEEE Workshop on Automatic Speech Recognition & Understanding
    • 2009
    • 129
    • Highly Influential
    • PDF
    Speech Recognition Using Augmented Conditional Random Fields
    • 92
    • Highly Influential
    • PDF
    Deep Belief Networks for phone recognition
    • 367
    • Highly Influential
    • PDF
    Discriminative Training for Large-Vocabulary Speech Recognition Using Minimum Classification Error
    • 606
    • PDF