Text Normalization using Memory Augmented Neural Networks

@article{Pramanik2019TextNU,
  title={Text Normalization using Memory Augmented Neural Networks},
  author={Subhojeet Pramanik and Aman Hussain},
  journal={Speech Commun.},
  year={2019},
  volume={109},
  pages={15-23}
}
Abstract We perform text normalization, i.e. the transformation of words from the written to the spoken form, using a memory augmented neural network. With the addition of dynamic memory access and storage mechanism, we present a neural architecture that will serve as a language-agnostic text normalization system while avoiding the kind of unacceptable errors made by the LSTM-based recurrent neural networks. By successfully reducing the frequency of such mistakes, we show that this novel… Expand
Neural Models of Text Normalization for Speech Applications
TLDR
Neural network models that treat text normalization for TTS as a sequence-to-sequence problem, in which the input is a text token in context, and the output is the verbalization of that token, are proposed. Expand
Neural Models of Text Normalization for Speech Applications
Machine learning, including neural network techniques, have been applied to virtually every domain in natural language processing. One problem that has been somewhat resistant to effective machineExpand
Neural Inverse Text Normalization
TLDR
This work proposes an efficient and robust neural solution for ITN leveraging transformer based seq2seq models and FST-based text normalization techniques for data preparation and shows that this can be easily extended to other languages without the need for a linguistic expert to manually curate them. Expand
The Roles of Language Models and Hierarchical Models in Neural Sequence-to-Sequence Prediction
TLDR
It is shown how traditional symbolic statistical machine translation models can still improve neural machine translation while reducing the risk of common pathologies of NMT such as hallucinations and neologisms. Expand
NeMo Inverse Text Normalization: From Development To Production
TLDR
An open-source Python WFST-based library for ITN is introduced which enables a seamless path from development to production and the specification of ITN grammar rules for English is described, but the library can be adapted for other languages. Expand
Dual Encoder Classifier Models as Constraints in Neural Text Normalization
TLDR
This paper explores an alternative approach to error mitigation: using dual encoder classifiers trained with both positive and negative examples to implement soft constraints on acceptability. Expand
A Hybrid Method for Vietnamese Text Normalization
TLDR
A hybrid method for normalizing written text often found on newspapers to its spoken form and can predict effectively the expansions of abbreviations in context. Expand
Numbers Normalisation in the Inflected Languages: a Case Study of Polish
TLDR
This paper proposes a combination of morphological tagger and transducer supported by a dictionary of numbers in their spoken forms for text normalisation in Polish without having a large dedicated data set and using solutions prepared for other NLP tasks. Expand
Neural Machine Translation: A Review
TLDR
This work traces back the origins of modern NMT architectures to word and sentence embeddings and earlier examples of the encoder-decoder network family and concludes with a survey of recent trends in the field. Expand
Creating Data in Icelandic for Text Normalization
TLDR
This work manually annotated the first normalized corpus in Icelandic, 40,000 sentences, and developed Regína, a rule-based system for text normalization, a significant improvement in accuracy when compared to an older normalization system for Icelandic. Expand
...
1
2
...

References

SHOWING 1-10 OF 38 REFERENCES
RNN Approaches to Text Normalization: A Challenge
TLDR
A data set of general text where the normalizations were generated using an existing text normalization component of a text-to-speech system is presented, and it is shown that a simple FST-based filter can mitigate errors, and achieve a level of accuracy not achievable by the RNN alone. Expand
One-shot Learning with Memory-Augmented Neural Networks
TLDR
The ability of a memory-augmented neural network to rapidly assimilate new data, and leverage this data to make accurate predictions after only a few samples is demonstrated. Expand
Sequence to Sequence Learning with Neural Networks
TLDR
This paper presents a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure, and finds that reversing the order of the words in all source sentences improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier. Expand
Effective Approaches to Attention-based Neural Machine Translation
TLDR
A global approach which always attends to all source words and a local one that only looks at a subset of source words at a time are examined, demonstrating the effectiveness of both approaches on the WMT translation tasks between English and German in both directions. Expand
A Mostly Data-Driven Approach to Inverse Text Normalization
TLDR
This work presents a mostly data-driven ITN system that leverages a set of simple rules and a few handcrafted grammars to cast ITN as a labeling problem and applies a compact bi-directional LSTM to this labeling problem. Expand
Hybrid computing using a neural network with dynamic external memory
TLDR
A machine learning model called a differentiable neural computer (DNC), which consists of a neural network that can read from and write to an external memory matrix, analogous to the random-access memory in a conventional computer. Expand
Long Short-Term Memory
TLDR
A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Expand
Neural Machine Translation by Jointly Learning to Align and Translate
TLDR
It is conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and it is proposed to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly. Expand
Normalization of non-standard words
TLDR
A taxonomy of NSWs was developed on the basis of four rather distinct text types, and several general techniques including n-gram language models, decision trees and weighted finite-state transducers were investigated, demonstrating that a systematic treatment can lead to better results than have been obtained by the ad hoc treatments that have typically been used in the past. Expand
Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets
TLDR
The limitations of standard deep learning approaches are discussed and it is shown that some of these limitations can be overcome by learning how to grow the complexity of a model in a structured way. Expand
...
1
2
3
4
...