Filters for Efficient Composition of Weighted Finite-State Transducers

  title={Filters for Efficient Composition of Weighted Finite-State Transducers},
  author={Cyril Allauzen and Michael Riley and Johan Schalkwyk},
This paper describes a weighted finite-state transducer composition algorithm that generalizes the concept of the composition filter and presents various filters that process epsilon transitions, look-ahead along paths, and push forward labels along epsilon paths. These filters, either individually or in combination, make it possible to compose some transducers much more efficiently in time and space than otherwise possible. We present examples of this drawn, in part, from demanding… 
Dynamic Grammars with Lookahead Composition for WFST-based Speech Recognition
A novel, generic approach to dynamic grammar handling in the context of the Weighted Finite-State Transducer (WFST) paradigm, which relies on a straightforward extension of the lexicon and underlying grammar components, and leverages the ideas of on-the-fly composition and delayed construction to efficiently generate the recognition search space on- the-fly.
A Pushdown Transducer Extension for the OpenFst Library
This work presents several weighted pushdown algorithms, some with clear finite-state analogues, describe their library usage and give some applications of these methods to recognition, parsing and translation.
Hierarchical Phrase-based Translation Representations
This paper compares several translation representations for a synchronous context-free grammar parse including CFGs/hypergraphs, finite-state automata (FSA), and pushdown automata, and introduces a two-pass search strategy which is analyzed in terms of search errors and translation performance.
Pushdown Automata in Statistical Machine Translation
The use of pushdown automata (PDA) in the context of statistical machine translation and alignment under a synchronous context-free grammar and a two-pass decoding strategy involving a weaker language model in the first-pass is proposed to address the results of PDA complexity analysis.
A comparative analysis of dynamic network decoding
This paper investigates the properties of two well-known search strategies for dynamic network decoding, namely history conditioned tree search and WFST-based search using dynamic transducer composition and analyzes the impact of the differences in search graph representation, search space structure, and language model look-ahead techniques.
of the Workshop Workshop on Trends in Tree Automata and Tree Transducers
An algorithm for computing the N best roots of a weighted hypergraph is proposed, in which the weight function is given over an idempotent and multiplicatively monotone semiring, and it is proved that the proposed algorithm is correct.
Efficient algorithm for rational kernel evaluation in large lattice sets
  • J. Svec, P. Ircing
  • Computer Science
    2013 IEEE International Conference on Acoustics, Speech and Signal Processing
  • 2013
The described algorithm is optimized for processing speed and thus facilitates the usage of state-of-the-art machine learning techniques like Support Vector Machines even in the real-time application of speech and language processing, such as dialogue systems and speech retrieval engines.
The Kestrel TTS text normalization system
The architecture of Kestrel, the protocol buffer representations of semiotic classes, and some examples of grammars for various languages are described, which are used by millions of people in nineteen languages and counting.
Silence is golden: Modeling non-speech events in WFST-based dynamic network decoders
Several options for the transducer construction with multiple non-speech models are described, their considerable different characteristics in memory and runtime efficiency are shown, and the impact on the recognition performance is analyzed.
Hierarchical hybrid language models for open vocabulary continuous speech recognition using WFST
This work augments the first level hybrid language model with an OOV word class, which is replaced by character level graphone sequences using a second-level graphone based character language and acoustic model during search.


An algorithm for fast composition of weighted finite-state transducers
This work shows how the size of the decoding graph can be reduced and the necessity of determinizing it can be eliminated by removing the ambiguity associated with transitions to the backoff state or states in G, and shows how this construction can be avoided entirely by performing fast on-the-fly composition of HC and L o G.
A Generalized Dynamic Composition Algorithm of Weighted Finite State Transducers for Large Vocabulary Speech Recognition
We propose a generalized dynamic composition algorithm of weighted finite state transducers (WFST), which avoids the creation of noncoaccessible paths, performs weight look-ahead and does not impose
A specialized on-the-fly algorithm for lexicon and language model composition
This paper presents an algorithm for the composition of weighted finite-state transducers which is specially tailored to speech recognition applications: it composes the lexicon with the language
OpenFst: A General and Efficient Weighted Finite-State Transducer Library
We describe OpenFst, an open-source library for weighted finite-state transducers (WFSTs). OpenFst consists of a C++ template library with efficient WFST representations and over twenty-five
Implementation and evaluation of fast on-the-fly WFST composition algorithms
The results show that when using on-the-fly composition with a fully dynamically composed language model component the performance degrades substantially even when avoiding dead-end states, and in these cases the recognition performance can be dramatically improved with the addition of dynamic pushing and state sharing.
Speech Recognition with Weighted Finite-State Transducers
General algorithms for building and optimizing transducer models are presented, including composition for combining models, weighted determinization and minimization for optimizing time and space requirements, and a weight pushing algorithm for redistributing transition weights optimally for speech recognition.
Statistical Modeling for Unit Selection in Speech Synthesis
A general statistical modeling framework for unit selection inspired by automatic speech recognition is introduced and techniques based on that framework can result in a more accurate unit selection, thereby improving the general quality of a speech synthesizer.
General Algorithms for Testing the Ambiguity of Finite Automata
Efficient algorithms for testing the finite, polynomial, and exponential ambiguity of finite automata with i¾?-transitions and an application of these algorithms to an approximate computation of the entropy of a probabilistic automaton are presented.
Finding Common Motifs with Gaps Using Finite Automata
We present an algorithm that uses finite automata to find the common motifs with gaps occurring in all strings belonging to a finite set S = {S1,S2,...,Sr}. In order to find these common motifs we
Semiring Frameworks and Algorithms for Shortest-Distance Problems
  • M. Mohri
  • Computer Science, Mathematics
    J. Autom. Lang. Comb.
  • 2002
A generic algorithm for finding single-source shortest distances in a weighted directed graph when the weights satisfy the conditions of the general semiring framework is given.