• Publications
  • Influence
Theano: A Python framework for fast computation of mathematical expressions
TLDR
The performance of Theano is compared against Torch7 and TensorFlow on several machine learning models and recently-introduced functionalities and improvements are discussed.
Attention-Based Models for Speech Recognition
TLDR
The attention-mechanism is extended with features needed for speech recognition and a novel and generic method of adding location-awareness to the attention mechanism is proposed to alleviate the issue of high phoneme error rate.
Deep Complex Networks
TLDR
This work relies on complex convolutions and present algorithms for complex batch-normalization, complex weight initialization strategies for complex-valued neural nets and uses them in experiments with end-to-end training schemes and demonstrates that such complex- valued models are competitive with their real-valued counterparts.
End-to-end attention-based large vocabulary speech recognition
TLDR
This work investigates an alternative method for sequence modelling based on an attention mechanism that allows a Recurrent Neural Network (RNN) to learn alignments between sequences of input frames and output labels.
Towards End-to-end Spoken Language Understanding
TLDR
This study showed that the trained model can achieve reasonable good result and demonstrated that the model can capture the semantic attention directly from the audio features.
Twin Networks: Matching the Future for Sequence Generation
TLDR
This work proposes a simple technique for encouraging generative RNNs to plan ahead, and hypothesizes that this approach eases modeling of long-term dependencies by implicitly forcing the forward states to hold information about the longer-term future (as contained in the backward states).
Blocks and Fuel: Frameworks for deep learning
TLDR
This work introduces two Python frameworks to train neural networks on large datasets: Blocks and Fuel, which provides a standard format for machine learning datasets.
Unsupervised adversarial domain adaptation for acoustic scene classification
TLDR
The first method of unsupervised adversarial domain adaptation for acoustic scene classification is presented, which employs a model pre-trained on data from one set of conditions and by using data from other set of Conditions, which adapt the model in order that its output cannot be used for classifying the set of condition that input data belong to.
Fortified Networks: Improving the Robustness of Deep Networks by Modeling the Manifold of Hidden Representations
TLDR
Fortified Networks is proposed, a simple transformation of existing networks, which fortifies the hidden layers in a deep network by identifying when the hidden states are off of the data manifold, and maps these hidden states back to parts of theData manifold where the network performs well.
Task Loss Estimation for Sequence Prediction
TLDR
This work proposes another method for deriving differentiable surrogate losses that provably meet the requirement of consistency with the task loss, and focuses on the broad class of models that define a score for every input-output pair.
...
...