• Publications
  • Influence
Linguistic Knowledge and Transferability of Contextual Representations
TLDR
It is found that linear models trained on top of frozen contextual representations are competitive with state-of-the-art task-specific models in many cases, but fail on tasks requiring fine-grained linguistic knowledge. Expand
Synthetic and Natural Noise Both Break Neural Machine Translation
TLDR
It is found that a model based on a character convolutional neural network is able to simultaneously learn representations robust to multiple kinds of noise, including structure-invariant word representations and robust training on noisy texts. Expand
Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks
TLDR
This work proposes a framework that facilitates better understanding of the encoded representations of sentence vectors and demonstrates the potential contribution of the approach by analyzing different sentence representation mechanisms. Expand
What do Neural Machine Translation Models Learn about Morphology?
TLDR
This work analyzes the representations learned by neural MT models at various levels of granularity and empirically evaluates the quality of the representations for learning morphology through extrinsic part-of-speech and morphological tagging tasks. Expand
Analysis Methods in Neural Language Processing: A Survey
TLDR
Analysis methods in neural language processing are reviewed, categorize them according to prominent research trends, highlight existing limitations, and point to potential directions for future work. Expand
End-to-End Bias Mitigation by Modelling Biases in Corpora
TLDR
This work proposes two learning strategies to train neural models, which are more robust to such biases and transfer better to out-of-domain datasets and better transfer to other textual entailment datasets. Expand
Arabic Diacritization with Recurrent Neural Networks
TLDR
This work develops a recurrent neural network with long shortterm memory layers for predicting diacritics in Arabic text and shows experimentally that this model can rival state-of-the-art methods that have access to additional resources. Expand
Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck Languages
TLDR
This work provides the first demonstration of neural networks recognizing the generalized Dyck languages, which express the core of what it means to be a language with hierarchical structure. Expand
Evaluating Layers of Representation in Neural Machine Translation on Part-of-Speech and Semantic Tagging Tasks
TLDR
This paper investigates the quality of vector representations learned at different layers of NMT encoders and finds that higher layers are better at learning semantics while lower layers tend to be better for part-of-speech tagging. Expand
LSTM Networks Can Perform Dynamic Counting
TLDR
This work is the first study to introduce the shuffle languages to analyze the computational power of neural networks, and shows that a single-layer LSTM with only one hidden unit is practically sufficient for recognizing the Dyck-1 language. Expand
...
1
2
3
4
5
...