• Publications
  • Influence
Learning Longer-term Dependencies in RNNs with Auxiliary Losses
tl;dr
This paper proposes a simple method that improves the ability to capture long term dependencies in RNNs by adding an unsupervised auxiliary loss to the original objective. Expand
  • 70
  • 8
  • Open Access
A Simple Method for Commonsense Reasoning
tl;dr
We present a simple method for commonsense reasoning with neural networks, using unsupervised learning, without using expensive annotated knowledge bases or hand-engineered features. Expand
  • 108
  • 7
  • Open Access
Selfie: Self-supervised Pretraining for Image Embedding
tl;dr
We introduce a pretraining technique called Selfie, which stands for SELFie supervised Image Embedding. Expand
  • 24
  • 1
  • Open Access
Attention Pooling A Softmax-Cross Entropy with true label � Distractor Patch
We introduce a pretraining technique called Selfie, which stands for SELFsupervised Image Embedding. Selfie generalizes the concept of masked language modeling to continuous data, such as images.Expand
  • 4
  • 1
Do Language Models Have Common Sense
It has been argued that current machine learning models do not have common sense, and therefore must be hard-coded with prior knowledge (Marcus, 2018). Here we show surprising evidence that languageExpand
  • 6