• Publications
  • Influence
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
This systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks and achieves state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. Expand
MixMatch: A Holistic Approach to Semi-Supervised Learning
This work unify the current dominant approaches for semi-supervised learning to produce a new algorithm, MixMatch, that works by guessing low-entropy labels for data-augmented unlabeled examples and mixing labeled and unlabeling data using MixUp. Expand
Theano: A Python framework for fast computation of mathematical expressions
The performance of Theano is compared against Torch7 and TensorFlow on several machine learning models and recently-introduced functionalities and improvements are discussed. Expand
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
This paper demonstrates the power of a simple combination of two common SSL methods: consistency regularization and pseudo-labeling, and shows that FixMatch achieves state-of-the-art performance across a variety of standard semi-supervised learning benchmarks. Expand
Realistic Evaluation of Deep Semi-Supervised Learning Algorithms
This work creates a unified reimplemention and evaluation platform of various widely-used SSL techniques and finds that the performance of simple baselines which do not use unlabeled data is often underreported, that SSL methods differ in sensitivity to the amount of labeled and unlabeling data, and that performance can degrade substantially when the unlabelED dataset contains out-of-class examples. Expand
librosa: Audio and Music Signal Analysis in Python
A brief overview of the librosa library's functionality is provided, along with explanations of the design goals, software development practices, and notational conventions. Expand
A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
This work proposes the use of a hierarchical decoder, which first outputsembeddings for subsequences of the input and then uses these embeddings to generate each subsequence independently, thereby avoiding the "posterior collapse" problem, which remains an issue for recurrent VAEs. Expand
Lasagne: First release.
ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
A variant of AutoAugment which learns the augmentation policy while the model is being trained, and is significantly more data-efficient than prior work, requiring between $5\times and $16\times less data to reach the same accuracy. Expand
mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer
The recent “Text-to-Text Transfer Transformer” (T5) leveraged a unified text-to-text format and scale to attain state-of-the-art results on a wide variety of English-language NLP tasks. In thisExpand