• Publications
  • Influence
Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty
TLDR
This work finds that self-supervision can benefit robustness in a variety of ways, including robustness to adversarial examples, label corruption, and common input corruptions, and greatly benefits out-of-distribution detection on difficult, near-dist distribution outliers.
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
TLDR
It is found that using larger models and artificial data augmentations can improve robustness on real-world distribution shifts, contrary to claims in prior work.
Measuring Mathematical Problem Solving With the MATH Dataset
TLDR
This work introduces MATH, a new dataset of 12, 500 challenging competition mathematics problems which can be used to teach models to generate answer derivations and explanations, and shows that accuracy remains relatively low, even with enormous Transformer models.
Measuring Coding Challenge Competence With APPS
TLDR
APPS is introduced, a benchmark for code generation that measures the ability of models to take an arbitrary natural language specification and generate satisfactory Python code and shows that machine learning models are now beginning to learn how to code.
Angular analysis of the rare decay $$ {B}_s^0 $$ → ϕμ+μ−
Abstract An angular analysis of the rare decay $$ {B}_s^0 $$ B s 0 → ϕμ+μ− is presented, using proton-proton collision data collected by the LHCb experiment at centre-of-mass energies of 7,
Search for the radiative $\Xi_b^-\to\Xi^-\gamma$ decay
The first search for the rare radiative decay Ξ− b → Ξ −γ is performed using data collected by the LHCb experiment in proton-proton collisions at a center-of-mass energy of 13 TeV, corresponding to
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
TLDR
An iterated online mode of training, where preference models and RL policies are updated on a weekly cadence with fresh human feedback data, and a roughly linear relation between the RL reward and the square root of the KL divergence between the policy and its initialization is identified.
Pretraining & Reinforcement Learning: Sharpening the Axe Before Cutting the Tree
TLDR
Evaluation of the effectiveness of pretraining for RL tasks, with and without distracting backgrounds, using both large, publicly available datasets with minimal relevance, as well as case-by-case generated datasets labeled via selfsupervision suggests filters learned during training on less relevant datasets render pretraining ineffective.
Evidence for a New Structure in the J/ψp and J/ψp[over ¯] Systems in B_{s}^{0}→J/ψpp[over ¯] Decays.
An amplitude analysis of flavor-untagged B_{s}^{0}→J/ψpp[over ¯] decays is performed using a sample of 797±31 decays reconstructed with the LHCb detector. The data, collected in proton-proton
A Critical Analysis of Distribution Shift
TLDR
It is found that some methods consistently help with distribution shifts in texture and local image statistics, but these methods do not help with some other distribution shifts like geographic changes, so no evaluated method consistently improves robustness.