Corpus ID: 229153349

Better scalability under potentially heavy-tailed feedback

@article{Holland2020BetterSU,
  title={Better scalability under potentially heavy-tailed feedback},
  author={Matthew J. Holland},
  journal={ArXiv},
  year={2020},
  volume={abs/2012.07346}
}

Matthew J. Holland

Published 2020

Computer Science, Mathematics

ArXiv

We study scalable alternatives to robust gradient descent (RGD) techniques that can be used when the losses and/or gradients can be heavy-tailed, though this will be unknown to the learner. The core technique is simple: instead of trying to robustly aggregate gradients at each step, which is costly and leads to sub-optimal dimension dependence in risk bounds, we instead focus computational effort on robustly choosing (or newly constructing) a strong candidate based on a collection of cheap… CONTINUE READING

View PDF on arXiv

Save to Library

Create Alert

Cite

Launch Research Feed