Corpus ID: 198968245

DETOX: A Redundancy-based Framework for Faster and More Robust Gradient Aggregation

@inproceedings{Rajput2019DETOXAR,
  title={DETOX: A Redundancy-based Framework for Faster and More Robust Gradient Aggregation},
  author={Shashank Rajput and H. Wang and Zachary Charles and Dimitris Papailiopoulos},
  booktitle={NeurIPS},
  year={2019}
}
  • Shashank Rajput, H. Wang, +1 author Dimitris Papailiopoulos
  • Published in NeurIPS 2019
  • Computer Science, Mathematics
  • To improve the resilience of distributed training to worst-case, or Byzantine node failures, several recent approaches have replaced gradient averaging with robust aggregation methods. Such techniques can have high computational costs, often quadratic in the number of compute nodes, and only have limited robustness guarantees. Other methods have instead used redundancy to guarantee robustness, but can only tolerate limited number of Byzantine failures. In this work, we present DETOX, a… CONTINUE READING
    28 Citations
    Fast and Robust Distributed Learning in High Dimension
    ByzShield: An Efficient and Robust System for Distributed Training
    • 1
    • Highly Influenced
    • PDF
    Better scalability under potentially heavy-tailed gradients
    • 1
    • PDF
    Better scalability under potentially heavy-tailed feedback
    • PDF
    Randomized Reactive Redundancy for Byzantine Fault-Tolerance in Parallelized Learning
    • 1
    • Highly Influenced
    • PDF
    Fast Machine Learning with Byzantine Workers and Servers
    • 2
    • PDF
    Collaborative Learning as an Agreement Problem
    • 1
    • PDF
    Federated Variance-Reduced Stochastic Gradient Descent With Robustness to Byzantine Attacks
    • 10
    • PDF
    Distributed Momentum for Byzantine-resilient Learning
    • 2
    • PDF

    References

    SHOWING 1-10 OF 28 REFERENCES
    DRACO: Byzantine-resilient Distributed Training via Redundant Gradients
    • 91
    • PDF
    Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent
    • 224
    • Highly Influential
    • PDF
    Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
    • 185
    • Highly Influential
    • PDF
    Fast and Secure Distributed Learning in High Dimension
    • 10
    • PDF
    SGD: Decentralized Byzantine Resilience
    • 12
    • PDF
    Data Encoding for Byzantine-Resilient Distributed Gradient Descent
    • 10
    Defending Against Saddle Point Attack in Byzantine-Robust Distributed Learning
    • 34
    • PDF