• Publications
  • Influence
QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
TLDR
We propose Quantized SGD (QSGD), a family of compression schemes for gradient updates which provides convergence guarantees for convex and non-convex objectives. Expand
  • 503
  • 111
  • PDF
QSGD: Communication-Optimal Stochastic Gradient Descent, with Applications to Training Neural Networks
TLDR
We propose Quantized SGD (QSGD), a family of compression schemes which allow the compression of gradient updates at each node, while guaranteeing convergence under standard assumptions. Expand
  • 5
  • 3
  • PDF
Synchronous Multi-GPU Deep Learning with Low-Precision Communication: An Experimental Study
TLDR
In this paper, we present an empirical study which focuses on a subspace of the whole tradeoff, that is, the tradeoff introduced by the precision of communication when training deep neural networks with a synchronous multi-GPUs system. Expand
  • 12
  • 1
  • PDF
Communication-Efficient Stochastic Gradient Descent, with Applications to Neural Networks
TLDR
We propose Quantized SGD (QSGD), a family of compression schemes for gradient updates which provides convergence guarantees, and can be extended to stochastic variance-reduced techniques. Expand
  • 7
  • 1
Graph Sparsification in the Semi-streaming Model [ 1 ]
Semi-stremining model. The title of the paper is “Graph Sparsification in the Semi-streaming Model”. First, we should note that it says semi-streaming and not streaming. In graph problems there is aExpand