Corpus ID: 30149560

Parallelizing Big Data Machine Learning Applications with Model Rotation

@inproceedings{Zhang2017ParallelizingBD,
  title={Parallelizing Big Data Machine Learning Applications with Model Rotation},
  author={Bingjing Zhang and Bo Peng and J. Qiu},
  year={2017}
}
This paper proposes model rotation as a general approach to parallelize big data machine learning applications. To solve the big model problem in parallelization, we distribute the model parameters to inter-node workers and rotate different model parts in a ring topology. The advantage of model rotation comes from maximizing the effect of parallel model updates for algorithm convergence while minimizing the overhead of communication. We formulate a solution using computation models, programming… Expand
3 Citations

Figures and Tables from this paper

HarpGBDT: Optimizing Gradient Boosting Decision Tree for Parallel Efficiency
Learning Everywhere: Pervasive Machine Learning for Effective High-Performance Computation
  • G. Fox, J. Glazier, +10 authors S. Jha
  • Computer Science, Physics
  • 2019 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)
  • 2019
  • 24
  • PDF

References

SHOWING 1-10 OF 27 REFERENCES
STRADS: a distributed framework for scheduled model parallel machine learning
  • 56
  • PDF
On Model Parallelization and Scheduling Strategies for Distributed Machine Learning
  • 83
  • Highly Influential
  • PDF
Model-centric computation abstractions in machine learning applications
  • 6
  • PDF
High Performance LDA through Collective Model Communication Optimization
  • 14
  • PDF
Distributed Matrix Completion
  • 135
  • PDF
Scaling Distributed Machine Learning with the Parameter Server
  • 1,102
  • PDF
Petuum: A New Platform for Distributed Machine Learning on Big Data
  • 188
Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
  • 1,836
  • PDF
A fast parallel SGD for matrix factorization in shared memory systems
  • 196
  • PDF
...
1
2
3
...