SINGA: A Distributed Deep Learning Platform

@article{Ooi2015SINGAAD,
  title={SINGA: A Distributed Deep Learning Platform},
  author={Beng Chin Ooi and Kian-Lee Tan and Sheng Wang and Wei Wang and Qingchao Cai and Gang Chen and Jinyang Gao and Zhaojing Luo and Anthony K. H. Tung and Yuan Wang and Zhongle Xie and Meihui Zhang and Kaiping Zheng},
  journal={Proceedings of the 23rd ACM international conference on Multimedia},
  year={2015}
}
  • B. OoiK. Tan Kaiping Zheng
  • Published 13 October 2015
  • Computer Science
  • Proceedings of the 23rd ACM international conference on Multimedia
Deep learning has shown outstanding performance in various machine learning tasks. [] Key Method SINGA architecture supports both synchronous and asynchronous training frameworks. Hybrid training frameworks can also be customized to achieve good scalability. SINGA provides different neural net partitioning schemes for training large models. SINGA is an Apache Incubator project released under Apache License 2.

Figures from this paper

DeepSpark: A Spark-Based Distributed Deep Learning Framework for Commodity Clusters

To support parallel operations, DeepSpark automatically distributes workloads and parameters to Caffe/Tensorflow-running nodes using Spark, and iteratively aggregates training results by a novel lock-free asynchronous variant of the popular elastic averaging stochastic gradient descent based update scheme, effectively complementing the synchronized processing capabilities of Spark.

Deep Learning Frameworks on Apache Spark: A Review

All the recent deep learning frameworks are reviewed exhaustively with a detailed assessment and comparison and experimentation results are provided to evaluate which frameworks are more suitable for deep models.

Deep Learning at Scale and at Ease

This article designs a distributed deep learning platform called SINGA, which has an intuitive programming model based on the common layer abstraction of deep learning models, and shows that it outperforms many other state-of-the-art deep learning systems.

Improving the performance of dataflow systems for deep neural network training

Ako, a DNN system that uses partial gradient exchange for synchronising replicas in a peer-to-peer fashion and exhibits a 25% lower convergence time than a hand-tuned parameter-server deployments is presented.

DeepSpark: Spark-Based Deep Learning Supporting Asynchronous Updates and Caffe Compatibility

DeepSpark is proposed, a distributed and parallel deep learning framework that simultaneously exploits Apache Spark for large-scale distributed data management and Caffe for GPU-based acceleration.

Towards a Scalable and Distributed Infrastructure for Deep Learning Applications

Phylanx presents a productivity-oriented frontend where user Python code is translated to a futurized execution tree that can be executed efficiently on multiple nodes using the C++ standard library for parallelism and concurrency (HPX), leveraging fine-grained threading and an active messaging task-based runtime system.

Scalable Deep Learning on Distributed Infrastructures

This survey performs a broad and thorough investigation on challenges, techniques and tools for scalable DL on distributed infrastructures, and highlights future research trends in DL systems that deserve further research.

Deep Learning Training Management Platform Based on Distributed Technologies in Resource-Constrained Scenarios

A deep learning training management platform based on distributed technologies, which integrates different kinds of deep learning frameworks through virtualization technologies and coordinates machines through distributed technologies is proposed.

Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques and Tools

This survey performs a broad and thorough investigation on challenges, techniques and tools for scalable DL on distributed infrastructures, and highlights future research trends in DL systems that deserve further research.

SINGA: Putting Deep Learning in the Hands of Multimedia Users

This paper designs a distributed deep learning platform called SINGA which has an intuitive programming model and good scalability, and experience with developing and training deep learning models for real-life multimedia applications in SINGSA shows that the platform is both usable and scalable.
...

References

SHOWING 1-10 OF 13 REFERENCES

SINGA: Putting Deep Learning in the Hands of Multimedia Users

This paper designs a distributed deep learning platform called SINGA which has an intuitive programming model and good scalability, and experience with developing and training deep learning models for real-life multimedia applications in SINGSA shows that the platform is both usable and scalable.

Purine: A bi-graph based deep learning framework

It is demonstrated that different parallelism schemes over GPUs and/or CPUs on single or multiple PCs can be universally implemented by graph composition, which eases researchers from coding for various parallelization schemes, and the same dispatcher can be used for solving variant graphs.

Caffe: Convolutional Architecture for Fast Feature Embedding

Caffe provides multimedia scientists and practitioners with a clean and modifiable framework for state-of-the-art deep learning algorithms and a collection of reference models for training and deploying general-purpose convolutional neural networks and other deep models efficiently on commodity architectures.

ImageNet classification with deep convolutional neural networks

A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.

Deep Image: Scaling up Image Recognition

A state-of-the-art image recognition system, Deep Image, developed using end-to-end deep learning, which achieves excellent results on multiple challenging computer vision benchmarks.

Effective deep learning-based multi-modal retrieval

This paper proposes a general learning objective that effectively captures both intramodal and intermodal semantic relationships of data from heterogeneous sources and proposes two learning algorithms to realize it: an unsupervised approach that uses stacked auto-encoders and requires minimum prior knowledge on the training data and a supervised approach using deep convolutional neural network and neural language model.

Effective Multi-Modal Retrieval based on Stacked Auto-Encoders

This paper proposes an effective mapping mechanism based on deep learning (i.e., stacked auto-encoders) for multi-modal retrieval that achieves significant improvement in search accuracy over the state-of-the-art methods.

Fast Convolutional Nets With fbfft: A GPU Performance Evaluation

We examine the performance profile of Convolutional Neural Network training on the current generation of NVIDIA Graphics Processing Units. We introduce two new Fast Fourier Transform convolution

Effiicient BackProp

Many undesirable behaviors of backprop can be avoided with tricks that are rarely exposed in serious technical publications, and this paper gives some of those tricks, and offers explanations of why they work.

Deep Neural Networks for Acoustic Modeling in Speech Recognition

This paper provides an overview of this progress and repres nts the shared views of four research groups who have had recent successes in using deep neural networks for a coustic modeling in speech recognition.