SINGA: Putting Deep Learning in the Hands of Multimedia Users

@article{Wang2015SINGAPD,
  title={SINGA: Putting Deep Learning in the Hands of Multimedia Users},
  author={Wei Wang and Gang Chen and Tien Tuan Anh Dinh and Jinyang Gao and Beng Chin Ooi and Kian-Lee Tan and Sheng Wang},
  journal={Proceedings of the 23rd ACM international conference on Multimedia},
  year={2015}
}
  • Wei WangGang Chen Sheng Wang
  • Published 13 October 2015
  • Computer Science
  • Proceedings of the 23rd ACM international conference on Multimedia
Recently, deep learning techniques have enjoyed success in various multimedia applications, such as image classification and multi-modal data analysis. Two key factors behind deep learning's remarkable achievement are the immense computing power and the availability of massive training datasets, which enable us to train large models to capture complex regularities of the data. There are two challenges to overcome before deep learning can be widely adopted in multimedia and other applications… 

Deep Learning at Scale and at Ease

This article designs a distributed deep learning platform called SINGA, which has an intuitive programming model based on the common layer abstraction of deep learning models, and shows that it outperforms many other state-of-the-art deep learning systems.

HPDL: Towards a General Framework for High-performance Distributed Deep Learning

HPDL is proposed, a general framework for high-performance distributed deep learning which is compatible with existing frameworks and adaptive to various hardware architectures, including optimization algorithm, hybrid communication mechanism, model parallelization, resource scheduling and single-node execution optimization.

SINGA: A Distributed Deep Learning Platform

A distributed deep learning system, called SINGA, for training big models over large datasets, which supports a variety of popular deep learning models and provides different neural net partitioning schemes for training large models.

Database Meets Deep Learning: Challenges and Opportunities

Possible improvements for deep learning systems from a database perspective are discussed, and database applications that may benefit from deep learning techniques are analyzed.

Improving the performance of dataflow systems for deep neural network training

Ako, a DNN system that uses partial gradient exchange for synchronising replicas in a peer-to-peer fashion and exhibits a 25% lower convergence time than a hand-tuned parameter-server deployments is presented.

A New Approach to Compute CNNs for Extremely Large Images

This paper proposes a new approach that adopts the BSP (bulk synchronization parallel) model to compute CNNs for images of any size and shows that using one Titan-X GPU, it can transfer the style of an image with 10,000×10,000 pixels within 1 minute.

Evaluation of Deep Learning Frameworks Over Different HPC Architectures

This work investigates the performance characteristics of NVIDIA's state-of-the-art hardware technology, NVLink, and also Intel's Knights Landing, the most advanced Intel product for deep learning, with respect to training time and utilization, and provides analysis of the frameworks' performance over different hardware environments in terms of speed and scaling.

AL‐DDCNN: a distributed crossing semantic gap learning for person re‐identification

A novel approach called attribute learning based on distributed deep convolutional neural network model is proposed to address person re‐identification task and shows how attributes, namely the mid‐level medium between classes and features, are obtained automatically and how they are employed to re-identify person with semantics when an author‐topic model is used to mapping category.

A Survey on Deep Learning Methods for Robot Vision

A comprehensive overview of deep learning and its usage in computer vision is given, that includes a description of the most frequently used neural models and their main application areas, and a review of the principal work using deep learning in robot vision.

Poseidon : An Efficient Communication Interface for Distributed Deep Learning on GPU Clusters

Poseidon is presented, an efficient communication interface for distributed DL on GPUs that exploits the layered model structures in DL programs to overlap communication and computation, reducing bursty network communication.

References

SHOWING 1-10 OF 39 REFERENCES

Effective deep learning-based multi-modal retrieval

This paper proposes a general learning objective that effectively captures both intramodal and intermodal semantic relationships of data from heterogeneous sources and proposes two learning algorithms to realize it: an unsupervised approach that uses stacked auto-encoders and requires minimum prior knowledge on the training data and a supervised approach using deep convolutional neural network and neural language model.

SINGA: A Distributed Deep Learning Platform

A distributed deep learning system, called SINGA, for training big models over large datasets, which supports a variety of popular deep learning models and provides different neural net partitioning schemes for training large models.

Caffe: Convolutional Architecture for Fast Feature Embedding

Caffe provides multimedia scientists and practitioners with a clean and modifiable framework for state-of-the-art deep learning algorithms and a collection of reference models for training and deploying general-purpose convolutional neural networks and other deep models efficiently on commodity architectures.

Deep Learning for Content-Based Image Retrieval: A Comprehensive Study

This paper investigates a framework of deep learning with application to CBIR tasks with an extensive set of empirical studies by examining a state-of-the-art deep learning method (Convolutional Neural Networks) for CBIr tasks under varied settings.

Exploring Inter-feature and Inter-class Relationships with Deep Neural Networks for Video Classification

A novel unified framework that jointly learns feature relationships and exploits the class relationships for improved video classification performance is proposed and demonstrates that the proposed framework exhibits superior performance over several state-of-the-art approaches.

Effective Multi-Modal Retrieval based on Stacked Auto-Encoders

This paper proposes an effective mapping mechanism based on deep learning (i.e., stacked auto-encoders) for multi-modal retrieval that achieves significant improvement in search accuracy over the state-of-the-art methods.

Improving Content-based and Hybrid Music Recommendation using Deep Learning

This work uses a novel model based on deep belief network and probabilistic graphical model to unify the two stages of collaborative filtering into an automated process that simultaneously learns features from audio content and makes personalized recommendations.

ImageNet classification with deep convolutional neural networks

A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.

Building high-level features using large scale unsupervised learning

Contrary to what appears to be a widely-held intuition, the experimental results reveal that it is possible to train a face detector without having to label images as containing a face or not.

Project Adam: Building an Efficient and Scalable Deep Learning Training System

The design and implementation of a distributed system called Adam comprised of commodity server machines to train large deep neural network models that exhibits world-class performance, scaling and task accuracy on visual recognition tasks and shows that task accuracy improves with larger models.