MLI: An API for Distributed Machine Learning

@article{Sparks2013MLIAA,
  title={MLI: An API for Distributed Machine Learning},
  author={Evan R. Sparks and Ameet S. Talwalkar and Virginia Smith and Jey Kottalam and Xinghao Pan and Joseph E. Gonzalez and Michael J. Franklin and Michael I. Jordan and Tim Kraska},
  journal={2013 IEEE 13th International Conference on Data Mining},
  year={2013},
  pages={1187-1192}
}
MLI is an Application Programming Interface designed to address the challenges of building Machine Learning algorithms in a distributed setting based on data-centric computing. Its primary goal is to simplify the development of high-performance, scalable, distributed algorithms. Our initial results show that, relative to existing systems, this interface can be used to build distributed implementations of a wide variety of common Machine Learning algorithms with minimal complexity and highly… 

Figures from this paper

Distributed Machine Learning Using Data Parallelism on Mobile Platform
  • M. Szabó
  • Computer Science
    J. Mobile Multimedia
  • 2020
TLDR
This paper investigates the expansion of data parallelism to mobile, which became the most popular platform, and shows that doing distributed training on mobile cluster is possible and safe, but its performance depends on the algorithm’s implementation.
Communication Efficient Distributed Machine Learning with the Parameter Server
TLDR
An in-depth analysis of two large scale machine learning problems ranging from l1 -regularized logistic regression on CPUs to reconstruction ICA on GPUs, using 636TB of real data with hundreds of billions of samples and dimensions is presented.
A Resource Effective Approach for Distributed Machine Learning over a Local Network
TLDR
This work attempts to propose a system which aims to solve the problem of distributed machine learning by providing a platform that allows distributing the machine learning process over a local network (using locally available nodes) to ensure the effective use of all the resources available.
Brook : An Easy and Efficient Framework for Distributed Machine Learning
TLDR
Brook is a new framework for distributed machine learning problems that adopts the parameter server paradigm that simplifies the task of distributed programming and provides a novel system component called parameter agent that masks the communication details between workers and servers by mapping remote servers to local in-memory file.
Proposal Scaling Distributed Machine Learning with System and Algorithm Co-design
  • Mu Li
  • Computer Science
  • 2016
TLDR
This thesis proposes two distributed computing frameworks: a parameter server framework which features efficient data communication, and MXNet, a multi-language library aiming to simplify the development of deep neural network algorithms that are specialized for large machine learning problems.
STRADS: a distributed framework for scheduled model parallel machine learning
TLDR
Schedule model parallelism (SchMP), a programming approach that improves ML algorithm convergence speed by efficiently scheduling parameter updates, taking into account parameter dependencies and uneven convergence, is proposed.
SystemML: Declarative Machine Learning on Spark
TLDR
This paper describes SystemML on Apache Spark, end to end, including insights into various optimizer and runtime techniques as well as performance characteristics.
SMPFRAME : A Distributed Framework for Scheduled Model Parallel Machine Learning
TLDR
Schedule model parallellism (SMP) is proposed, a programming approach where selection of parameters to be updated (the schedule) is explicitly separated from parameter update logic and can improve ML algorithm convergence speed by planning for parameter dependencies and uneven convergence.
Autonomic Machine Learning for Intelligent Databases
TLDR
This paper addresses an architecture of autonomic machine learning platform with which developers get some assistance in choosing a machine learning algorithm appropriate to a task and in selecting the values of hyperparameters of the algorithm.
MLlib: Machine Learning in Apache Spark
TLDR
MLlib is presented, Spark's open-source distributed machine learning library that provides efficient functionality for a wide range of learning settings and includes several underlying statistical, optimization, and linear algebra primitives.
...
...

References

SHOWING 1-10 OF 27 REFERENCES
MLbase: A Distributed Machine-learning System
TLDR
This work presents the vision for MLbase, a novel system harnessing the power of machine learning for both end-users and ML researchers, which provides a simple declarative way to specify ML tasks and a novel optimizer to select and dynamically adapt the choice of learning algorithm.
MLbase : A Distributed Machine Learning Wrapper
TLDR
This work presents the vision for MLbase, a novel system harnessing the power of machine learning for both end-users and ML researchers, which provides a simple declarative way to specify ML tasks and a novel optimizer to select and dynamically adapt the choice of learning algorithm.
Scaling Datalog for Machine Learning on Big Data
TLDR
This paper argues for the use of recursive queries to program a variety of machine learning systems using database query optimization techniques to identify effective execution plans, and the resulting runtime plans can be executed on a single unified data-parallel query processing engine.
SystemML: Declarative machine learning on MapReduce
TLDR
This paper proposes SystemML in which ML algorithms are expressed in a higher-level language and are compiled and executed in a MapReduce environment and describes and empirically evaluate a number of optimization strategies for efficiently executing these algorithms on Hadoop, an open-source mapReduce implementation.
GraphLab: A New Framework For Parallel Machine Learning
TLDR
The expressiveness of the GraphLab framework is demonstrated by designing and implementing parallel versions of belief propagation, Gibbs sampling, Co-EM, Lasso and Compressed Sensing and it is shown that using GraphLab the authors can achieve excellent parallel performance on large scale real-world problems.
MLPACK: a scalable C++ machine learning library
TLDR
MLPACK is a state-of-the-art, scalable, multi-platform C++ machine learning library providing cutting-edge algorithms whose benchmarks exhibit far better performance than other leading machine learning libraries.
Presto: distributed machine learning and graph processing with sparse matrices
TLDR
Presto is built, a distributed system that extends R and addresses many of its limitations, and shows the promise of this approach: many important machine learning and graph algorithms can be expressed in a single framework and are substantially faster than those in Hadoop and Spark.
OptiML: An Implicitly Parallel Domain-Specific Language for Machine Learning
TLDR
OptiML is an implicitly parallel, expressive and high performance alternative to MATLAB and C++ and shows that OptiML outperforms explicitly parallelized MATLAB code in nearly all cases.
The MADlib Analytics Library or MAD Skills, the SQL
TLDR
The MADlib project is introduced, including the background that led to its beginnings, and the motivation for its open-source nature, and an overview of the library's architecture and design patterns is provided, and a description of various statistical methods in that context is provided.
Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing
TLDR
Resilient Distributed Datasets is presented, a distributed memory abstraction that lets programmers perform in-memory computations on large clusters in a fault-tolerant manner and is implemented in a system called Spark, which is evaluated through a variety of user applications and benchmarks.
...
...