Cost-efficient parallel processing of irregularly structured problems in cloud computing environments

@article{Haussmann2018CostefficientPP,
  title={Cost-efficient parallel processing of irregularly structured problems in cloud computing environments},
  author={Jens Haussmann and Wolfgang Blochinger and Wolfgang Kuechlin},
  journal={Cluster Computing},
  year={2018},
  pages={1-23}
}
In this paper, we deal with optimizing the monetary costs of executing parallel applications in cloud-based environments. Specifically, we investigate on how scalability characteristics of parallel applications impact the total costs of computations. We focus on a specific class of irregularly structured problems, where the scalability typically depends on the input data. Consequently, dynamic optimization methods are required for minimizing the costs of computation. For quantifying the total… 
Cost-Optimized Parallel Computations Using Volatile Cloud Resources
TLDR
This paper proposes a cost model for quantifying the monetary costs of executing parallel applications in cloud environments, leveraging volatile resources and determines a configuration of a cloud-based parallel system that minimizes the total cost of executing an application.
Equilibrium: an elasticity controller for parallel tree search in the cloud
TLDR
The experimental results show that, by means of elastic scaling, the performance can be controlled according to user-defined thresholds, which cannot be achieved with static resource provisioning.
Development and Operation of Elastic Parallel Tree Search Applications Using TASKWORK
TLDR
This work discusses the design and implementation of TASKWORK, a cloud-aware runtime system specifically designed for elastic parallel tree search, which enables the implementation of elastic applications by means of higher-level development frameworks.
Cloud Computing Resources Impacts on Heavy-Load Parallel Processing Approaches
TLDR
The scheduling concepts give easy method to use the resources and process the data in parallel and decreasing the overall implementation time of processing algorithms, which give us and open new doors for using the suitable technique in parallel data processing filed.
Elastic Parallel Systems for High Performance Cloud Computing: State-of-the-Art and Future Directions
With on-demand access to compute resources, pay-per-use, and elasticity, the cloud evolved into an attractive execution environment for High Performance Computing (HPC). Whereas elasticity, which is
TASKWORK: A Cloud-aware Runtime System for Elastic Task-parallel HPC Applications
TLDR
This paper presents TASKWORK, a cloud-aware runtime system that enables the implementation of elastic HPC applications by means of higher-level development frameworks and solves corresponding coordination problems based on Apache ZooKeeper.
Self-tuning serverless task farming using proactive elasticity control
TLDR
This work introduces a novel approach to free developers from both parallelism and resource management issues and presents a prototypical elastic parallel system architecture for self-tuning serverless task farming and implements two applications based on the framework.
An Elasticity Description Language for Task-parallel Cloud Applications
TLDR
An elasticity description language is proposed that facilitates users to specify the elasticity behavior at both cloud infrastructure level and application level, and illustrates the capabilities of this approach through real-world scenarios.
Tree-Like Distributed Computation Environment with Shapp Library
TLDR
This article proposes the use in the .NET environment of a new Shapp library that allows remote task execution using fork-like operations from Portable Operating System Interface for UNIX (POSIX) systems.
Cloud Computing and Services Science: 9th International Conference, CLOSER 2019, Heraklion, Crete, Greece, May 2–4, 2019, Revised Selected Papers
TLDR
Two ways to build decision support services on top of the proposed cloud environment for problems where workflows are not (or cannot be) defined in advance, using the idea of human-machine collective intelligence environment.
...
...

References

SHOWING 1-10 OF 48 REFERENCES
Introduction to Parallel Computing
TLDR
Message Passing Interface, POSIX threads and OpenMP have been selected as programming models and the evolving application mix of parallel computing is reflected in various examples throughout the book.
Automatic Communication Optimization of Parallel Applications in Public Clouds
TLDR
A novel solution to perform communication-aware task mapping in the context of commercial cloud environments with multiple instances with up to 11 times faster performance improvements compared to the default scheduling policies is presented.
Towards a Cost Model for Scheduling Scientific Workflows Activities in Cloud Environments
TLDR
A cost model based on concepts of quality of service (QoS) in clouds is proposed to help determining an adequate configuration of the environment according to restrictions imposed by scientists.
Cost models for view materialization in the cloud
TLDR
New cost models that fit into the pay-as-you-go paradigm of cloud computing are defined and these cost models help achieve a multi-criteria optimization of the view materialization vs. CPU power consumption problem, under budget constraints.
Evaluation of HPC Applications on Cloud
TLDR
The results show that Cloud is viable platform for some applications, specifically, non communicationintensive applications such as embarrassingly parallel and tree-structured computations up to high processor count and for communication-intensive applications up to low processor count.
Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud
TLDR
This work represents the most comprehensive evaluation to date comparing conventional HPC platforms to Amazon EC2, using real applications representative of the workload at a typical supercomputing center, and results indicate that EC2 is six times slower than a typical mid-range Linux cluster, and twenty times faster than a modern HPC system.
Cost-Wait Trade-Offs in Client-Side Resource Provisioning with Elastic Clouds
  • S. Genaud, J. Gossa
  • Computer Science
    2011 IEEE 4th International Conference on Cloud Computing
  • 2011
TLDR
This article presents how billing models can be exploited by provisioning strategies to find a trade-off between fast/expensive computations and slow/cheap ones for indepedent sequential jobs.
Towards Cloud-based Asynchronous Elasticity for Iterative HPC Applications
TLDR
A PaaS-based elasticity model that acts as a middleware that allows iterative HPC applications to take advantage of dynamic resource provisioning of cloud infrastructures without any major modification is proposed, named AutoElastic.
Designing Self-Tuning Split-Map-Merge Applications for High Cost-Efficiency in the Cloud
TLDR
This work argues and demonstrates that concurrent applications in cloud platforms must be self-tuning, and shows that applications must incorporate a model of the overheads of operation.
...
...