HPC Cloud for Scientific and Business Applications

@article{Netto2019HPCCF,
  title={HPC Cloud for Scientific and Business Applications},
  author={Marco Aur{\'e}lio Stelmar Netto and Rodrigo Neves Calheiros and Eduardo Rocha Rodrigues and Renato L. F. Cunha and Rajkumar Buyya},
  journal={ACM Computing Surveys (CSUR)},
  year={2019},
  volume={51},
  pages={1 - 29}
}
High performance computing (HPC) clouds are becoming an alternative to on-premise clusters for executing scientific applications and business analytics services. Most research efforts in HPC cloud aim to understand the cost benefit of moving resource-intensive applications from on-premise environments to public cloud platforms. Industry trends show that hybrid environments are the natural path to get the best of the on-premise and cloud resources—steady (and sensitive) workloads can run on on… 

Figures and Tables from this paper

Boosting HPC Applications in the Cloud Through JIT Traffic-Aware Path Provisioning
TLDR
A mechanism for dynamic, just-in-time path provisioning in cloud infrastructures that continuously monitors the network conditions and systematically (re)programs paths to avoid uncongested links and reduce end-to-end delays is proposed.
HPCCloud Seer: A Performance Model Based Predictor for Parallel Applications on the Cloud
TLDR
A predictor for the execution time of the message passing interface (MPI) based applications on the cloud is proposed, based on an analytical performance model through considering the cloud resources as a queueing network, and the parallel applications as jobs contesting for the shared resources.
TASKWORK: A Cloud-aware Runtime System for Elastic Task-parallel HPC Applications
TLDR
This paper presents TASKWORK, a cloud-aware runtime system that enables the implementation of elastic HPC applications by means of higher-level development frameworks and solves corresponding coordination problems based on Apache ZooKeeper.
Migrating parallel applications to the cloud: assessing cloud readiness based on parallel design decisions
TLDR
This paper presents a novel approach to assess the cloud readiness of parallel applications based on the design decisions made and introduces an assessment procedure, its underlying meta model, and a corresponding instantiation to structure this multi-dimensional design space.
Elastic Parallel Systems for High Performance Cloud Computing: State-of-the-Art and Future Directions
With on-demand access to compute resources, pay-per-use, and elasticity, the cloud evolved into an attractive execution environment for High Performance Computing (HPC). Whereas elasticity, which is
IMPROVING RESOURCE USAGE IN HPC CLOUDS
TLDR
This paper presents and checks hypothesis: “MPI programs that don’t require a lot of computing resources can effectively share the same set of resources” and shows that it can improve the CPU usage in the cloud with negligible performance degradation of HPC-applications execution.
Survey of Memory Management Techniques for HPC and Cloud Computing
TLDR
In this survey, challenges of memory management in HPC and Cloud Computing, different memory management systems and optimisation techniques to increase memory utilisation are discussed in detail.
Designing Parallel Operation for High-Performance Cloud Computing Using Partition Algorithm
TLDR
The partition algorithm is used for designing parallel operation for HPCC to lift in selection at the expense of supply and to reduce the performance time of taking input operations (or) conditions.
Evaluating the Impact of Full Virtualized High- Performance Computing Platform on Large Scale Scientific Data using Quantum Espresso
TLDR
It was observed that running job using HPC applications like QE on a fully virtualized HPC cluster to analyze large scale scientific data has a negative performance impact on the completion of the job as compared to the traditional cluster.
Contextual Contracts for Component-Oriented Resource Abstraction in a Cloud of HPC Services
TLDR
Alite, the system of contextual contracts of HPC Shelf, aimed at selecting component implementations according to requirements of applications, features of targeting parallel computing platforms (e.g. clusters), QoS (Quality-of-Service) properties and cost restrictions is introduced.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 153 REFERENCES
Case study for running HPC applications in public clouds
TLDR
The results show that 1) virtualization technology, which is widely used by cloud computing, adds little performance overhead; 2) most current public clouds are not designed for running scientific applications primarily due to their poor networking capabilities; however, a cloud with moderately better network will deliver a significant performance improvement.
Evaluation of HPC Applications on Cloud
TLDR
The results show that Cloud is viable platform for some applications, specifically, non communicationintensive applications such as embarrassingly parallel and tree-structured computations up to high processor count and for communication-intensive applications up to low processor count.
Evaluating and Improving the Performance and Scheduling of HPC Applications in Cloud
TLDR
This paper performs comprehensive performance and cost evaluation and analysis of running a set of HPC applications on a range of platforms, varying from supercomputers to clouds, and presents novel heuristics for online application-aware job scheduling in multi-platform environments.
High Performance Computing in the cloud: Deployment, performance and cost efficiency
TLDR
A detailed comparison of HPC applications running on three cloud providers, Amazon EC2, Microsoft Azure and Rackspace, shows that HPC in the cloud can have a higher performance and cost efficiency than a traditional cluster, up to 27% and 41%, respectively.
Early Prediction of the Cost of Cloud Usage for HPC Applications
TLDR
This paper presents a framework built on the top of a cloud-aware programming platform (mOSAIC) for the development of bag-of-tasks scientific applications and presents the approach followed for the performance simulation of applications, focusing on a software development methodology that hinges on early performance prediction.
Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud
TLDR
This work represents the most comprehensive evaluation to date comparing conventional HPC platforms to Amazon EC2, using real applications representative of the workload at a typical supercomputing center, and results indicate that EC2 is six times slower than a typical mid-range Linux cluster, and twenty times faster than a modern HPC system.
A unified framework for the deployment, exposure and access of HPC applications as services in clouds
Efficient resource selection framework to enable cloud for HPC applications
TLDR
This work is proposing a method to form cluster of heterogeneous compute resources for HPC base applications on cloud and observed homogeneity among the resources of the best cluster chosen by the proposed method.
The Who, What, Why, and How of High Performance Computing in the Cloud
TLDR
Overall results indicate that current public clouds are cost-effective only at small scale for the chosen HPC applications, when considered in isolation, but can complement supercomputers using business models such as cloud burst and application-aware mapping.
Next Generation HPC Clouds: A View for Large-Scale Scientific and Data-Intensive Applications
TLDR
A holistic vision is contributed that imagines a tight integration between programming models, runtime middlewares and the virtualization infrastructure in order to provide a framework that transparently handles allocation and utilization of heterogeneous resources while dealing with performance and elasticity issues.
...
1
2
3
4
5
...