Data-Intensive HPC Tasks Scheduling with SDN to Enable HPC-as-a-Service

@article{Jamalian2015DataIntensiveHT,
  title={Data-Intensive HPC Tasks Scheduling with SDN to Enable HPC-as-a-Service},
  author={Saba Jamalian and Hassan Rajaei},
  journal={2015 IEEE 8th International Conference on Cloud Computing},
  year={2015},
  pages={596-603}
}
  • S. Jamalian, H. Rajaei
  • Published 27 June 2015
  • Computer Science
  • 2015 IEEE 8th International Conference on Cloud Computing
Advances in Cloud Computing attracted scientists to deploy their HPC applications to the cloud to benefit from the flexibility of the platform such as scalability and on-demand services. Nevertheless, HPC programs can face serious challenges in the cloud that could undermine the gained benefits. This paper first compares the performance of several HPC benchmarks on a commodity cluster and Amazon public cloud to illustrate the confronted challenges. To mitigate the problem, we have introduced a… 

Seamlessly Managing HPC Workloads Through Kubernetes

By using hpc-connector, the cloud job scheduler can manage the jobs in the HPC infrastructure without requiring any special privilege, as it does not need changes on the Job scheduler.

HPC as a Service on Cloud: A Market Oriented Approach

This paper provides solution for one of these problems as a costing model for HPC as a services on the cloud environment so that it will enable the small organization to have their own HPC systems at the low costs.

A dististributed simulation platform for cloud computing

A new platform Distributed Simulation for Could Computing (DSC) developed, to facilitate implementation and parallel execution of simulations models in the cloud and enables simulation users to seamlessly interact with a cloud environment and receive benefits of the cloud for their applications.

Optimized provisioning of SDN-enabled virtual networks in geo-distributed cloud computing datacenters

This study formulate virtual network provisioning in SDN-enabled, geographically distributed cloud computing datacenters as a mixed integer linear programming (MILP) problem and the results verify the effectiveness of the proposed approach.

Network Parallelization in HPC Clusters

This paper proposes a parallelization for the underlying network based on the different running HPC experiments or jobs, and shows the impact that such network parallelization/slicing will bring to HPC clusters in terms of cluster efficiency and experiments' performance.

Integrating SDN-Enhanced MPI with Job Scheduler to Support Shared Clusters

A plugin for the job scheduler that collects and reports the job information to the interconnect controller is developed and demonstrated that applications could gain up to 2.56× speedup in communication.

Scheduling of distributed applications for high performance computing as a service

Experimental results showed that the proposed method for scheduling parallel applications in HPCaaS cloud gives significantly better utilization of computational resources than classical scheduling methods.

Deployment and Analysis of a Hybrid Shared/Distributed-Memory Parallel Visualization Tool for 3-D Oil Reservoir Grid on OpenStack Cloud Computing

This article experimentally study the parallelization of intensive computation for a 3-D (three dimensional) oil reservoir data visualization tool, and proposes a hybrid (shared memory and distributed memory) parallelization technique to adapt with the data processing scalability.

High Performance Computing: ISC High Performance 2020 International Workshops, Frankfurt, Germany, June 21–25, 2020, Revised Selected Papers

TyCart is a tool for type-safe checkpoint/restart and extends the memory allocation sanitizer tool TypeART with type asserts, which shows runtime and memory overhead below 5% in smaller benchmarks.

References

SHOWING 1-10 OF 27 REFERENCES

ASETS: A SDN Empowered Task Scheduling System for HPCaaS on the Cloud

A new scheme called ASETS is presented which targets dynamic configuration and monitoring of cloud networking using SDN to improve the performance of HPC applications and in particular task scheduling for HPC as a service on the cloud (HPCaaS).

Environment-conscious scheduling of HPC applications on distributed Cloud-oriented data centers

Scheduling strategies for HPC as a service (HPCaaS)

The paper reviews the concept of HPCaaS and explores a smart scheduling algorithm for a subset of bioscience applications and shows that smart scheduling can accommodate multiple applications and multiple jobs simultaneously while increasing the overall system productivity and efficiency.

Scheduling Concurrent Workflows in HPC Cloud through Exploiting Schedule Gaps

This paper proposes a method which exploits schedule gaps to efficiently schedule concurrent workflows in HPC cloud and can deliver good performance and outperform the existing method significantly in terms of average makespan, up to 18% performance improvement.

Moldable Job Scheduling for HPC as a Service

A moldable job scheduling approach is proposed which relieves HPC users’ burden of selecting an appropriate number of processors and can achieve even better system performance than existing job scheduling methods.

Evaluating GPU Passthrough in Xen for High Performance Cloud Computing

Results show PCI passthrough of GPUs within virtual machines is a viable use case for many scientific computing workflows, and could help support high performance cloud infrastructure in the near future.

Bandwidth-Aware Scheduling With SDN in Hadoop: A New Trend for Big Data

BASS is the first to exploit talent of SDN for job scheduling of big data processing and believes that it points out a new trend for large-scale data processing.

Scalability of parallel scientific applications on the cloud

The analysis of DOUG showed that Hadoop MapReduce has significant problems with iterative methods, while it suits well for embarrassingly parallel algorithms, which raises the necessity for better frameworks or optimizations for Map Reduce.

Cloud computing networking: challenges and opportunities for innovations

This work presents the networking issues in IaaS and federation challenges that are currently addressed with existing technologies and presents innovative software-defined networking proposals, which are applied to some of the challenges and could be used in future deployments as efficient solutions.