Data-Intensive HPC Tasks Scheduling with SDN to Enable HPC-as-a-Service

@article{Jamalian2015DataIntensiveHT,
  title={Data-Intensive HPC Tasks Scheduling with SDN to Enable HPC-as-a-Service},
  author={Saba Jamalian and Hassan Rajaei},
  journal={2015 IEEE 8th International Conference on Cloud Computing},
  year={2015},
  pages={596-603}
}
  • S. Jamalian, H. Rajaei
  • Published 2015
  • Computer Science
  • 2015 IEEE 8th International Conference on Cloud Computing
Advances in Cloud Computing attracted scientists to deploy their HPC applications to the cloud to benefit from the flexibility of the platform such as scalability and on-demand services. Nevertheless, HPC programs can face serious challenges in the cloud that could undermine the gained benefits. This paper first compares the performance of several HPC benchmarks on a commodity cluster and Amazon public cloud to illustrate the confronted challenges. To mitigate the problem, we have introduced a… Expand
Seamlessly Managing HPC Workloads Through Kubernetes
TLDR
By using hpc-connector, the cloud job scheduler can manage the jobs in the HPC infrastructure without requiring any special privilege, as it does not need changes on the Job scheduler. Expand
HPC as a Service on Cloud: A Market Oriented Approach
Cloud is a latest technology used in most of enterprise. Every enterprise is moving towards the cloud platform. On the other hand High Performance Computing (HPC) is a necessity for manyExpand
A dististributed simulation platform for cloud computing
TLDR
A new platform Distributed Simulation for Could Computing (DSC) developed, to facilitate implementation and parallel execution of simulations models in the cloud and enables simulation users to seamlessly interact with a cloud environment and receive benefits of the cloud for their applications. Expand
Optimized provisioning of SDN-enabled virtual networks in geo-distributed cloud computing datacenters
TLDR
This study formulate virtual network provisioning in SDN-enabled, geographically distributed cloud computing datacenters as a mixed integer linear programming (MILP) problem and the results verify the effectiveness of the proposed approach. Expand
Network Parallelization in HPC Clusters
TLDR
This paper proposes a parallelization for the underlying network based on the different running HPC experiments or jobs, and shows the impact that such network parallelization/slicing will bring to HPC clusters in terms of cluster efficiency and experiments' performance. Expand
Integrating SDN-Enhanced MPI with Job Scheduler to Support Shared Clusters
TLDR
A plugin for the job scheduler that collects and reports the job information to the interconnect controller is developed and demonstrated that applications could gain up to 2.56× speedup in communication. Expand
Scheduling of distributed applications for high performance computing as a service
TLDR
Experimental results showed that the proposed method for scheduling parallel applications in HPCaaS cloud gives significantly better utilization of computational resources than classical scheduling methods. Expand
A systematic review of scheduling approaches on multi-tenancy cloud platforms
TLDR
A systematic literature review of research studies related to multi-tenancy scheduling approaches on cloud platforms determine the primary scheduling approaches currently used and the challenges for addressing key multi-Tenancy scheduling issues. Expand
Deployment and Analysis of a Hybrid Shared/Distributed-Memory Parallel Visualization Tool for 3-D Oil Reservoir Grid on OpenStack Cloud Computing
TLDR
This article experimentally study the parallelization of intensive computation for a 3-D (three dimensional) oil reservoir data visualization tool, and proposes a hybrid (shared memory and distributed memory) parallelization technique to adapt with the data processing scalability. Expand
High Performance Computing: ISC High Performance 2020 International Workshops, Frankfurt, Germany, June 21–25, 2020, Revised Selected Papers
TLDR
TyCart is a tool for type-safe checkpoint/restart and extends the memory allocation sanitizer tool TypeART with type asserts, which shows runtime and memory overhead below 5% in smaller benchmarks. Expand
...
1
2
...

References

SHOWING 1-10 OF 27 REFERENCES
ASETS: A SDN Empowered Task Scheduling System for HPCaaS on the Cloud
TLDR
A new scheme called ASETS is presented which targets dynamic configuration and monitoring of cloud networking using SDN to improve the performance of HPC applications and in particular task scheduling for HPC as a service on the cloud (HPCaaS). Expand
Evaluating and Improving the Performance and Scheduling of HPC Applications in Cloud
TLDR
This paper performs comprehensive performance and cost evaluation and analysis of running a set of HPC applications on a range of platforms, varying from supercomputers to clouds, and presents novel heuristics for online application-aware job scheduling in multi-platform environments. Expand
Environment-conscious scheduling of HPC applications on distributed Cloud-oriented data centers
TLDR
This work proposes near-optimal scheduling policies that exploit heterogeneity across multiple data centers for a Cloud provider that are able to achieve on average up to 25% of energy savings in comparison to profit based scheduling policies leading to higher profit and less carbon emissions. Expand
CLOUDRB: A framework for scheduling and managing High-Performance Computing (HPC) applications in science cloud
TLDR
A CLOUD Resource Broker (CLOUDRB) for efficiently managing cloud resources and completing jobs for scientific applications within a user-specified deadline is designed and integrated with a Deadline-based Job Scheduling and Particle Swarm Optimization-based Resource Allocation mechanism. Expand
Scheduling strategies for HPC as a service (HPCaaS)
TLDR
The paper reviews the concept of HPCaaS and explores a smart scheduling algorithm for a subset of bioscience applications and shows that smart scheduling can accommodate multiple applications and multiple jobs simultaneously while increasing the overall system productivity and efficiency. Expand
Scheduling Concurrent Workflows in HPC Cloud through Exploiting Schedule Gaps
TLDR
This paper proposes a method which exploits schedule gaps to efficiently schedule concurrent workflows in HPC cloud and can deliver good performance and outperform the existing method significantly in terms of average makespan, up to 18% performance improvement. Expand
Moldable Job Scheduling for HPC as a Service
TLDR
A moldable job scheduling approach is proposed which relieves HPC users’ burden of selecting an appropriate number of processors and can achieve even better system performance than existing job scheduling methods. Expand
Evaluating GPU Passthrough in Xen for High Performance Cloud Computing
TLDR
Results show PCI passthrough of GPUs within virtual machines is a viable use case for many scientific computing workflows, and could help support high performance cloud infrastructure in the near future. Expand
Bandwidth-Aware Scheduling With SDN in Hadoop: A New Trend for Big Data
TLDR
BASS is the first to exploit talent of SDN for job scheduling of big data processing and believes that it points out a new trend for large-scale data processing. Expand
Cloud computing networking: challenges and opportunities for innovations
TLDR
This work presents the networking issues in IaaS and federation challenges that are currently addressed with existing technologies and presents innovative software-defined networking proposals, which are applied to some of the challenges and could be used in future deployments as efficient solutions. Expand
...
1
2
3
...