METAQ: Bundle Supercomputing Tasks

E. Berkowitz

Corpus ID: 119102244

METAQ: Bundle Supercomputing Tasks

@article{Berkowitz2017METAQBS,
  title={METAQ: Bundle Supercomputing Tasks},
  author={E. Berkowitz},
  journal={arXiv: Computational Physics},
  year={2017}
}

E. Berkowitz
Published 2017
Computer Science, Physics
arXiv: Computational Physics

We describe a light-weight system of bash scripts for efficiently bundling supercomputing tasks into large jobs, so that one can take advantage of incentives or discounts for requesting large allocations. The software can backfill computational tasks, avoiding wasted cycles, and can streamline collaboration between different users. It is simple to use, functioning similarly to batch systems like PBS, MOAB, and SLURM.

View PDF on arXiv

15 Citations

Highly Influential Citations

Background Citations

Methods Citations

Figures and Topics from this paper

figure 1

Job Management with mpi_jm

E. Berkowitz, Gustav R. Jansen, Kenneth McElvain, A. Walker-Loud
Computer Science
ISC Workshops
2018

Autonomous Resource Management for High Performance Datacenters

A. Souza
2020

Application-aware resource management for datacenters

A. C. D. Souza
Computer Science
2018

Hybrid Resource Management for HPC and Data Intensive Workloads

A. Souza, Mohamad Rezaei, E. Laure, Johan Tordsson
Computer Science
2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)
2019

Characterizing the Performance of Executing Many-tasks on Summit

Matteo Turilli, A. Merzky, T. Naughton, W. Elwasif, S. Jha
Computer Science
2019 IEEE/ACM Third Annual Workshop on Emerging Parallel and Distributed Runtime Systems and Middleware (IPDRM)
2019

4
Highly Influenced
PDF

EspressoDB: A scientific database for managing high-performance computing workflows

C. Chang, Christopher Körber, A. Walker-Loud
Computer Science, Physics
J. Open Source Softw.
2020

Simulating the Weak Death of the Neutron in a Femtoscale Universe with Near-Exascale Computing

E. Berkowitz, M. A. Clark, +9 authors K. Orginos
Physics, Computer Science
SC18: International Conference for High Performance Computing, Networking, Storage and Analysis
2018