• Corpus ID: 234680952

Toward Real-time Analysis of Experimental Science Workloads on Geographically Distributed Supercomputers

@article{Salim2021TowardRA,
  title={Toward Real-time Analysis of Experimental Science Workloads on Geographically Distributed Supercomputers},
  author={Michael A. Salim and Thomas D. Uram and J. Taylor Childers and Venkatram Vishwanath and Michael E. Papka},
  journal={ArXiv},
  year={2021},
  volume={abs/2105.06571}
}
Massive upgrades to science infrastructure are driving data velocities upwards while stimulating adoption of increasingly dataintensive analytics. While next-generation exascale supercomputers promise strong support for I/O-intensive workflows, HPC remains largely untapped by live experiments, because data transfers and disparate batch-queueing policies are prohibitive when faced with scarce instrument time. To bridge this divide, we introduce Balsam: a distributed orchestration platform… 
Intelligent Resolution: Integrating Cryo-EM with AI-driven Multi-resolution Simulations to Observe the SARS-CoV-2 Replication-Transcription Machinery in Action
TLDR
This study provides unprecedented access to study the SARS-CoV-2 RTC machinery, while providing general capability for AI-enabled multi-resolution simulations at scale, and develops an innovative workflow that bridges the gap between these resolutions.
ExaWorks: Workflows for Exascale
TLDR
The ExaWorks project is leading a co-design process to create a workflow Software Development Toolkit (SDK) consisting of a wide range of workflow management tools that can be composed and interoperate through common interfaces.

References

SHOWING 1-10 OF 31 REFERENCES
Outlook for artificial intelligence and machine learning at the NSLS-II
TLDR
The opportunity for using the AI/ML tools and techniques developed in the data and computational science areas to greatly improve the scientific output of large scale experimental user facilities is discussed.
Serverless Containers – Rising Viable Approach to Scientific Workflows
TLDR
Evaluating the capabilities of elastic containers and their usefulness for scientific computing in the scientific workflow paradigm using AWS Fargate and Google Cloud Run infrastructures shows that serverless containers can be successfully utilized for running scientific workflows.
Cross-facility science with the Superfacility Project at LBNL
  • B. Enders, D. Bard, +20 authors Gabor Torok
  • Computer Science
    2020 IEEE/ACM 2nd Annual Workshop on Extreme-scale Experiment-in-the-Loop Computing (XLOOP)
  • 2020
TLDR
Recent innovations in the Superfacility project are reported, including advanced data management, API-based automation, real-time interactive user interfaces, and supported infrastructure for “edge” services.
Toward Interoperable Cyberinfrastructure: Common Descriptions for Computational Resources and Applications
TLDR
A solution to the problem of a common description and catalog of resources is sketched, an approach to implementing a resource registry for use by the community is described, and potential approaches to some long-term challenges are discussed.
funcX: A Federated Function Serving Fabric for Science
Exploding data volumes and velocities, new computational methods and platforms, and ubiquitous connectivity demand new approaches to computation in the sciences. These new approaches must enable
Balsam: Near Real-Time Experimental Data Analysis on Supercomputers
TLDR
This work describes how the Balsam edge service has enabled near-real-time analysis of data collected at the Advanced Photon Source with an X-ray Photon Correlation Spectroscopy application running on Argonne Leadership Computing Facility (ALCF) resources.
OAuth 2.0 Device Authorization Grant
The OAuth 2.0 Device Authorization Grant is designed for internet- connected devices that either lack a browser to perform a user-agent based authorization, or are input-constrained to the extent
Parsl: Pervasive Parallel Programming in Python
TLDR
Parsl is a parallel scripting library that augments Python with simple, scalable, and flexible constructs for encoding parallelism that satisfy the needs of many-task, interactive, online, and machine learning applications in fields such as biology, cosmology, and materials science.
Serverless Science for Simple, Scalable, and Shareable Scholarship
  • K. Chard, I. Foster
  • Computer Science
    2019 15th International Conference on eScience (eScience)
  • 2019
TLDR
To demonstrate how serverless computing can transform scientific computing, three serverless Computing models are described: service-oriented computing, research automation, and function as a service, presenting illustrative case studies for each.
...
1
2
3
4
...