Convergence of artificial intelligence and high performance computing on NSF-supported cyberinfrastructure

  title={Convergence of artificial intelligence and high performance computing on NSF-supported cyberinfrastructure},
  author={Eliu A. Huerta and Asad Khan and Edward Davis and Colleen Bushell and William Gropp and Daniel S. Katz and V. Kindratenko and Seid Koric and William T. C. Kramer and Brendan McGinty and Kenton McHenry and Aaron Saxton},
  journal={Journal of Big Data},
Significant investments to upgrade and construct large-scale scientific facilities demand commensurate investments in R&D to design algorithms and computing approaches to enable scientific and engineering breakthroughs in the big data era. Innovative Artificial Intelligence (AI) applications have powered transformational solutions for big data challenges in industry and technology that now drive a multi-billion dollar industry, and which play an ever increasing role shaping human social… 

AIPerf: Automated machine learning as an AI-HPC benchmark

This work proposes an end-to-end benchmark suite utilizing automated machine learning (AutoML) that represents real AI scenarios and implements the algorithms in a highly parallel and flexible way to ensure the efficiency and optimization potential on diverse systems with customizable configurations.

Advances in Machine and Deep Learning for Modeling and Real-time Detection of Multi-Messenger Sources

Pioneering efforts to adapt artificial intelligence algorithms to address computational grand challenges in Multi-Messenger Astrophysics are described.

Inference-Optimized AI and High Performance Computing for Gravitational Wave Detection at Scale

An inference-optimized AI ensemble that identifies all known binary black hole mergers previously identified in this advanced LIGO dataset and reports no misclassifications, while also providing a 3X inference speedup compared to traditional artificial intelligence models is introduced.

Fighting viruses with computers, right now.

A New Generation of ResNet Model Based on Artificial Intelligence and Few Data Driven and Its Construction in Image Recognition Model

The paper proposes an A-ResNet model to improve ResNet. The residual attention module with shortcut connection is introduced to enhance the focus on the target object; the dropout layer is introduced

Interpreting a Machine Learning Model for Detecting Gravitational Waves

A case study of translational research, applying interpretability techniques developed for computer vision to machine learning models used to search for and find gravitational waves, which suggests that different parts of the network appear to specialize in local versus global features.

Deep Learning for Cardiologist-level Myocardial Infarction Detection in Electrocardiograms

This article studies the feasibility of using deep learning to identify suggestive electrocardiographic changes that may correctly classify heart conditions using the Physikalisch-Technische Bundesanstalt (PTB) database, and fine-tune the ConvNetQuake neural network model.

National Power After AI

AI technologies will likely alter great power competitions in foundational ways, changing both how nations create power and their motives for wielding it against one another. This paper is a first



Big data and extreme-scale computing

It is argued that the rapid proliferation of digital data generators, the unprecedented growth in the volume and diversity of the data they generate, and the intense evolution of the methods for analyzing and using that data are radically reshaping the landscape of scientific computing.

Enabling real-time multi-messenger astrophysics discoveries with deep learning

The key challenges of real-time observations of gravitational wave sources and their electromagnetic and astroparticle counterparts are reviewed, and a number of recommendations to maximize their potential for scientific discovery are made.

DLHub: Model and Data Serving for Science

This work presents the Data and Learning Hub for science (DLHub), a multi-tenant system that provides both model repository and serving capabilities with a focus on science applications and shows that relative to other model serving systems, DLHub provides greater capabilities, comparable performance without memoization and batching, and significantly better performance when the latter two techniques can be employed.

Training distributed deep recurrent neural networks with mixed precision on GPU clusters

A distributed, data-parallel, synchronous training algorithm is implemented by integrating TensorFlow and CUDA-aware MPI to enable execution across multiple GPU nodes and making use of high-speed interconnects, facilitating neural network convergence at up to O(100) workers.

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

The TensorFlow interface and an implementation of that interface that is built at Google are described, which has been used for conducting research and for deploying machine learning systems into production across more than a dozen areas of computer science and other fields.

The unreasonable effectiveness of deep learning in artificial intelligence

  • T. Sejnowski
  • Computer Science
    Proceedings of the National Academy of Sciences
  • 2020
Deep learning was inspired by the architecture of the cerebral cortex and insights into autonomy and general intelligence may be found in other brain regions that are essential for planning and survival, but major breakthroughs will be needed to achieve these goals.

HAL: Computer System for Scalable Deep Learning

A custom management software stack is built to enable an efficient use of the system by a diverse community of users and provide guides and recipes for running deep learning workloads at scale utilizing all available GPUs.

Brown Dog: Leveraging everything towards autocuration

We present Brown Dog, two highly extensible services that aim to leverage any existing pieces of code, libraries, services, or standalone software (past or present) towards providing users with a

Machine learning accelerated topology optimization of nonlinear structures

Two convolutional neural network models are developed to predict optimized designs for a given set of boundary conditions, loads, and volume constraints and are capable of accurately predicting the optimized designs without requiring an iterative scheme and with negligible computational time.

An effective algorithm for hyperparameter optimization of neural networks

This paper addresses the problem of choosing appropriate parameters for the NN by formulating it as a box-constrained mathematical optimization problem, and applying a derivative-free optimization tool that automatically and effectively searches the parameter space.