Physically Interpretable Neural Networks for the Geosciences: Applications to Earth System Variability

@article{Toms2020PhysicallyIN,
  title={Physically Interpretable Neural Networks for the Geosciences: Applications to Earth System Variability},
  author={Benjamin A. Toms and Elizabeth A. Barnes and Imme Ebert‐Uphoff},
  journal={Journal of Advances in Modeling Earth Systems},
  year={2020},
  volume={12}
}
Neural networks have become increasingly prevalent within the geosciences, although a common limitation of their usage has been a lack of methods to interpret what the networks learn and how they make decisions. As such, neural networks have often been used within the geosciences to most accurately identify a desired output given a set of inputs, with the interpretation of what the network learns used as a secondary metric to ensure the network is making the right decision for the right reason… 
Testing the Reliability of Interpretable Neural Networks in Geoscience Using the Madden-Julian Oscillation
TLDR
The reliability of two neural network interpretation techniques, backward optimization and layerwise relevance propagation, within geoscientific applications are tested by applying them to a commonly studied geophysical phenomenon, the Madden-Julian Oscillation, and it is found that the conventionally defined extended seasons should be shifted later by one month.
Evaluation, Tuning and Interpretation of Neural Networks for Meteorological Applications
TLDR
This article highlights several strategies and practical considerations for neural network development that have not yet received much attention in the meteorological community, such as the concept of effective receptive fields, underutilized meteorological performance measures, and methods for NN interpretation,such as synthetic experiments and layer-wise relevance propagation.
Identifying Opportunities for Skillful Weather Prediction with Interpretable Neural Networks
TLDR
This work demonstrates how neural networks can be used, not only to leverage these states of the climate system to make skillful predictions, but moreover to identify the climatic conditions that lead to enhanced predictability.
Indicator Patterns of Forced Change Learned by an Artificial Neural Network
TLDR
This work trains an artificial neural network (ANN) to identify the year of input maps of temperature and precipitation from forced climate model simulations and applies a neural network visualization technique to visualize the spatial patterns that lead the ANN to successfully predict the year.
EXPLAIN IT TO ME – FACING REMOTE SENSING CHALLENGES IN THE BIO- AND GEOSCIENCES WITH EXPLAINABLE MACHINE LEARNING
TLDR
This work discusses explainable machine learning approaches which are used to tackle common challenges in the bio- and geosciences, such as limited amount of labeled data or the provision of reliable and scientific consistent results.
Interpreting and Stabilizing Machine-Learning Parametrizations of Convection
Neural networks are a promising technique for parameterizing subgrid-scale physics (e.g., moist atmospheric convection) in coarse-resolution climate models, but their lack of interpretability and
Will Artificial Intelligence supersede Earth System and Climate Models?
TLDR
A perspective of an entirely new research branch in Earth and climate sciences, where deep neural networks and Earth system models are dismantled as individual methodological approaches and reassembled as learning, self-validating, and interpretable Earth system model-network hybrids, hereafter named Neural Earth System Modelling (NESYM).
Physics-informed machine learning: case studies for weather and climate modelling
TLDR
This work surveys systematic approaches to incorporating physics and domain knowledge into ML models and distill these approaches into broad categories, and shows how these approaches have been used successfully for emulating, downscaling, and forecasting weather and climate processes.
Bridging observations, theory and numerical simulation of the ocean using machine learning
Progress within physical oceanography has been concurrent with the increasing sophistication of tools available for its study. The incorporation of machine learning (ML) techniques offers exciting
Assessing the Potential of Deep Learning for Emulating Cloud Superparameterization in Climate Models With Real‐Geography Boundary Conditions
We explore the potential of feed‐forward deep neural networks (DNNs) for emulating cloud superparameterization in realistic geography, using offline fits to data from the superparameterized community
...
...

References

SHOWING 1-10 OF 84 REFERENCES
Testing the Reliability of Interpretable Neural Networks in Geoscience Using the Madden-Julian Oscillation
TLDR
The reliability of two neural network interpretation techniques, backward optimization and layerwise relevance propagation, within geoscientific applications are tested by applying them to a commonly studied geophysical phenomenon, the Madden-Julian Oscillation, and it is found that the conventionally defined extended seasons should be shifted later by one month.
Deep Learning for Scientific Inference from Geophysical Data: The Madden-Julian Oscillation as a Test Case
TLDR
It is shown that deep learning can correctly identify geophysical phenomena by "learning" the variables and spatial patterns important to their evolution, and suggest thatDeep learning models are interpretable and viable for scientific inference in geoscientific applications.
Deep learning and process understanding for data-driven Earth system science
TLDR
It is argued that contextual cues should be used as part of deep learning to gain further process understanding of Earth system science problems, improving the predictive ability of seasonal forecasting and modelling of long-range spatial connections across multiple timescales.
Machine Learning for the Geosciences: Challenges and Opportunities
TLDR
Some of the cross-cutting research themes in machine learning that are applicable across several geoscience problems, and the importance of a deep collaboration between machine learning and geosciences for synergistic advancements in both disciplines are discussed.
Machine learning for data-driven discovery in solid Earth geoscience
TLDR
Solid Earth geoscience is a field that has very large set of observations, which are ideal for analysis with machine-learning methods, and how these methods can be applied to solid Earth datasets is reviewed.
Applications of Deep Learning to Ocean Data Inference and Subgrid Parameterization
  • T. Bolton, L. Zanna
  • Environmental Science
    Journal of Advances in Modeling Earth Systems
  • 2019
TLDR
The results indicate that data-driven approaches can be exploited to predict both subgrid and large-scale processes, while respecting physical principles, even when data are limited to a particular region or external forcing.
A Neural Network Approach for a Fast and Accurate Computation of a Longwave Radiative Budget
TLDR
The authors show that their neural network–based model, NeuroFlux, can be used successfully for accurately deriving the longwave radiative budget from the top of the atmosphere to the surface.
Spatially Extended Tests of a Neural Network Parametrization Trained by Coarse‐Graining
TLDR
An NN parametrization trained by coarse-graining a near-global CRM simulation with a 4~km horizontal grid spacing is described, which reveals that it learns to exploit a strong synchrony between precipitation and the atmospheric state above 10 km.
Deep learning for multi-year ENSO forecasts
TLDR
It is shown that a statistical forecast model employing a deep-learning approach produces skilful ENSO forecasts for lead times of up to one and a half years, overcoming a weakness of dynamical forecast models.
Using Machine Learning to Parameterize Moist Convection: Potential for Modeling of Climate, Climate Change, and Extreme Events
The parameterization of moist convection contributes to uncertainty in climate modeling and numerical weather prediction. Machine learning (ML) can be used to learn new parameterizations directly
...
...