Tools for Extracting Spatio-Temporal Patterns in Meteorological Image Sequences: From Feature Engineering to Attention-Based Neural Networks

@article{Bansal2022ToolsFE,
  title={Tools for Extracting Spatio-Temporal Patterns in Meteorological Image Sequences: From Feature Engineering to Attention-Based Neural Networks},
  author={Akansha Singh Bansal and Yoonjin Lee and Kyle Hilburn and Imme Ebert‐Uphoff},
  journal={ArXiv},
  year={2022},
  volume={abs/2210.12310}
}
Atmospheric processes involve both space and time. This is why human analysis of atmospheric imagery can often extract more information from animated loops of image sequences than from individual images. Automat-ing such an analysis requires the ability to identify spatio-temporal patterns in image sequences which is a very challenging task, because of the endless possibilities of patterns in both space and time. In this paper we review different concepts and techniques that are useful to… 
[PDF] Semantic Reader

123 References

Topological data analysis and machine learning for recognizing atmospheric river patterns in large climate datasets

Abstract. Identifying weather patterns that frequently lead to extreme weather events is a crucial first step in understanding how they may vary under different climate change scenarios. Here, we…

A Primer on Topological Data Analysis to Support Image Analysis Tasks in Environmental Science

It is demonstrated how persistent homology and its vectorization, persistence landscapes, can be used in a workflow with a simple machine learning algorithm to obtain good results, and how to explain this behavior in terms of image-level features.

Evaluation, Tuning, and Interpretation of Neural Networks for Working with Images in Meteorological Applications

This article highlights several strategies and practical considerations for neural network development that have not yet received much attention in the meteorological community, such as the concept of receptive fields, underutilized meteorological performance measures, and methods for Neural network interpretation,such as synthetic experiments and layer-wise relevance propagation.

Attention-augmented Spatio-Temporal Segmentation for Land Cover Mapping

A novel architecture is introduced that incorporates the UNet structure with Bidirectional LSTM and Attention mechanism to jointly exploit the spatial and temporal nature of satellite data and to better identify the unique temporal patterns of each land cover.

Panoptic Segmentation of Satellite Image Time Series with Convolutional Temporal Attention Networks

This paper presents the first end-to-end, single-stage method for panoptic segmentation of Satellite Image Time Series (SITS), which can be combined with the novel image sequence encoding network which relies on temporal selfattention to extract rich and adaptive multi-scale spatiotemporal features.

Improving Historical Data Discovery in Weather Radar Image Data Sets Using Transfer Learning

This work illustrates a methodology for performing image classification of precipitation regimes on colormapped weather radar scan images, as opposed to the raw data itself, reducing the data needed to perform this classification by two orders of magnitude and increasing throughput and democratizing usage of the deep learning tools for this task.

A Deep Framework for Eddy Detection and Tracking From Satellite Sea Surface Height Data

This work presents a framework for automatic ocean eddy detection and tracking by leveraging state-of-the-art machine learning algorithms, and proposes a new convolutional neural network model for multieddies detection.

The color out of space: learning self-supervised representations for Earth Observation imagery

This work proposes to learn meaningful representations from satellite imagery, leveraging its high-dimensionality spectral bands to reconstruct the visible colors and qualitatively observes that guesses based on natural images and colorization rely on different parts of the input.

Applying machine learning methods to detect convection using Geostationary Operational Environmental Satellite-16 (GOES-16) advanced baseline imager (ABI) data

A neural network model with convolutional layers is proposed to identify convection from the high-temporal resolution GOES-16 data, and has correctly learned features of convective clouds and resulted in a reasonably low false alarm ratio (FAR) and high probability of detection (POD).

Can we integrate spatial verification methods into neural-network loss functions for atmospheric science?

In the last decade, much work in atmospheric science has focused on spatial verification (SV) methods for gridded prediction, which overcome serious disadvantages of pixelwise verification. However,…
...