Evaluation of Machine Learning Techniques for Forecast Uncertainty Quantification

  title={Evaluation of Machine Learning Techniques for Forecast Uncertainty Quantification},
  author={Maximiliano A. Sacco and Juan Jos{\'e} Ruiz and Manuel Pulido and Pierre Tandeo},
  journal={Quarterly Journal of the Royal Meteorological Society},
  • M. SaccoJ. J. Ruiz P. Tandeo
  • Published 29 November 2021
  • Environmental Science
  • Quarterly Journal of the Royal Meteorological Society
model duce a reliable quantification of the forecast uncertainty. in good agreement with the location of larger forecast errors. It is important to note that we do not expect a perfect match between these two quantities since the forecast error is one realization from a probability density function whose standard deviation is being approximated from the forecast ensemble. The ANN also provides an estimation of the forecast error standard deviation in good agreement with the evolution of the… 

Figures from this paper

Uncertainty-aware Evaluation of Machine Learning Performance in binary Classification Tasks

Evaluation metrics for machine learning approaches that are able to attach a probability distribution to the utilized threshold and include uncertainty measures are presented.



Data assimilation in the geosciences: An overview of methods, issues, and perspectives

We commonly refer to state estimation theory in geosciences as data assimilation (DA). This term encompasses the entire sequence of operations that, starting from the observations of a system, and

Estimation of the functional form of subgrid‐scale parametrizations using ensemble‐based data assimilation: a simple model experiment

Oceanic and atmospheric global numerical models represent explicitly the large‐scale dynamics while the smaller‐scale processes are not resolved, so that their effects in the large‐scale dynamics are

Probabilistic forecasts, calibration and sharpness

Summary.  Probabilistic forecasts of continuous variables take the form of predictive densities or predictive cumulative distribution functions. We propose a diagnostic approach to the evaluation of

Deep Uncertainty Quantification: A Machine Learning Approach for Weather Forecasting

This paper designs a data-driven method augmented by an effective information fusion mechanism to learn from historical data that incorporates prior knowledge from NWP by proposing a novel negative log-likelihood error (NLE) loss function.

Predictability: a problem partly solved

  • Shinfield Park, Reading: ECMWF;
  • 1995

GEFSv12 reforecast dataset for supporting subseasonal and hydrometeorological applications

  • Environmental Science
    Monthly Weather Review
  • 2022
For the newly implemented Global Ensemble Forecast System version 12 (GEFSv12), a 31-year (1989-2019) ensemble reforecast dataset has been generated at the National Centers for Environmental

Probabilistic Predictions from Deterministic Atmospheric River Forecasts with Deep Learning

Deep Learning (DL) post-processing methods are examined to obtain reliable and accurate probabilistic forecasts from single-member numerical weather predictions of integrated vapor transport (IVT).

A comparison of combined data assimilation and machine learning methods for offline and online model error correction

Statistical Postprocessing of Wind Speed Forecasts Using Convolutional Neural Networks

The probabilistic forecasts from the CNNs are shown to have higher Brier skill scores for medium to higher wind speeds, as well as a better continuous ranked probability score (CRPS) and logarithmic score, than the forecasts from fully connected neural networks and quantile regression forests.