• Corpus ID: 237532741

Machine-Learned HASDM Model with Uncertainty Quantification

@article{Licata2021MachineLearnedHM,
  title={Machine-Learned HASDM Model with Uncertainty Quantification},
  author={Richard Joseph Licata and Piyush M. Mehta and W. Kent Tobiska and Snehalata V. Huzurbazar},
  journal={ArXiv},
  year={2021},
  volume={abs/2109.07651}
}

• R. Licata, P. Mehta, +1 author S. Huzurbazar
• Published 16 September 2021
• Computer Science
• ArXiv

The first thermospheric neutral mass density model with robust and reliable uncertainty estimates is developed based on the SET HASDM density database. This database, created by Space Environment Technologies (SET), contains 20 years of outputs from the U.S. Space Force’s High Accuracy Satellite Drag Model (HASDM), which represents the state-of-the-art for density and drag modeling. We utilize principal component analysis (PCA) for dimensionality reduction, creating the coefficients upon which nonlinear machinelearned (ML) regression models are trained. These models use three unique loss functions: mean square error (MSE), negative logarithm of predictive density (NLPD), and continuous ranked probability score (CRPS). Three input sets are also tested, showing improved performance when introducing time histories for geomagnetic indices. These models leverage Monte Carlo (MC) dropout to provide uncertainty estimates, and the use of the NLPD loss function results in well-calibrated uncertainty estimates without sacrificing model accuracy (<10% mean absolute error). By comparing the best HASDM-ML model to the HASDM database along satellite orbits, we found that the model provides robust and reliable uncertainties in the density space over all space weather conditions. A storm-time comparison shows that HASDM-ML also supplies meaningful uncertainty measurements during extreme events. Plain Language Summary The first upper-atmospheric density model with robust and reliable uncertainty estimates is developed based on the SET HASDM density database. This database contains 20 years of outputs from the High Accuracy Satellite Drag Model (HASDM), which represents the state-of-the-art for density and drag modeling. We use a decomposition tool called principal component analysis (PCA) to reduce the dimensionality of the dataset. Three loss functions, mean square error (MSE), negative logarithm of predictive density (NLPD), and continuous ranked probability score (CRPS), are tested with three input sets to identify the best-performing model. We optimize nine models (all three loss functions and input sets) and compare the prediction accuracy and the reliability of its uncertainty estimates. The models leverage Monte Carlo dropout to generate probabilistic outputs from which we extract model uncertainty. We find that using an input set containing a time series for the geomagnetic indices results in the most accurate models. In addition, the model using these inputs with the NLPD loss function has sufficient performance (∼10% absolute error) and the most calibrated/reliable uncertainty estimates on independent data. We test this model’s uncertainty capabilities in the density space along satellite orbits from 2002-2010, showing the model’s reliability across all conditions.