Objective Evaluation of Deep Uncertainty Predictions for COVID-19 Detection

@article{Asgharnezhad2020ObjectiveEO,
  title={Objective Evaluation of Deep Uncertainty Predictions for COVID-19 Detection},
  author={Hamzeh Asgharnezhad and Afshar Shamsi Jokandan and Roohallah Alizadehsani and Abbas Khosravi and Saeid Nahavandi and Zahra Alizadeh Sani and Dipti Srinivasan},
  journal={ArXiv},
  year={2020},
  volume={abs/2012.11840}
}
Deep neural networks (DNNs) have been widely applied for detecting COVID-19 in medical images. Existing studies mainly apply transfer learning and other data representation strategies to generate accurate point estimates. The generalization power of these networks is always questionable due to being developed using small datasets and failing to report their predictive confidence. Quantifying uncertainties associated with DNN predictions is a prerequisite for their trusted deployment in… 

Figures and Tables from this paper

Confidence Aware Neural Networks for Skin Cancer Detection
TLDR
This work presents three different methods for quantifying uncertainties for skin cancer detection from images and comprehensively evaluates and compares performance of these DNNs using novel uncertainty-related metrics.
Improving Uncertainty Estimation With Semi-Supervised Deep Learning for COVID-19 Detection Using Chest X-Ray Images
TLDR
This work implements a COVID-19 infection detection system based on chest X-ray images with uncertainty estimation, and proposes the usage of the Jensen-Shannon distance between the uncertainty distributions of correct and incorrect estimations.
Uncertainty-Aware Semi-Supervised Method Using Large Unlabeled and Limited Labeled COVID-19 Data
TLDR
A Semi-supervised Classification using Limited Labeled Data (SCLLD) relying on Sobel edge detection and Generative Adversarial Networks (GANs) to automate the COVID-19 diagnosis, which significantly outperforms the supervised training of Convolutional Neural Network (CNN) when labeled training data is scarce.
Combining a convolutional neural network with autoencoders to predict the survival chance of COVID-19 patients
TLDR
This work proposes a novel method, named the CNN-AE, to predict the survival chance of COVID-19 patients using a CNN trained with clinical information, and demonstrates the generality of the method using a publicly available clinical dataset.
Improving Uncertainty Estimations for Mammogram Classification using Semi-Supervised Learning
TLDR
The impact of using unlabeled data through the implementation of a recent approach, known as MixMatch, for mammogram images is explored and the usage of the uncertainty balanced accuracy metric is proposed.
CNN AE: Convolution Neural Network combined with Autoencoder approach to detect survival chance of COVID 19 patients
TLDR
Experimental results reveal the superiority of CNN-AE method compared to the standard CNN as well as other methods such as random forest and Naive Bayes, and demonstrate the generality of the augmentation method.
Out of Distribution Detection and Adversarial Attacks on Deep Neural Networks for Robust Medical Image Analysis
TLDR
This work experimentally evaluates the robustness of a Mahalanobis distance-based confidence score, a simple yet effective method for detecting abnormal input samples, in classifying malaria parasitized cells and uninfected cells, and achieves state of theart performance on both out-of-distribution (OOD) and adversarial samples.
Uncertainty-Aware Credit Card Fraud Detection Using Deep Learning
TLDR
This study proposes three uncertainty quantification (UQ) techniques named Monte Carlo dropout, ensemble, and ensemble Monte Carlodropout for card fraud detection applied on transaction data and demonstrates that the ensemble is more effective in capturing uncertainty corresponding to generated predictions.
Improving MC-Dropout Uncertainty Estimates with Calibration Error-based Optimization
TLDR
This study proposes two new loss functions by combining cross entropy with Expected Calibration Error (ECE) and Predictive Entropy (PE) and shows that the new proposed loss functions lead to having a calibrated MC-Dropout method.
Fusion of convolution neural network, support vector machine and Sobel filter for accurate detection of COVID-19 patients using X-ray images
TLDR
A fusion of convolutional neural network, support vector machine, and Sobel filter is proposed to detect COVID-19 using X-ray images and showed that using Sobel Filter can improve the performance of CNN.
...
1
2
...

References

SHOWING 1-10 OF 34 REFERENCES
Estimating Uncertainty and Interpretability in Deep Learning for Coronavirus (COVID-19) Detection
TLDR
This paper investigates how drop-weights based Bayesian Convolutional Neural Networks (BCNN) can estimate uncertainty in Deep Learning solution to improve the diagnostic performance of the human-machine team using publicly available COVID-19 chest X-ray dataset and shows that the uncertainty in prediction is highly correlates with accuracy of prediction.
An Uncertainty-Aware Transfer Learning-Based Framework for COVID-19 Diagnosis
TLDR
Comprehensive simulation results for X-ray and CT image data sets indicate that linear support vector machine and neural network models achieve the best results as measured by accuracy, sensitivity, specificity, and area under the receiver operating characteristic (ROC) curve (AUC).
Estimating Uncertainty in Deep Learning for Reporting Confidence to Clinicians when Segmenting Nuclei Image Data
TLDR
The proposed uncertainty quantification framework using the Bayesian Residual U-Net (BRUNet) provides additional insight for clinicians when analysing images with help from deep learners and demonstrates how the resulting uncertainty depends on the dropout rates using images from nuclei in divergent medical images.
Can Your AI Differentiate Cats from Covid-19?Sample Efficient Uncertainty Estimation for Deep Learning Safety
Deep Neural Networks (DNNs) are known to make highly overconfident predictions on Outof-Distribution data. Recent research has shown that uncertainty-aware models, such as, Bayesian Neural Network
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
TLDR
A labeler is designed to automatically detect the presence of 14 observations in radiology reports, capturing uncertainties inherent in radiograph interpretation, in CheXpert, a large dataset that contains 224,316 chest radiographs of 65,240 patients.
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
TLDR
This work proposes an alternative to Bayesian NNs that is simple to implement, readily parallelizable, requires very little hyperparameter tuning, and yields high quality predictive uncertainty estimates.
A Review on Deep Learning Techniques for the Diagnosis of Novel Coronavirus (COVID-19)
TLDR
This review specifically discusses the systems developed for COVID-19 diagnosis using deep learning techniques and provides insights on well-known data sets used to train these networks and highlights the data partitioning techniques and various performance measures developed by researchers in this field.
Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation, and Diagnosis for COVID-19
  • F. Shi, J. Wang, +6 authors D. Shen
  • Engineering, Computer Science
    IEEE Reviews in Biomedical Engineering
  • 2021
TLDR
This review paper covers the entire pipeline of medical imaging and analysis techniques involved with COVID-19, including image acquisition, segmentation, diagnosis, and follow-up, and particularly focuses on the integration of AI with X-ray and CT, both of which are widely used in the frontline hospitals.
Chest Radiograph Interpretation with Deep Learning Models: Assessment with Radiologist-adjudicated Reference Standards and Population-adjusted Evaluation.
TLDR
Expert-level models for detecting clinically relevant chest radiograph findings were developed for this study by using adjudicated reference standards and with population-level performance estimation.
Uncertainty-Aware Audiovisual Activity Recognition Using Deep Bayesian Variational Inference
TLDR
A novel approach that combines deterministic and variational layers to scale Bayesian DNNs to deeper architectures is demonstrated and can identify out-of-distribution data on the UCF101 and MiT datasets.
...
1
2
3
4
...