• Corpus ID: 221554270

Improved Trainable Calibration Method for Neural Networks on Medical Imaging Classification

@article{Liang2020ImprovedTC,
  title={Improved Trainable Calibration Method for Neural Networks on Medical Imaging Classification},
  author={Gongbo Liang and Yu Zhang and Xiaoqin Wang and Nathan Jacobs},
  journal={ArXiv},
  year={2020},
  volume={abs/2009.04057}
}
Recent works have shown that deep neural networks can achieve super-human performance in a wide range of image classification tasks in the medical imaging domain. However, these works have primarily focused on classification accuracy, ignoring the important role of uncertainty quantification. Empirically, neural networks are often miscalibrated and overconfident in their predictions. This miscalibration could be problematic in any automatic decision-making system, but we focus on the medical… 
View PDF on arXiv
25 Citations
Highly Influential Citations
5
Background Citations
13
Methods Citations
13
Results Citations
1

Figures and Tables from this paper

25 Citations

Deep learning model calibration for improving performance in class-imbalanced medical image classification tasks

A systematic analysis of the effect of model calibration on its performance on two medical image modalities, namely, chest X-rays and fundus images, using various deep learning classifier backbones indicates that at the default classification threshold of 0.5, the performance achieved through calibration is significantly superior to using uncalibrated probabilities.

Multi-Head Multi-Loss Model Calibration

This paper introduces a form of simplified ensembling that bypasses the costly training and inference of deep ensembles, yet it keeps its calibration capabilities, and indicates that Multi-Head Multi-Loss classifiers are inherently well-calibrated, outperforming other recent calibration techniques and even challenging Deep Ensembles' performance.

Beware the Black-Box of Medical Image Generation: an Uncertainty Analysis by the Learned Feature Space

  • Yunni QuDavid Yan G. Liang
  • Computer Science
    2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)
  • 2022
It is demonstrated that the learned feature spaces of multiple U-Net architectures for image generation tasks are easily separable between different training trials of the same architecture with the same hyperparameter setting, indicating the models using different criteria for the same tasks.

Multi-loss ensemble deep learning for chest X-ray classification

This work benchmarked various state-of-the-art loss functions that are suitable for multi-class classification, critically analyze model performance, and propose improved loss functions to improve classification performance.

Colorectal polyp classification using confidence-calibrated convolutional neural networks

The results show that the proposed model for CRP classification with confidence calibration produces better calibrated predictions without sacrificing classification performance, and even a slightly improved classification performance is observed, compared to the uncalibrated equivalent.

A Stitch in Time Saves Nine: A Train-Time Regularizing Loss for Improved Neural Network Calibration

This paper argues for intervening at the train time itself, so as to directly produce calibrated dnn models, and proposes a novel auxiliary loss function: Multi-class Difference in Confidence and Accuracy (mdca), to achieve the same.

Towards Improving Calibration in Object Detection Under Domain Shift

A new, plug-and-play, train-time calibration loss for object detection (coined as TCD) that can be used with various application-specific loss functions as an auxiliary loss function to improve detection calibration and a new implicit calibration technique for improving calibration in self-training based domain adaptive detectors.

Weakly-Supervised Feature Learning via Text and Image Matching

The key idea is to use a contrastive loss to train image and text feature extractors to recognize if a given image-finding pair is a true match, then fine-tuned, in a transfer learning setting, for a supervised classification task.

A survey on artificial intelligence in histopathology image analysis

An overview of various learning‐based, XAI, and actionable techniques relevant to deep learning methods in histopathological imaging is presented and the evaluation of XAI methods and the need to ensure their reliability on the field is addressed.

NCTV: Neural Clamping Toolkit and Visualization for Neural Network Calibration

The Neural Clamping Toolkit is introduced, the first open-source framework designed to help developers employ state-of-the-art model-agnostic calibrated models and provides anima- tions and interactive sections in the demonstration to familiarize researchers with calibration in neural networks.

References

SHOWING 1-10 OF 41 REFERENCES

Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss

This paper introduces a new CT image denoising method based on the generative adversarial network (GAN) with Wasserstein distance and perceptual similarity that is capable of not only reducing the image noise level but also trying to keep the critical information at the same time.

U-Net: Convolutional Networks for Biomedical Image Segmentation

It is shown that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks.

On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks

DNNs trained with mixup are significantly better calibrated and are less prone to over-confident predictions on out-of-distribution and random-noise data, suggesting that mixup be employed for classification tasks where predictive uncertainty is a significant concern.

2D Convolutional Neural Networks for 3D Digital Breast Tomosynthesis Classification

This work proposes a novel deep 2D convolutional neural network (CNN) architecture for DBT classification that simultaneously overcomes both challenges of handling the variable number of slices and retaining slice-to-slice changes.

Detecting and classifying lesions in mammograms with Deep Learning

A CAD system based on one of the most successful object detection frameworks, Faster R-CNN, that detects and classifies malignant or benign lesions on a mammogram without any human intervention is proposed.

Inconsistent Performance of Deep Learning Models on Mammogram Classification.

Overview of deep learning in medical imaging

  • Kenji Suzuki
  • Computer Science
    Radiological Physics and Technology
  • 2017
It is shown that ML with feature input (or feature-based ML) was dominant before the introduction of deep learning, and that the major and essential difference between ML before and after deep learning is the learning of image data directly without object segmentation or feature extraction; thus, it is the source of the power of deepLearning.

GANai: Standardizing CT Images using Generative Adversarial Network with Alternative Improvement

A new GAN model called GANai is presented to mitigate the differences in radiomic features across CT images captured using non-standard imaging protocols, and is significantly better than the existing state-of-the-art image synthesis algorithms on CT image standardization.

Deep learning in medical imaging and radiation therapy.

The general principles of DL and convolutional neural networks are introduced, five major areas of application of DL in medical imaging and radiation therapy are surveyed, common themes are identified, methods for dataset expansion are discussed, and lessons learned, remaining challenges, and future directions are summarized.

Dermatologist-level classification of skin cancer with deep neural networks

This work demonstrates an artificial intelligence capable of classifying skin cancer with a level of competence comparable to dermatologists, trained end-to-end from images directly, using only pixels and disease labels as inputs.