Accounting for Dependencies in Deep Learning Based Multiple Instance Learning for Whole Slide Imaging

@article{Myronenko2021AccountingFD,
  title={Accounting for Dependencies in Deep Learning Based Multiple Instance Learning for Whole Slide Imaging},
  author={Andriy Myronenko and Ziyue Xu and Dong Yang and Holger R. Roth and Daguang Xu},
  journal={ArXiv},
  year={2021},
  volume={abs/2111.01556}
}
Multiple instance learning (MIL) is a key algorithm for classification of whole slide images (WSI). Histology WSIs can have billions of pixels, which create enormous computational and annotation challenges. Typically, such images are divided into a set of patches (a bag of instances), where only bag-level class labels are provided. Deep learning based MIL methods calculate instance features using convolutional neural network (CNN). Our proposed approach is also deep learning based, with the… 
DGMIL: Distribution Guided Multiple Instance Learning for Whole Slide Image Classification
TLDR
DGMIL is proposed: a feature distribution guided deep MIL framework for WSI classification and positive patch localization and a pseudo label-based iterative feature space refinement strategy so that in the final feature space the positive and negative instances can be easily separated.
Handcrafted Histological Transformer (H2T): Unsupervised Representation of Whole Slide Images
TLDR
A handcrafted framework based on deep CNN for constructing holistic WSI-level representations that offer competitive performance compared to recent state-of-theart methods and can be readily utilized for various downstream analysis tasks is presented.
Differentiable Zooming for Multiple Instance Learning on Whole-Slide Images
TLDR
Z OOM MIL is proposed, a method that learns to perform multi-level zooming in an end-to-end manner that outperforms the state-of-the-art MIL methods in WSI classification on two large datasets, while reducing the computational demands with regard to Floating-Point Operations (FLOPs) and processing time by up to 40 × .
Multiple Instance Learning with Mixed Supervision in Gleason Grading
TLDR
A mixed supervision Transformer based on the multiple instance learning framework that utilizes both slide-level label and instance-level labels to achieve more accurate Gleason grading at the slide level and achieves the state-of-the-art performance on the SICAPv2 dataset.
Local Attention Graph-based Transformer for Multi-target Genetic Alteration Prediction
TLDR
It is demonstrated that LA-MIL achieves state-of-the-art results in mutation prediction for gastrointestinal cancer, outper-forming existing models on important biomarkers such as microsatellite instability for colorectal cancer, and suggests that local self-attention models dependencies on par with global modules.
RTN: Reinforced Transformer Network for Coronary CT Angiography Vessel-level Image Quality Assessment
TLDR
A P rogressive R einforcement learning based D iscarding module (termed as PRID) to progressively remove quality-irrelevant/negative instances for CCTA VIQA based on end-to-end optimization.
D IFFERENTIABLE Z OOMING FOR M ULTIPLE I NSTANCE L EARNING ON W HOLE -S LIDE I MAGES
TLDR
Z OOM MIL is proposed, a method that learns to perform multi-level zooming in an end-to-end manner that outperforms the state-of-the-art MIL methods in WSI classification on two large datasets, while reducing the computational demands with regard to Floating-Point Operations (FLOPs) and processing time by up to 40 × .

References

SHOWING 1-10 OF 19 REFERENCES
Predicting Lymph Node Metastasis Using Histopathological Images Based on Multiple Instance Learning With Deep Graph Convolution
  • Yu Zhao, Fan Yang, Jianhua Yao
  • Computer Science
    2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2020
TLDR
This paper develops a self-supervised learning mechanism to train the feature extractor based on a combination model of variational autoencoder and generative adversarial network (VAE-GAN) and proposes a novel instance-level feature selection method to select the discriminative instance features.
Attention-based Deep Multiple Instance Learning
TLDR
This paper proposes a neural network-based permutation-invariant aggregation operator that corresponds to the attention mechanism that achieves comparable performance to the best MIL methods on benchmark MIL datasets and outperforms other methods on a MNIST-based MIL dataset and two real-life histopathology datasets without sacrificing interpretability.
Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images
TLDR
The method, which is named clustering-constrained-attention multiple-instance learning (CLAM), uses attention-based learning to identify subregions of high diagnostic value to accurately classify whole slides and instance-level clustering over the identified representative regions to constrain and refine the feature space.
Multi-scale Domain-adversarial Multiple-instance CNN for Cancer Subtype Classification with Unannotated Histopathological Images
TLDR
A new CNN-based cancer subtype classification method is developed by effectively combining multiple-instance, domain adversarial, and multi-scale learning frameworks in order to overcome practical difficulties.
Weakly supervised multiple instance learning histopathological tumor segmentation
TLDR
A weakly supervised framework for whole slide imaging segmentation that relies on standard clinical annotations, available in most medical systems, is proposed that exploits a multiple instance learning scheme for training models.
Clinical-grade computational pathology using weakly supervised deep learning on whole slide images
TLDR
A multiple instance learning-based deep learning system that uses only the reported diagnoses as labels for training, thereby avoiding expensive and time-consuming pixel-wise manual annotations, and has the ability to train accurate classification models at unprecedented scale.
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
TLDR
Vision Transformer (ViT) attains excellent results compared to state-of-the-art convolutional networks while requiring substantially fewer computational resources to train.
AMI-Net+: A Novel Multi-Instance Neural Network for Medical Diagnosis from Incomplete and Imbalanced Data
TLDR
This work introduces a novel multi-instance neural network, AMI-Net+, to train and predict from the incomplete and extremely imbalanced data, and proposes a novel self-adaptive multi- instance pooling method on instance-level to obtain the bag representation.
...
...