Multi-scale fully convolutional neural networks for histopathology image segmentation: from nuclear aberrations to the global tissue architecture

@article{Schmitz2021MultiscaleFC,
  title={Multi-scale fully convolutional neural networks for histopathology image segmentation: from nuclear aberrations to the global tissue architecture},
  author={R{\"u}diger Schmitz and Frederic Madesta and Maximilian Nielsen and Ren{\'e} Werner and Thomas R{\"o}sch},
  journal={Medical image analysis},
  year={2021},
  volume={70},
  pages={
          101996
        }
}
Histopathologic diagnosis relies on simultaneous integration of information from a broad range of scales, ranging from nuclear aberrations (≈O(0.1μm)) through cellular structures (≈O(10μm)) to the global tissue architecture (⪆O(1mm)). To explicitly mimic how human pathologists combine multi-scale information, we introduce a family of multi-encoder fully-convolutional neural networks with deep fusion. We present a simple block for merging model paths with differing spatial scales in a spatial… 

Figures and Tables from this paper

A Deep Learning Framework for Viable Tumor Burden Estimation
Liver masses have become a common clinical challenge since they require to be defined and accurately categorized as neoplastic or nonneoplastic lesions. Hepatocellular carcinoma (HCC), the most
A State-of-the-art Survey of Artificial Neural Networks for Whole-slide Image Analysis: from Popular Convolutional Neural Networks to Potential Visual Transformers
TLDR
WSI analysis methods based on ANN are reviewed, the development status of WSI and ANN methods is introduced, and the application prospect of the analytical method is discussed.
Development and operation of a digital platform for sharing pathology image data
TLDR
The development of a platform for providing and sharing digital pathology data to AI researchers, and challenges to overcome in operating a sustainable platform in conjunction with pathologists are described.
PAIP 2019: Liver cancer segmentation challenge
TLDR
The PAIP Liver Cancer Segmentation Challenge, organized in conjunction with the Medical Image Computing and Computer Assurance Society (MICCAI 2019), is the first image analysis challenge to apply PAIP datasets and has the potential to aid the development and benchmarking of cancer diagnosis and segmentation.
Artificial intelligence in GI endoscopy: stumbling blocks, gold standards and the role of endoscopy societies
TLDR
An accessible guide to understand, assess and critically review the current ML endoscopy literature and proposes the establishment of a ‘rolling gold standard’ to meet requirements for continuous retesting and benchmarking of AI systems.
Widening the Focus: Biomedical Image Segmentation Challenges and the Underestimated Role of Patch Sampling and Inference Strategies
TLDR
This paper presents a meta-modelling framework that automates the very labor-intensive and therefore time-heavy and therefore expensive process of designing and training image recognition systems.
Hybrid guiding: A multi-resolution refinement approach for semantic segmentation of gigapixel histopathological images
TLDR
A cascaded convolutional neural network design, called H2G-Net, is proposed for semantic segmentation of gigapixel histopathological images and it is shown that deep neural networks can be trained using a random sampling scheme that balances on multiple different labels simultaneously, without the need of storing patches on disk.
...
1
2
...

References

SHOWING 1-10 OF 54 REFERENCES
Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images
TLDR
A context-aware stacked convolutional neural networks (CNN) for classification of breast WSIs into normal/benign, DCIS, and invasive ductal carcinoma (IDC) is presented, demonstrating its potential for routine diagnostics.
A dense multi-path decoder for tissue segmentation in histopathology images
TLDR
The proposed decoder is built upon a multi-path layout and dense shortcut connections between layers to maximize the learning and inference capability and could aid in improving tissue analysis in histopathology images.
U-Net: Convolutional Networks for Biomedical Image Segmentation
TLDR
It is shown that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks.
Y-Net: Joint Segmentation and Classification for Diagnosis of Breast Biopsy Images
TLDR
Y-Net extends and generalizes U-Net by adding a parallel branch for discriminative map generation and by supporting convolutional block modularity, which allows the user to adjust network efficiency without altering the network topology.
H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes
TLDR
This work proposes a novel hybrid densely connected UNet (H-DenseUNet), which consists of a 2-D Dense UNet for efficiently extracting intra-slice features and a 3-D counterpart for hierarchically aggregating volumetric contexts under the spirit of the auto-context algorithm for liver and tumor segmentation.
Clinical-grade computational pathology using weakly supervised deep learning on whole slide images
TLDR
A multiple instance learning-based deep learning system that uses only the reported diagnoses as labels for training, thereby avoiding expensive and time-consuming pixel-wise manual annotations, and has the ability to train accurate classification models at unprecedented scale.
Detecting Cancer Metastases on Gigapixel Pathology Images
TLDR
This work presents a framework to automatically detect and localize tumors as small as 100 x 100 pixels in gigapixel microscopy images sized 100,000 x100,000 pixels and achieves image-level AUC scores above 97% on both the Camelyon16 test set and an independent set of 110 slides.
A Multi-scale U-Net for Semantic Segmentation of Histological Images from Radical Prostatectomies
TLDR
This paper proposes a multi-scale U-Net model to classify images at the pixel-level using 224 histological image tiles from radical prostatectomies of 20 patients, outperforming other methods.
Attention U-Net: Learning Where to Look for the Pancreas
TLDR
A novel attention gate (AG) model for medical imaging that automatically learns to focus on target structures of varying shapes and sizes is proposed to eliminate the necessity of using explicit external tissue/organ localisation modules of cascaded convolutional neural networks (CNNs).
RMDL: Recalibrated multi-instance deep learning for whole slide gastric image classification
TLDR
This paper first select the discriminative instances, and then utilize these instances to diagnose diseases based on the proposed RMDL approach, and builds a large whole-slide gastric histopathology image dataset with detailed pixel-level annotations.
...
1
2
3
4
5
...