• Corpus ID: 170079074

A Hierarchical Probabilistic U-Net for Modeling Multi-Scale Ambiguities

  title={A Hierarchical Probabilistic U-Net for Modeling Multi-Scale Ambiguities},
  author={Simon A. A. Kohl and Bernardino Romera-Paredes and Klaus Maier-Hein and Danilo Jimenez Rezende and S. M. Ali Eslami and Pushmeet Kohli and Andrew Zisserman and Olaf Ronneberger},
Medical imaging only indirectly measures the molecular identity of the tissue within each voxel, which often produces only ambiguous image evidence for target measures of interest, like semantic segmentation. [] Key Method We show that this model formulation enables sampling and reconstruction of segmenations with high fidelity, i.e. with finely resolved detail, while providing the flexibility to learn complex structured distributions across scales. We demonstrate these abilities on the task of segmenting…

A Probabilistic Model for Controlling Diversity and Accuracy of Ambiguous Medical Image Segmentation

A novel probabilistic segmentation model, called Joint Probabilistic U-net, is proposed, which successfully achieves flexible control over the two abstract conceptions of diversity and accuracy, and two strategies for preventing the latent space collapse are explored.

Variational inference for quantifying inter-observer variability in segmentation of anatomical structures

This work proposes a novel variational inference framework to model the distribution of plausible segmentation maps, given a specific MR image, which explicitly represents the multi-reader variability, and applies a variational autoencoder network and optimize its evidence lower bound (ELBO) to efficiently approximate the distribution.

Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertainty

Stochastic segmentation networks (SSNs) are introduced, an efficient probabilistic method for modelling aleatoric uncertainty with any image segmentation network architecture and outperform state-of-the-art for modelling correlated uncertainty in ambiguous images while being much simpler, more flexible, and more efficient.

Uncertainty quantification in medical image segmentation with Normalizing Flows

A novel conditional generative model that is based on conditional Normalizing Flow (cFlow) is proposed, to increase the expressivity of the cVAE by introducing a cFlow transformation step after the encoder, allowing the model to capture richer segmentation variations.

Using Soft Labels to Model Uncertainty in Medical Image Segmentation

This work proposes a simple method to obtain soft labels from the annotations of multiple physicians and train models that, for each image, produce a single well-calibrated output that can be thresholded at multiple confidence levels, according to each application’s precision-recall requirements.

Calibrated Adversarial Refinement for Stochastic Semantic Segmentation

This work proposes a novel two-stage, cascaded approach for calibrated adversarial refinement, and shows that the core design can be adapted to other tasks requiring learning a calibrated predictive distribution by experimenting on a toy regression dataset.

Hypernet-Ensemble Learning of Segmentation Probability for Medical Image Segmentation with Ambiguous Labels

This paper proposes novel methods to improve the segmentation probability estimation without sacrificing performance in a real-world scenario that the authors have only one ambiguous annotation per image and proposes a unified hypernetwork ensemble method to alleviate the computational burden of training multiple networks.

Calibrated Adversarial Refinement for Multimodal Semantic Segmentation

This work proposes a novel two-stage, cascaded strategy for calibrated adversarial refinement, which can be used independently or integrated into any black-box segmentation framework to enable the synthesis of diverse predictions.

Improving Aleatoric Uncertainty Quantification in Multi-Annotated Medical ImageSegmentation with Normalizing Flows

This paper proves the hypothesis that a more flexible density model should be seriously considered in architectures that attempt to capture segmentation ambiguity through density modeling by adopting the Probabilistic U-Net and augmenting the posterior density with an NF, which enables the learnt densities to be more complex and facilitate more accurate modeling for uncertainty.

Modeling Multimodal Aleatoric Uncertainty in Segmentation with Mixture of Stochastic Expert

A novel mixture of stochastic experts (MoSE) model, where each expert network estimates a distinct mode of the aleatoric uncertainty and a gating network predicts the probabilities of an input image being segmented in those modes, yields an efficient two-level uncertainty representation.



Crowdsourcing the creation of image segmentation algorithms for connectomics

To stimulate progress in automating the reconstruction of neural circuits, we organized the first international challenge on 2D segmentation of electron microscopic (EM) images of the brain.

U-Net: Convolutional Networks for Biomedical Image Segmentation

It is shown that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks.

Instance Segmentation of Biological Images Using Harmonic Embeddings

  • V. KulikovV. Lempitsky
  • Computer Science
    2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2020
This work presents a new instance segmentation approach tailored to biological images, where instances may correspond to individual cells, organisms or plant parts, achieving state-of-the-art performance on a popular CVPPP benchmark.

TensorMask: A Foundation for Dense Object Segmentation

It is demonstrated that the tensor view leads to large gains over baselines that ignore this structure, and leads to results comparable to Mask R-CNN, suggesting that TensorMask can serve as a foundation for novel advances in dense mask prediction and a more complete understanding of the task.

Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding

A practical system which is able to predict pixel-wise class labels with a measure of model uncertainty, and shows that modelling uncertainty improves segmentation performance by 2-3% across a number of state of the art architectures such as SegNet, FCN and Dilation Network, with no additional parametrisation.

Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscaling

The Subscale Pixel Network (SPN) is proposed, a conditional decoder architecture that generates an image as a sequence of sub-images of equal size that compactly captures image-wide spatial dependencies and requires a fraction of the memory and the computation required by other fully autoregressive models.

BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling

This paper introduces the Bidirectional-Inference Variational Autoencoder (BIVA), characterized by a skip-connected generative model and an inference network formed by a bidirectional stochastic inference path, and shows that BIVA reaches state-of-the-art test likelihoods, generates sharp and coherent natural images, and uses the hierarchy of latent variables to capture different aspects of the data distribution.

A Variational U-Net for Conditional Appearance and Shape Generation

A conditional U-Net is presented for shape-guided image generation, conditioned on the output of a variational autoencoder for appearance, trained end-to-end on images, without requiring samples of the same object with varying pose or appearance.

Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses

This work proposes a frame-work for reformulating existing single-prediction models as multiple hypothesis prediction (MHP) models and an associated meta loss and optimization procedure to train them, and finds that MHP models outperform their single-hypothesis counterparts in all cases and expose valuable insights into the variability of predictions.

Instance Segmentation by Deep Coloring

This approach proceeds by introducing a fixed number of labels and then dynamically assigning object instances to those labels during training (coloring), and a standard semantic segmentation objective is then used to train a network that can color previously unseen images.