Multi-modal segmentation with missing MR sequences using pre-trained fusion networks

  title={Multi-modal segmentation with missing MR sequences using pre-trained fusion networks},
  author={Karin A. van Garderen and Marion Smits and Stefan Klein},
Missing data is a common problem in machine learning and in retrospective imaging research it is often encountered in the form of missing imaging modalities. We propose to take into account missing modalities in the design and training of neural networks, to ensure that they are capable of providing the best possible prediction even when multiple images are not available. The proposed network combines three modifications to the standard 3D UNet architecture: a training scheme with dropout of… Expand
2 Citations
Brain Tumor Segmentation for Missing Modalities by Supplementing Missing Features
Brain tumor segmentation in multi-modal magnetic resonance images is an essential step in brain cancer diagnosis and treatment. Despite the recent success of multi-Modalities fusion network for brainExpand
Unified generative adversarial networks for multimodal segmentation from unpaired 3D medical images
A 3D unified generative adversarial network is proposed, which unifies the any-to-any modality translation and multimodal segmentation in a single network and outperforms the existing unified methods on abdominal organ segmentation and brain tumor segmentation. Expand


HeMIS: Hetero-Modal Image Segmentation
A deep learning image segmentation framework that is extremely robust to missing imaging modalities, which learns, for each modality, an embedding of the input image into a single latent vector space for which arithmetic operations are well defined. Expand
Representation Learning for Cross-Modality Classification
It is found that adding the similarity term to the standard objective can produce representations that are more similar and can give a higher accuracy in these cross-modality classification experiments. Expand
3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
The proposed network extends the previous u-net architecture from Ronneberger et al. by replacing all 2D operations with their 3D counterparts and performs on-the-fly elastic deformations for efficient data augmentation during training. Expand
The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)
The set-up and results of the Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) organized in conjunction with the MICCAI 2012 and 2013 conferences are reported, finding that different algorithms worked best for different sub-regions, but that no single algorithm ranked in the top for all sub-Regions simultaneously. Expand
Cross-Domain Synthesis of Medical Images Using Efficient Location-Sensitive Deep Network
A novel architecture called location-sensitive deep network LSDN is proposed for synthesizing images across domains that integrates intensity feature from image voxels and spatial information in a principled manner and is computationally efficient, e.g. 26× faster than other sparse representation based methods. Expand
No New-Net
The effectiveness of a well trained U-Net in the context of the BraTS 2018 challenge is demonstrated given that researchers are currently besting each other with architectural modifications that are intended to improve the segmentation performance. Expand
Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features
This set of labels and features should enable direct utilization of the TCGA/TCIA glioma collections towards repeatable, reproducible and comparative quantitative studies leading to new predictive, prognostic, and diagnostic assessments, as well as performance evaluation of computer-aided segmentation methods. Expand
Missing data imputation using statistical and machine learning methods in a real breast cancer problem
The method based on machine learning techniques were the most suited for the imputation of missing values and led to a significant enhancement of prognosis accuracy compared to imputation methods based on statistical procedures. Expand
Adam: A Method for Stochastic Optimization
This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. Expand
Visualizing Data using t-SNE
We present a new technique called “t-SNE” that visualizes high-dimensional data by giving each datapoint a location in a two or three-dimensional map. The technique is a variation of StochasticExpand