Synth-by-Reg (SbR): Contrastive learning for synthesis-based registration of paired images

  title={Synth-by-Reg (SbR): Contrastive learning for synthesis-based registration of paired images},
  author={Adri{\`a} Casamitjana and Matteo Mancini and Juan Eugenio Iglesias},
  journal={Simulation and synthesis in medical imaging : ... International Workshop, SASHIMI ..., held in conjunction with MICCAI ..., proceedings. SASHIMI},
  • A. Casamitjana, M. Mancini, J. E. Iglesias
  • Published 30 July 2021
  • Medicine, Computer Science, Engineering
  • Simulation and synthesis in medical imaging : ... International Workshop, SASHIMI ..., held in conjunction with MICCAI ..., proceedings. SASHIMI
Nonlinear inter-modality registration is often challenging due to the lack of objective functions that are good proxies for alignment. Here we propose a synthesis-by-registration method to convert this problem into an easier intra-modality task. We introduce a registration loss for weakly supervised image translation between domains that does not require perfectly aligned training data. This loss capitalises on a registration U-Net with frozen weights, to drive a synthesis CNN towards the… 

Figures from this paper

Robust joint registration of multiple stains and MRI for multimodal 3D histology reconstruction: Application to the Allen human brain atlas
A probabilistic model of spatial deformation that yields reconstructions for multiple histological stains that are jointly smooth, robust to outliers, and follow the reference shape is presented.


DiCyc: GAN-based deformation invariant cross-domain information fusion for medical image synthesis
A deformation invariant cycle-consistency model is presented that can achieve better alignment between the source and target data while maintaining superior image quality of signal compared to several state-of-the-art CycleGAN-based methods.
Generative Adversarial Networks for MR-CT Deformable Image Registration
State-of-the-art DIR methods based on Normalized Mutual Information, Modality Independent Neighborhood Descriptor and their novel combination achieved a mean segmentation overlap ratio of 76.7%, which dropped to 69.1% or less when registering images synthesized by cycle-GAN based on local correlation, due to the poor performance on the thoracic region, where large lung volume changes were synthesized.
Contrastive Learning for Unpaired Image-to-Image Translation
The framework enables one-sided translation in the unpaired image-to-image translation setting, while improving quality and reducing training time, and can be extended to the training setting where each "domain" is only a single image.
Adversarial synthesis learning enables segmentation without target modality ground truth
A novel end-to-end synthesis and segmentation network (EssNet) is proposed to achieve the unpaired MRI to CT image synthesis and CT splenomegaly segmentation simultaneously without using manual labels on CT.
Nonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks
The proposed RegNet is trained using a large set of artificially generated DVFs, does not explicitly define a dissimilarity metric, and integrates image content at multiple scales to equip the network with contextual information, thereby greatly simplifying the training problem.
Deformable Image Registration Based on Similarity-Steered CNN Regression
A convolutional neural network (CNN) based regression model to directly learn the complex mapping from the input image pair to their corresponding deformation field, and it is found that the trained CNN model from one dataset can be successfully transferred to another dataset, although brain appearances across datasets are quite variable.
Adversarial learning for mono- or multi-modal registration
Experiments indicate that the proposed unsupervised adversarial similarity network for image registration yields promising registration performance in accuracy, efficiency and generalizability compared with state-of-the-art registration methods, including those based on deep learning.
Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation
This work bypasses the difficulties of developing cross-modality similarity measures, by training an image-to-image translation network on the two input modalities, and shows that by encouraging the authors' translation network to be geometry preserving, it manages to train an accurate spatial transformation network.
VoxelMorph: A Learning Framework for Deformable Medical Image Registration
VoxelMorph promises to speed up medical image analysis and processing pipelines while facilitating novel directions in learning-based registration and its applications and demonstrates that the unsupervised model’s accuracy is comparable to the state-of-the-art methods while operating orders of magnitude faster.
End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network
The results demonstrate that registration with DIRNet is as accurate as a conventional deformable image registration method with short execution times.