DeepDRR - A Catalyst for Machine Learning in Fluoroscopy-guided Procedures

@article{Unberath2018DeepDRRA,
  title={DeepDRR - A Catalyst for Machine Learning in Fluoroscopy-guided Procedures},
  author={M. Unberath and Jan-Nico Zaech and Sing Chun Lee and Bastian Bier and Javad Fotouhi and Mehran Armand and Nassir Navab},
  journal={ArXiv},
  year={2018},
  volume={abs/1803.08606}
}
Machine learning-based approaches outperform competing methods in most disciplines relevant to diagnostic radiology. Interventional radiology, however, has not yet benefited substantially from the advent of deep learning, in particular because of two reasons: 1) Most images acquired during the procedure are never archived and are thus not available for learning, and 2) even if they were available, annotations would be a severe challenge due to the vast amounts of data. When considering… 
Enabling machine learning in X-ray-based procedures via realistic simulation of image formation
TLDR
It is demonstrated that convolutional neural networks (ConvNets) trained on DeepDRRs generalize well to real data without re-training or domain adaptation, which has the potential to revolutionize intra-operative image analysis to simplify surgical workflows.
A Deep Learning Approach for Single Shot C-Arm Pose Estimation
TLDR
A C-arm position prediction system based on machine learning that can potentially reduce the number of intraoperatively acquired X-rays in a common orthopaedic surgical procedure is proposed.
Learning to detect anatomical landmarks of the pelvis in X-rays from arbitrary views
TLDR
This work presents the first tool to detect anatomical landmarks in X-ray images independent of their viewing direction, and constitutes a first step toward global initialization of 2D/3D registration without the need of calibration.
Digitally Reconstructed Radiograph Generation for Enabling AI/ML in Medical Imaging
TLDR
The proposed solution in this work has taken 1.2 seconds on average to generate DRR of image size 1000*1000 pixel, which implies the speed of writing as 1.5 µs per pixel.
The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective
TLDR
The impact of machine learning on 2D/3D registration is reviewed to systematically summarize the recent advances made by introduction of this novel technology and offers a perspective on the most pressing needs, significant open problems, and possible next steps.
Toward automatic C-arm positioning for standard projections in orthopedic surgery
TLDR
This work proposes a convolutional neural network regression model that predicts 5 degrees of freedom pose updates directly from a first X-ray image and demonstrates that learning based on simulations translates to acceptable performance on real X-rays.
Federated Simulation for Medical Imaging
TLDR
This work introduces a physics-driven generative approach that consists of two learnable neural modules: a module that synthesizes 3D cardiac shapes along with their materials, and a CT simulator that renders these into realistic 3D CT Volumes, with annotations.
Training Deep Learning Models for 2D Spine X-rays Using Synthetic Images and Annotations Created from 3D CT Volumes
In this paper, we present an approach for generating and using synthetic 2D X-ray images with its corresponding annotations, both created from available 3D CT volumes to aid in the task of detecting
A Gentle Introduction to Deep Learning in Medical Image Processing
TLDR
A gentle introduction to deep learning in medical image processing is given, proceeding from theoretical foundations to applications, including general reasons for the popularity of deep learning, including several major breakthroughs in computer science.
Deep Morphing: Detecting bone structures in fluoroscopic X-ray images with prior knowledge
TLDR
This work proposes computationally efficient two-stage approaches, which it calls deep morphing, for both representations by fitting the representation to the output of a deep segmentation network.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 15 REFERENCES
Fully Automatic and Real-Time Catheter Segmentation in X-Ray Fluoroscopy
TLDR
A real-time method to segment the catheter and guidewire in 2D X-ray fluoroscopic sequences based on deep convolutional neural networks is proposed, which is performed in a real- time fully-automatic manner.
Fast Prospective Detection of Contrast Inflow in X-ray Angiograms with Convolutional Neural Network and Recurrent Neural Network
TLDR
Two different approaches for prospective contrast inflow detection are proposed and achieve good performance on detection of the beginning contrast frame from X-ray sequences and are more robust than a state-of-the-art method.
Novel real-time tumor-contouring method using deep learning to prevent mistracking in X-ray fluoroscopy
TLDR
The results from a simulated fluoroscopy model showed accurate tracking of a low-visibility tumor with an error of approximately 1 mm, even if enhanced bone structure acted as an obstacle, and support the feasibility of robust real-time tumor contouring with fluoroscope.
X-ray-transform Invariant Anatomical Landmark Detection for Pelvic Trauma Surgery
TLDR
This work presents a method to automatically detect anatomical landmarks in X-ray images independent of the viewing direction and shows that its intra-operative landmark detection together with pre-operative CT enables X-rays pose estimation which, ultimately, benefits initialization of image-based 2D/3D registration.
V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation
TLDR
This work proposes an approach to 3D image segmentation based on a volumetric, fully convolutional, neural network, trained end-to-end on MRI volumes depicting prostate, and learns to predict segmentation for the whole volume at once.
Deep scatter estimation (DSE): feasibility of using a deep convolutional neural network for real-time x-ray scatter prediction in cone-beam CT
TLDR
The deep scatter estimation (DSE) uses a deep convolutional neural network which is trained to reproduce the output of Monte Carlo simulations using only the acquired projection data as input and performs in real-time.
Large scale deep learning for computer aided detection of mammographic lesions
TLDR
A head‐to‐head comparison between a state‐of‐the art in mammography CAD system, relying on a manually designed feature set and a Convolutional Neural Network (CNN), aiming for a system that can ultimately read mammograms independently.
Error Corrective Boosting for Learning Fully Convolutional Networks with Limited Data
TLDR
SkipDeconv-Net (SD-Net), a new F-CNN architecture for brain segmentation that combines skip connections with the unpooling strategy for upsampling, is introduced and outperforms the latest state-of-the-art F- CNN models.
Automatic lumbar vertebrae detection based on feature fusion deep learning for partial occluded C-arm X-ray images.
  • Yang Li, Wei Liang, Yinlong Zhang, Haibo An, Jindong Tan
  • Medicine
    Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference
  • 2016
TLDR
A novel convolutional neural network (CNN) model to automatically detect lumbar vertebrae for C-arm X-ray images is presented and performs more accurate in abnormal cases with pathologies and surgical implants in multi-angle views.
Fast generation of digitally reconstructed radiographs using attenuation fields with application to 2D-3D image registration
TLDR
The technique of light field rendering from the computer graphics community is extended, allowing most of the DRR computation to be performed in a preprocessing step; after this precomputation step, DRRs can be generated substantially faster than with conventional ray casting.
...
1
2
...