Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning

@article{Wu2019ThreedimensionalVR,
  title={Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning},
  author={Yichen Wu and Yair Rivenson and Hongda Wang and Yilin Luo and Eyal Ben-David and Laurent A Bentolila and Christian Pritz and Aydogan Ozcan},
  journal={Nature Methods},
  year={2019},
  pages={1-9}
}
We demonstrate that a deep neural network can be trained to virtually refocus a two-dimensional fluorescence image onto user-defined three-dimensional (3D) surfaces within the sample. Using this method, termed Deep-Z, we imaged the neuronal activity of a Caenorhabditis elegans worm in 3D using a time sequence of fluorescence images acquired at a single focal plane, digitally increasing the depth-of-field by 20-fold without any axial scanning, additional hardware or a trade-off of imaging… Expand
Exceeding the limits of 3D fluorescence microscopy using a dual-stage-processing network
Although three-dimensional (3D) fluorescence microscopy is an essential tool for life science research, the fundamentally limited optical throughput, as reflected in the compromise between speed andExpand
Full-color optically-sectioned imaging by wide-field microscopy via deep-learning.
TLDR
A deep-learning-based method is proposed, i.e., FC-WFM-Deep, which can reconstruct high-quality full-color 3D images with an extended optical sectioning capability directly from the FC- WFM z-stack data, which significantly reduces the 3D data acquisition requirements without losing detail and improves the3D imaging speed by extracting the Optical sectioning in the depth-of-field. Expand
Breaking the limit of 3D fluorescence microscopy by dual-stage-processing network
TLDR
A dual-stage mutual-feedback deep-learning approach that allows gradual reversion of microscopy degradation from high-resolution targets to low-resolution images is reported, using a single blurred-and-pixelated 3D image as input and infers a 3D output with notably higher resolution and improved contrast. Expand
A survey on applications of deep learning in microscopy image analysis
TLDR
The applications of deep learning algorithms in microscopy image analysis, which include image classification, region segmentation, object tracking and super-resolution reconstruction, are introduced. Expand
Deep learning enables confocal laser-scanning microscopy with enhanced resolution.
  • Weibo Wang, Biwei Wu, Baoyuan Zhang, Jie Ma, Jiubin Tan
  • Medicine
  • Optics letters
  • 2021
TLDR
A parameter-free post-processing strategy for laser-scanning microscopy based on deep learning enables a spatial resolution enhancement by a factor of ∼1.3, compared to conventional CLSM, and allowed us to correctly infer the fine structures of real experimental images. Expand
Deep Learning to Refocus 3D Images
High-throughput volumetric fluorescence microscopic imaging remains an important challenge in biology, life sciences and engineering. Imaging 3D fluorescent samples usually involves scanning toExpand
Deep-3D Microscope: 3D volumetric microscopy of thick scattering samples using a wide-field microscope and machine learning
Confocal microscopy is the standard approach for obtaining volumetric images of a sample with high axial and lateral resolution, especially when dealing with scattering samples. Unfortunately, aExpand
Deep learning based one-shot optically-sectioned structured illumination microscopy for surface measurement.
TLDR
A deep-learning-based one-shot optically-sectioned method, called Deep-OS-SIM, is proposed to improve the efficiency of OS-SIM for surface measurement and develops a convolutional neural network to learn the statistical invariance of optical sectioning across structured illumination images. Expand
3D Structured Illumination Microscopy via Channel Attention Generative Adversarial Network
Three-dimensional (3D) structured illumination microscopy (SIM) plays an important role in biological volumetric imaging with the capabilities of doubling the lateral and axial resolution and opticalExpand
Advancing biological super-resolution microscopy through deep learning: a brief review
TLDR
This brief Review surveys recent advances in using deep learning to enhance performance of superresolution microscopy and focuses primarily on how deep learning advances reconstruction of super-resolution images. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 88 REFERENCES
Deep Learning Microscopy
TLDR
It is demonstrated that a deep neural network can significantly improve optical microscopy, enhancing its spatial resolution over a large field-of-view and depth of field, and can be used to design computational imagers that get better and better as they continue to image specimen and establish new transformations among different modes of imaging. Expand
Deep learning enables cross-modality super-resolution in fluorescence microscopy
TLDR
This data-driven approach does not require numerical modeling of the imaging process or the estimation of a point-spread-function, and is based on training a generative adversarial network to transform diffraction-limited input images into super-resolved ones, and could serve to democratize super-resolution imaging. Expand
Isotropic reconstruction of 3D fluorescence microscopy images using convolutional neural networks
TLDR
A convolutional neural network is proposed that is trained end-to-end from the same anisotropic body of data the authors later apply the network to, and effectively learns to restore the full isotropic resolution by restoring the image under a trained, sample specific image prior. Expand
Three-Dimensional Super-Resolution Imaging by Stochastic Optical Reconstruction Microscopy
TLDR
3D stochastic optical reconstruction microscopy (STORM) is demonstrated by using optical astigmatism to determine both axial and lateral positions of individual fluorophores with nanometer accuracy, allowing the 3D morphology of nanoscopic cellular structures to be resolved. Expand
Content-aware image restoration: pushing the limits of fluorescence microscopy
TLDR
This work shows how content-aware image restoration based on deep learning extends the range of biological phenomena observable by microscopy by bypassing the trade-offs between imaging speed, resolution, and maximal light exposure that limit fluorescence imaging to enable discovery. Expand
Deep learning massively accelerates super-resolution localization microscopy
TLDR
Simulations and experimental imaging of microtubules, nuclear pores, and mitochondria show that high-quality, super-resolution images can be reconstructed from up to two orders of magnitude fewer frames than usually needed, without compromising spatial resolution. Expand
Deep-STORM: super-resolution single-molecule microscopy by deep learning
TLDR
Deep-STORM uses a deep convolutional neural network that can be trained on simulated data or experimental measurements, both of which are demonstrated, and achieves state-of-the-art resolution under challenging signal-to-noise conditions and high emitter densities. Expand
Interferometric synthetic aperture microscopy
State-of-the-art methods in high-resolution three-dimensional optical microscopy require that the focus be scanned through the entire region of interest. However, an analysis of the physics of theExpand
Compressive light-field microscopy for 3D neural activity recording
Understanding the mechanisms of perception, cognition, and behavior requires instruments that are capable of recording and controlling the electrical activity of many neurons simultaneously and atExpand
Enhancing the performance of the light field microscope using wavefront coding.
TLDR
An extended optical model for the wavefront coded light field microscope is presented and a performance metric based on Fisher information is developed, which is used to choose adequate phase masks parameters. Expand
...
1
2
3
4
5
...