Deep learning enables cross-modality super-resolution in fluorescence microscopy

@article{Wang2018DeepLE,
  title={Deep learning enables cross-modality super-resolution in fluorescence microscopy},
  author={Hongda Wang and Yair Rivenson and Yiyin Jin and Zhensong Wei and Ronald Gao and Harun G{\"u}naydın and Laurent A Bentolila and Comert Kural and Aydogan Ozcan},
  journal={Nature Methods},
  year={2018},
  volume={16},
  pages={103-110}
}
We present deep-learning-enabled super-resolution across different fluorescence microscopy modalities. This data-driven approach does not require numerical modeling of the imaging process or the estimation of a point-spread-function, and is based on training a generative adversarial network (GAN) to transform diffraction-limited input images into super-resolved ones. Using this framework, we improve the resolution of wide-field images acquired with low-numerical-aperture objectives, matching… Expand
Fast confocal microscopy imaging based on deep learning
TLDR
The proposed Back-Projection Generative Adversarial Network (BPGAN) consists of a feature extraction step followed by a back-projection feedback module (BPFM) and an associated reconstruction network, these together allow for super-resolution of low-resolution confocal scans. Expand
Evaluation and development of deep neural networks for image super-resolution in optical microscopy.
TLDR
The deep Fourier channel attention network (DFCAN) is devised, which leverages the frequency content difference across distinct features to learn precise hierarchical representations of high-frequency information about diverse biological structures and achieves comparable image quality to SIM over a tenfold longer duration in multicolor live-cell imaging experiments. Expand
High-Throughput Deep Learning Microscopy Using Multi-Angle Super-Resolution
TLDR
This work designs a multiple-branch deep residual network which extracts high-frequency information and color information in obliquely-illuminated low-resolution input images and generates high-resolution output and carries out detailed experiments to demonstrate the effectiveness and compare it with a computational imaging technique termed Fourier ptychographic microscopy (FPM). Expand
Deep learning enables structured illumination microscopy with low light levels and enhanced speed
TLDR
Using deep learning to augment SIM, a five-fold reduction in the number of raw images required for super-resolution SIM is obtained, and images under extreme low light conditions are generated. Expand
Deep learning-enhanced fluorescence microscopy via degeneration decoupling.
TLDR
A novel deconvolution algorithm based on an imaging model, deep-learning priors and the alternating direction method of multipliers is proposed, which outperforms existing state-of-the-art deconVolution algorithms in resolution enhancement and generalization. Expand
Deep learning-enabled efficient image restoration for 3D microscopy of turbid biological specimens.
TLDR
A deep-learning approach, termed ScatNet, is reported that enables reversion of 3D fluorescence microscopy from high-resolution targets to low-quality, light-scattered measurements, thereby allowing restoration for a blurred and light- scattered 3D image of deep tissue. Expand
Video-rate acquisition fluorescence microscopy via generative adversarial networks
TLDR
It is shown here, for the first time, that video-rate image acquisition, up to 20x the speed of equivalent standard high resolution acquisition, can be obtained without significant reduction in image quality. Expand
Deep learning-based super-resolution in coherent imaging systems
TLDR
This deep learning-based super-resolution framework can be broadly applied to enhance the space-bandwidth product of coherent imaging systems using image data and convolutional neural networks, and provides a rapid, non-iterative method for solving inverse image reconstruction or enhancement problems in optics. Expand
High-resolution photoacoustic microscopy with deep penetration through learning
  • Shengfu Cheng, Yingying Zhou, Jiangbo Chen, Huanhao Li, Lidai Wang, Puxiang Lai
  • Medicine
  • Photoacoustics
  • 2022
TLDR
The proposed method provides new horizons to expand the scope of OR-PAM towards deep-tissue imaging and wide applications in biomedicine, and it was shown experimentally that the deep-learning method can retain high resolution at tissue depths beyond one optical transport mean free path. Expand
3D high resolution generative deep-learning network for fluorescence microscopy imaging.
TLDR
A new, to the best of the knowledge, 3D deep learning network based on a dual generative adversarial network (dual-GAN) framework for recovering high-resolution volume images from high speed acquired low-resolution (LR) volume images is developed. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 49 REFERENCES
Deep learning enhanced mobile-phone microscopy
TLDR
The use of deep learning is reported on to correct distortions introduced by mobile-phone-based microscopes, facilitating the production of high-resolution, denoised and colour-corrected images, matching the performance of benchtop microscopes with high-end objective lenses, also extending their limited depth-of-field. Expand
Extended depth-of-field in holographic image reconstruction using deep learning based auto-focusing and phase-recovery
TLDR
A convolutional neural network based approach that simultaneously performs auto-focusing and phase-recovery to significantly extend the depth-of-field (DOF) in holographic image reconstruction and can be broadly applicable to computationally extend the DOF of other imaging modalities. Expand
Time-lapse two-color 3D imaging of live cells with doubled resolution using structured illumination
TLDR
A previously undescribed SIM setup that is fast enough to record 3D two-color datasets of living whole cells and shows volume rates as high as 4 s in one color and 8.5 s in two colors over tens of time points is reported. Expand
PALM and STORM: Into large fields and high-throughput microscopy with sCMOS detectors.
TLDR
The decisions researchers face when considering how to adapt hardware on a new system for sCMOS sensors with high-throughput in mind are outlined. Expand
DeconvolutionLab2: An open-source software for deconvolution microscopy.
TLDR
This paper examines several standard algorithms used in deconvolution microscopy, notably: Regularized inverse filter, Tikhonov regularization, Landweber, T Sheikhonov-Miller, Richardson-Lucy, and fast iterative shrinkage-thresholding and distinguishes the algorithms in terms of image quality, performance, usability and computational requirements. Expand
Direct stochastic optical reconstruction microscopy with standard fluorescent probes
TLDR
A step-by-step protocol for dSTORM imaging in fixed and living cells on a wide-field fluorescence microscope, with standard fluorescent probes focusing especially on the photoinduced fine adjustment of the ratio of fluorophores residing in the ON and OFF states is presented. Expand
Model-Based 2.5-D Deconvolution for Extended Depth of Field in Brightfield Microscopy
TLDR
This paper proposes a method that jointly estimates the texture and topography of a specimen from a series of brightfield optical sections based on an image formation model that is described by the convolution of a thick specimen model with the microscope's point spread function. Expand
Super-resolution imaging in live cells
  • S. Cox
  • Biology, Medicine
  • Developmental biology
  • 2015
TLDR
The various approaches to super-resolution microscopy, the various approaches which can be successfully used in live cells, the tradeoffs in resolution, speed, and ease of implementation which one must make for each approach, and the quality of results that one might expect from each technique are discussed. Expand
U-Net: Convolutional Networks for Biomedical Image Segmentation
TLDR
It is shown that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks. Expand
Evaluation of fluorophores for optimal performance in localization-based super-resolution imaging
TLDR
This analysis quantitatively characterized the switching properties of 26 organic dyes and directly related these properties to the quality of super-resolution images, providing guidelines for characterization ofsuper-resolution probes and a resource for selecting probes based on performance. Expand
...
1
2
3
4
5
...