# Deep Learning Microscopy

@article{Rivenson2017DeepLM,
title={Deep Learning Microscopy},
author={Yair Rivenson and Zoltan Gorocs and Harun Gunaydin and Yibo Zhang and Hongda Wang and Aydogan Ozcan},
journal={ArXiv},
year={2017},
volume={abs/1705.04709}
}
We demonstrate that a deep neural network can significantly improve optical microscopy, enhancing its spatial resolution over a large field-of-view and depth-of-field. After its training, the only input to this network is an image acquired using a regular optical microscope, without any changes to its design. We blindly tested this deep learning approach using various tissue samples that are imaged with low-resolution and wide-field systems, where the network rapidly outputs an image with… Expand
383 Citations

#### Figures and Topics from this paper

Deep learning achieves super-resolution in fluorescence microscopy
This work presents a deep learning-based method for achieving super-resolution in fluorescence microscopy, which is solely based on training a generative adversarial network, which statistically learns to transform low resolution input images into super-resolved ones. Expand
Fluorescence microscopy datasets for training deep neural networks
• Medicine
• GigaScience
• 2021
High-quality datasets are provided that can be used to train and evaluate deep learning methods under development and are vital for training convolutional neural networks that are used in current machine learning approaches. Expand
Fluorescence Microscopy Datasets for Training Deep Neural Networks
• Biology, Computer Science
• 2020
High quality data sets are provided which can be used to train and evaluate deep learning methods under development and are vital for training convolutional neural networks which are used in current machine learning approaches. Expand
Fast structured illumination microscopy via deep learning
• Computer Science
• 2020
This study shows that convolutional neural networks (CNNs) can be used to improve the performance of structured illumination microscopy to enable it to reconstruct a super-resolution image usingExpand
Three-dimensional localization microscopy using deep learning.
• Medicine
• Optics express
• 2018
This article reports on the use of a deep convolutional neural network for localizing particles in three dimensions on the basis of single images and shows that the precision obtained with a CNN can be comparable to that of maximum likelihood estimation (MLE), which is the accepted gold standard. Expand
Multicolor localization microscopy by deep learning
• Physics
• 2018
Deep learning has become an extremely effective tool for image classification and image restoration problems. Here, we apply deep learning to microscopy, and demonstrate how neural networks canExpand
High-resolution photoacoustic microscopy with deep penetration through learning
• Shengfu Cheng, Yingying Zhou, Jiangbo Chen, Huanhao Li
• Medicine
• Photoacoustics
• 2022
The proposed method provides new horizons to expand the scope of OR-PAM towards deep-tissue imaging and wide applications in biomedicine, and it was shown experimentally that the deep-learning method can retain high resolution at tissue depths beyond one optical transport mean free path. Expand
Deep learning-enhanced fluorescence microscopy via degeneration decoupling.
• Computer Science, Medicine
• Optics express
• 2020
A novel deconvolution algorithm based on an imaging model, deep-learning priors and the alternating direction method of multipliers is proposed, which outperforms existing state-of-the-art deconVolution algorithms in resolution enhancement and generalization. Expand
Super-Resolution Reconstruction of Transmission Electron Microscopy Images Using Deep Learning
• Computer Science
• 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019)
• 2019
This work presents a convolutional neural network model, utilizing both local and global skip connections, aiming for $4 \times$ SR reconstruction of TEM images, and investigates the domain adaptation capacity of the CNN-based model by testing it on Tem images of a cilia sample, having different image characteristics as compared to the calibration-grid. Expand
Three-dimensional propagation and time-reversal of fluorescence images
• Materials Science, Computer Science
• ArXiv
• 2019
It is demonstrated that a neural network can inherently learn the physical laws governing fluorescence wave propagation and time-reversal to enable 3D imaging of fluorescent samples using a single 2D image, without mechanical scanning, additional hardware, or a trade-off of resolution or speed. Expand

#### References

SHOWING 1-10 OF 27 REFERENCES
Image Super-Resolution Using Deep Convolutional Networks
• Computer Science, Medicine
• IEEE Transactions on Pattern Analysis and Machine Intelligence
• 2016
We propose a deep learning method for single image super-resolution (SR). Our method directly learns an end-to-end mapping between the low/high-resolution images. The mapping is represented as a deepExpand
Complex wavelets for extended depth‐of‐field: A new method for the fusion of multichannel microscopy images
• Computer Science, Medicine
• Microscopy research and technique
• 2004
This work shows how the wavelet‐based image fusion technique can be improved and easily extended to multichannel data and proposes the use of complex‐valued wavelet bases, which seem to outperform traditional real‐valuedWavelet transforms. Expand
Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network
• W. Shi, +5 authors Zehan Wang
• Computer Science, Mathematics
• 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
• 2016
This paper presents the first convolutional neural network capable of real-time SR of 1080p videos on a single K2 GPU and introduces an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output. Expand
Phase recovery and holographic image reconstruction using deep learning in neural networks
• Computer Science, Medicine
• Light, science & applications
• 2018
It is demonstrated that a neural network can learn to perform phase recovery and holographic image reconstruction after appropriate training, and this deep learning-based approach provides an entirely new framework to conduct holographic imaging by rapidly eliminating twin-image and self-interference-related spatial artifacts. Expand
Deep bilateral learning for real-time image enhancement
• Computer Science
• ACM Trans. Graph.
• 2017
This work introduces a new neural network architecture inspired by bilateral grid processing and local affine color transforms that processes high-resolution images on a smartphone in milliseconds, provides a real-time viewfinder at 1080p resolution, and matches the quality of state-of theart approximation techniques on a large class of image operators. Expand
Super-resolution image reconstruction: a technical overview
• Computer Science
• IEEE Signal Process. Mag.
• 2003
The goal of this article is to introduce the concept of SR algorithms to readers who are unfamiliar with this area and to provide a review for experts to present the technical review of various existing SR methodologies which are often employed. Expand
Deep Convolutional Neural Network for Inverse Problems in Imaging
• Computer Science, Mathematics
• IEEE Transactions on Image Processing
• 2017
The proposed network outperforms total variation-regularized iterative reconstruction for the more realistic phantoms and requires less than a second to reconstruct a <inline-formula> <tex-math notation="LaTeX">$512\times 512$ </tex- math></inline- formula> image on the GPU. Expand
Deep Pyramidal Residual Networks
• Computer Science
• 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
• 2017
This research gradually increases the feature map dimension at all units to involve as many locations as possible in the network architecture and proposes a novel residual unit capable of further improving the classification accuracy with the new network architecture. Expand
Accelerating magnetic resonance imaging via deep learning
• Shanshan Wang, +5 authors D. Liang
• Computer Science, Medicine
• 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI)
• 2016
This paper proposes a deep learning approach for accelerating magnetic resonance imaging (MRI) using a large number of existing high quality MR images as the training datasets. An off-lineExpand
Deep Residual Learning for Image Recognition
• Kaiming He, Jian Sun
• Computer Science
• 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
• 2016
This work presents a residual learning framework to ease the training of networks that are substantially deeper than those used previously, and provides comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. Expand