Corpus ID: 215547992

Physics-enhanced machine learning for virtual fluorescence microscopy

  title={Physics-enhanced machine learning for virtual fluorescence microscopy},
  author={Colin L. Cooke and Fanjie Kong and Amey Chaware and Kevin C Zhou and Kanghyun Kim and Rong Xu and D. Michael Ando and Samuel J. Yang and Pavan Chandra Konda and Roarke Horstmeyer},
This paper introduces a new method of data-driven microscope design for virtual fluorescence microscopy. Our results show that by including a model of illumination within the first layers of a deep convolutional neural network, it is possible to learn task-specific LED patterns that substantially improve the ability to infer fluorescence image information from unstained transmission microscopy images. We validated our method on two different experimental setups, with different magnifications… Expand
Towards an Intelligent Microscope: Adaptively Learned Illumination for Optimal Sample Classification
A reinforcement learning system that adaptively explores optimal patterns to illuminate specimens for immediate classification by synthesizing knowledge over multiple snapshots is presented, demonstrating a smarter way to physically capture task-specific information. Expand
Virtual UV Fluorescence Microscopy from Hematoxylin and Eosin Staining of Liver Images Using Deep Learning Convolutional Neural Network
The results show that it is possible to perform this type of conversion (the studies used liver slides for 100× magnification), and in some cases there was an additional improvement in image quality. Expand
Learning Optimal Wavefront Shaping for Multi-Channel Imaging
At extreme densities, it is shown that a split-signal system, with end-to-end learned phase masks, doubles the detection rate and reaches improved precision compared to the current state-of-the-art, single-channel design. Expand
Multi-element microscope optimization by a learned sensing network with composite physical layers
The interplay between optimization of programmable illumination and pupil transmission is explored, using experimentally imaged blood smears for automated malaria parasite detection, to show that multi-element "learned sensing" outperforms its single-element counterpart. Expand


Convolutional neural networks that teach microscopes how to image
This work increases the classification accuracy of a microscope's recorded images by merging an optical model of image formation into the pipeline of a CNN, and simultaneously determines an ideal illumination arrangement to highlight important sample features during image acquisition. Expand
Multicolor localization microscopy and point-spread-function engineering by deep learning.
It is demonstrated how neural networks can exploit the chromatic dependence of the point-spread function to classify the colors of single emitters imaged on a grayscale camera and how deep learning can be used to design new phase-modulating elements that result in further improved color differentiation between species. Expand
Physics-Based Learned Design: Optimized Coded-Illumination for Quantitative Phase Imaging
This paper proposes a new data-driven approach for optimizing coded-illumination patterns for an LED array microscope for a given phase reconstruction algorithm that incorporates both the physics of the measurement scheme and the nonlinearity of the reconstruction algorithm into the design problem. Expand
Content-aware image restoration: pushing the limits of fluorescence microscopy
This work shows how content-aware image restoration based on deep learning extends the range of biological phenomena observable by microscopy by bypassing the trade-offs between imaging speed, resolution, and maximal light exposure that limit fluorescence imaging to enable discovery. Expand
Learned sensing: jointly optimized microscope hardware for accurate image classification.
This work presents a method to co-optimize how a sample is illuminated in a microscope, along with a pipeline to automatically classify the resulting image, using a deep neural network, to increase the speed and accuracy of automated image classification. Expand
Isotropic reconstruction of 3D fluorescence microscopy images using convolutional neural networks
A convolutional neural network is proposed that is trained end-to-end from the same anisotropic body of data the authors later apply the network to, and effectively learns to restore the full isotropic resolution by restoring the image under a trained, sample specific image prior. Expand
Illumination Pattern Design With Deep Learning For Single-Shot Fourier Ptychographic Microscopy
Fourier ptychographic microscopy allows for the collection of images with a high space-bandwidth product at the cost of temporal resolution. In Fourier ptychographic microscopy, the light source of aExpand
Illumination pattern design with deep learning for single-shot Fourier ptychographic microscopy.
This work uses deep learning to achieve single-shot imaging without sacrificing the space-bandwidth product, reducing the acquisition time in Fourier ptychographic microscopy by a factor of 69. Expand
Applications , Promises , and Pitfalls of Deep Learning for Fluorescence Image Reconstruction
Deep Learning is a recent and important addition to the computational toolbox available for image reconstruction in fluorescence microscopy. We review state-of-the-art applications such as imageExpand
Using machine-learning to optimize phase contrast in a low-cost cellphone microscope
It is shown that the effect of a varied light source shape, using the pre-trained CNN, does not only improve the phase contrast, but also the impression of an improvement in optical resolution without adding any special optics, as demonstrated by measurements. Expand