Multi-element microscope optimization by a learned sensing network with composite physical layers

@article{Kim2020MultielementMO,
  title={Multi-element microscope optimization by a learned sensing network with composite physical layers},
  author={Kanghyun Kim and Pavan Chandra Konda and Colin L. Cooke and Ron Appel and Roarke Horstmeyer},
  journal={Optics letters},
  year={2020},
  volume={45 20},
  pages={
          5684-5687
        }
}
Standard microscopes offer a variety of settings to help improve the visibility of different specimens to the end microscope user. Increasingly, however, digital microscopes are used to capture images for automated interpretation by computer algorithms (e.g., for feature classification, detection, or segmentation), often without any human involvement. In this work, we investigate an approach to jointly optimize multiple microscope settings, together with a classification network, for improved… 
[PDF] Semantic Reader
8 Citations
Background Citations
3
Methods Citations
2

Figures and Tables from this paper

8 Citations

Image sensing with multilayer, nonlinear optical neural networks

Tianyu Wang,1, ∗ Mandar M. Sohoni,1, ∗ Logan G. Wright,1, 2 Martin M. Stein,1 Shi-Yuan Ma,1 Tatsuhiro Onodera,1, 2 Maxwell Anderson,1 and Peter L. McMahon1, 3, † School of Applied and Engineering…

Differentiable Microscopy Designs an All Optical Quantitative Phase Microscope

.

Differentiable Microscopy for Content and Task Aware Compressive Fluorescence Imaging

This work proposes differentiable compressive microscopy which includes a realistic generalizable forward model with learnable-physical parameters, and a novel physics-inspired inverse model that outperforms widely used traditional compressive sampling schemes at higher compressions in terms of reconstruction quality.

Intelligent Meta-Imagers: From Compressed to Learned Sensing

This merging of the physical world of metamaterial engineering and the digital world of AI enables the remarkable latency gains of intelligent meta-imagers and provides explicit design tutorials for the integration of programmable meta-atoms as trainable physical weights into an intelligent end-to-end sensing pipeline.

Label-free imaging and classification of live P. falciparum enables high performance parasitemia quantification without fixation or staining

It is demonstrated that deep learning enables both life stage classification and accurate parasitemia quantification of ordinary brightfield microscopy images of live, unstained red blood cells, establishing that label-free Parasitemia analysis of live cells is possible in a biomedical laboratory setting without the need for complex optical instrumentation.

A Baseline for Cross-Database 3D Human Pose Estimation

The experimental results showed the usefulness of the joint harmonization, of the scale normalization, and of augmenting virtual cameras to significantly improve cross-database and in-database generalization and showed that there were dataset biases that could not be compensated and call for new datasets covering more diversity.

Learning Optimal Wavefront Shaping for Multi-Channel Imaging

At extreme densities, it is shown that a split-signal system, with end-to-end learned phase masks, doubles the detection rate and reaches improved precision compared to the current state-of-the-art, single-channel design.

Physics-Enhanced Machine Learning for Virtual Fluorescence Microscopy

A deep neural network is used to effectively design optical patterns for specimen illumination that substantially improve upon the ability to infer fluorescence image information from unstained microscope images, using an illumination model within the DNN's first layers that is jointly optimized during network training.

References

SHOWING 1-10 OF 13 REFERENCES

Learned sensing: jointly optimized microscope hardware for accurate image classification.

This work presents a method to co-optimize how a sample is illuminated in a microscope, along with a pipeline to automatically classify the resulting image, using a deep neural network, to increase the speed and accuracy of automated image classification.

Physics-Enhanced Machine Learning for Virtual Fluorescence Microscopy

A deep neural network is used to effectively design optical patterns for specimen illumination that substantially improve upon the ability to infer fluorescence image information from unstained microscope images, using an illumination model within the DNN's first layers that is jointly optimized during network training.

Physics-Based Learned Design: Optimized Coded-Illumination for Quantitative Phase Imaging

This paper proposes a new data-driven approach for optimizing coded-illumination patterns for an LED array microscope for a given phase reconstruction algorithm that incorporates both the physics of the measurement scheme and the nonlinearity of the reconstruction algorithm into the design problem.

Multicolor localization microscopy and point-spread-function engineering by deep learning.

It is demonstrated how neural networks can exploit the chromatic dependence of the point-spread function to classify the colors of single emitters imaged on a grayscale camera and how deep learning can be used to design new phase-modulating elements that result in further improved color differentiation between species.

Illumination pattern design with deep learning for single-shot Fourier ptychographic microscopy.

This work uses deep learning to achieve single-shot imaging without sacrificing the space-bandwidth product, reducing the acquisition time in Fourier ptychographic microscopy by a factor of 69.

Multicolour localization microscopy by point-spread-function engineering.

By exploiting chromatic dispersion, a new class of optical phase masks are designed that simultaneously yield controllably different PSFs for different wavelengths, enabling simultaneous multicolour tracking or super-resolution imaging in a single optical path.

End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging

A fully-differentiable simulation model is built that maps the true source image to the reconstructed one and jointly optimize the optical parameters and the image processing algorithm parameters so as to minimize the deviation between the true and reconstructed image, over a large set of images.

Wide-field, high-resolution Fourier ptychographic microscopy

An imaging method, termed Fourier ptychographic microscopy (FPM), which iteratively stitches together a number of variably illuminated, low-resolution intensity images in Fourier space to produce a wide-field, high-resolution complex sample image, which can also correct for aberrations and digitally extend a microscope's depth-of-focus beyond the physical limitations of its optics.

DiffuserCam: Lensless Single-exposure 3D Imaging

A compact and easy-to-build computational camera for single-shot 3D imaging that solves the large-scale inverse problem in a computationally efficient way and provides new theory for analyzing resolution in such systems.

High numerical aperture Fourier ptychography: principle, implementation and characterization.

This work indicates that Fourier ptychography is an attractive method to improve the resolution-versus-NA performance, increase the working distance, and enlarge the field-of-view of high-resolution bright-field microscopes by employing lower NA objectives.