Faces as Lighting Probes via Unsupervised Deep Highlight Extraction
@inproceedings{Yi2018FacesAL, title={Faces as Lighting Probes via Unsupervised Deep Highlight Extraction}, author={Renjiao Yi and Chenyang Zhu and Ping Tan and Stephen Lin}, booktitle={ECCV}, year={2018} }
We present a method for estimating detailed scene illumination using human faces in a single image. [] Key Method Based on the observation that faces can exhibit strong highlight reflections from a broad range of lighting directions, we propose a deep neural network for extracting highlights from faces, and then trace these reflections back to the scene to acquire the environment map.
27 Citations
Hybrid Face Reflectance, Illumination, and Shape from a Single Image.
- Computer ScienceIEEE transactions on pattern analysis and machine intelligence
- 2021
This work proposes HyFRIS-Net to jointly estimate the hybrid reflectance and illumination models, as well as the refined face shape from a single unconstrained face image in a pre-defined texture space to ensure photometric face appearance modeling in both parametric and non-parametric spaces for efficient learning.
Highlight Removal in Facial Images
- Computer SciencePRCV
- 2020
This work adopts the structure of conditional generative adversarial network (CGAN) to generate highlight-free images in facial images, which is, to the best knowledge, the largest image dataset for facial highlight removal.
Spec-Net and Spec-CGAN: Deep learning models for specularity removal from faces
- Computer ScienceImage Vis. Comput.
- 2020
High-Dynamic-Range Lighting Estimation From Face Portraits
- Computer Science2020 International Conference on 3D Vision (3DV)
- 2020
It is shown that the predicted HDR environment maps can be used as accurate illumination sources for scene renderings, with potential applications in 3D object insertion for augmented reality.
Leveraging Multi-view Image Sets for Unsupervised Intrinsic Image Decomposition and Highlight Separation
- Computer ScienceAAAI
- 2020
An unsupervised approach for factorizing object appearance into highlight, shading, and albedo layers, trained by multi-view real images is presented, with a proposed image representation based on local color distributions that allows training to be insensitive to the local misalignments of multi-View images.
Deep Neural Models for Illumination Estimation and Relighting: A Survey
- Computer ScienceComput. Graph. Forum
- 2021
This contribution aims to bring together in a coherent manner current advances in this conjunction, presented in three categories: scene illumination estimation, relighting with reflectance‐aware scene‐specific representations and finally relighting as image‐to‐image transformations.
Object-based Illumination Estimation with Rendering-aware Neural Networks
- Computer ScienceECCV
- 2020
An approach that takes advantage of physical principles from inverse rendering to constrain the solution, while also utilizing neural networks to expedite the more computationally expensive portions of its processing, to increase robustness to noisy input data as well as to improve temporal and spatial stability is proposed.
Learning Illumination from Diverse Portraits
- Computer ScienceSIGGRAPH Asia Technical Communications
- 2020
This work presents a learning-based technique for estimating high dynamic range, omnidirectional illumination from a single low dynamic range portrait image captured under arbitrary indoor or outdoor lighting conditions, and shows that this technique outperforms the state-of-the-art technique for portrait-based lighting estimation.
Learning Scene Illumination by Pairwise Photos from Rear and Front Mobile Cameras
- Computer ScienceComput. Graph. Forum
- 2018
A learning based method to recover low‐frequency scene illumination represented as spherical harmonic functions by pairwise photos from rear and front cameras on mobile devices is proposed and produces visually and quantitatively superior results compared to the state‐of‐the‐arts.
Outdoor illumination estimation via all convolutional neural networks
- Computer ScienceComput. Electr. Eng.
- 2021
References
SHOWING 1-10 OF 55 REFERENCES
Real-time illumination estimation from faces for coherent rendering
- Computer Science2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)
- 2014
The approach provides plausible results, which considerably enhance the visual realism in real-time Augmented Reality applications, and recovers the most plausible real-world lighting conditions for measured reflections in a face, represented by a function depending on incident light angle using Spherical Harmonics.
From Faces to Outdoor Light Probes
- Computer ScienceComput. Graph. Forum
- 2018
This paper presents an approach to directly estimate an HDR light probe from a single LDR photograph, shot outdoors with a consumer camera, without specialized calibration targets or equipment, and shows that relighting objects with HDR light probes estimated by the method yields realistic results in a wide variety of settings.
Deep Reflectance Maps
- Computer Science2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- 2016
A convolutional neural architecture to estimate reflectance maps of specular materials in natural lighting conditions is proposed in an end-to-end learning formulation that directly predicts a reflectance map from the image itself.
Learning to predict indoor illumination from a single image
- Computer ScienceACM Trans. Graph.
- 2017
An end-to-end deep neural network is trained that directly regresses a limited field-of-view photo to HDR illumination, without strong assumptions on scene geometry, material properties, or lighting, which allows to automatically recover high-quality HDR illumination estimates that significantly outperform previous state- of-the-art methods.
Deep Outdoor Illumination Estimation
- Computer Science2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- 2017
It is demonstrated that the approach allows the recovery of plausible illumination conditions and enables photorealistic virtual object insertion from a single image and significantly outperforms previous solutions to this problem.
Specular Highlight Removal in Facial Images
- Engineering2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- 2017
This work employs a melanin and hemoglobin based model to represent the diffuse color variations in facial skin, and utilizes this model to constrain the highlight removal solution in a manner that is effective even for partially saturated pixels.
Illumination estimation and cast shadow detection through a higher-order graphical model
- Computer ScienceCVPR 2011
- 2011
A higher-order Markov Random Field illumination model, which combines low-level shadow evidence with high-level prior knowledge for the joint estimation of cast shadows and the illumination environment, is described.
Learning Detailed Face Reconstruction from a Single Image
- Computer Science2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- 2017
This work proposes to leverage the power of convolutional neural networks to produce a highly detailed face reconstruction from a single image, and introduces an end- to-end CNN framework which derives the shape in a coarse-to-fine fashion.
Illumination from Shadows
- Mathematics, Computer ScienceIEEE Trans. Pattern Anal. Mach. Intell.
- 2003
The effectiveness of using occluding information of incoming light in estimating an illumination distribution of a scene is demonstrated and an adaptive sampling framework for efficient estimation of illumination distribution is introduced.
Reflectance and Illumination Recovery in the Wild
- Computer ScienceIEEE Transactions on Pattern Analysis and Machine Intelligence
- 2016
A reflectance model and priors are developed that precisely capture the space of real-world object reflectance and a flexible illumination model that can represent real- world illumination with priors that combat the deleterious effects of image formation.