• Corpus ID: 236957052

Learning Foveated Reconstruction to Preserve Perceived Image Statistics

@article{Surace2021LearningFR,
  title={Learning Foveated Reconstruction to Preserve Perceived Image Statistics},
  author={Luca Surace and Marek Wernikowski and O. Tursun and Karol Myszkowski and Radoslaw Mantiuk and Piotr Didyk},
  journal={ArXiv},
  year={2021},
  volume={abs/2108.03499}
}
Foveated image reconstruction recovers full image from a sparse set of samples distributed according to the human visual system’s retinal sensitivity that rapidly drops with eccentricity. Recently, the use of Generative Adversarial Networks was shown to be a promising solution for such a task as they can successfully hallucinate missing image information. Like for other supervised learning approaches, also for this one, the definition of the loss function and training strategy heavily… 

Foveated rendering: A state-of-the-art survey

This survey reviews foveated rendering research from 1990 to 2021, revisits the visual perceptual models related to foveate rendering, proposes a new foveation rendering taxonomy, and discusses potential opportunities and open questions in the field.

An Effective Foveated 360° Image Assessment Based on Graph Convolution Network

This paper addresses the above problems by proposing an accurate machine-learning-based quality assessment model for foveated 360° images that is proven to outperform the three cutting-edge machine- learning-based models, which apply deep learning techniques and 25 traditional-metric- based models (or analytical-function-based-models), which utilize analytical functions.

Metameric Varifocal Holograms

A new CGH method is proposed that exploits gaze-contingency and perceptual graphics to accelerate the development of practical holographic display systems and prioritises and improves foveal visual quality without causing perceptually visible distortions at the periphery.

Metameric Varifocal Holography

A new CGH method is proposed that exploits gaze-contingency and perceptual graphics to accelerate the development of practical holographic display systems and prioritises and improves foveal visual quality without causing perceptually visible distortions at the periphery.

References

SHOWING 1-10 OF 71 REFERENCES

Retinal limits to the detection and resolution of gratings.

Quantitative comparison of the data with retinal anatomy and physiology suggests that pattern resolution islimited by the spacing of primate beta (midget) retinal ganglion cells, whereas pattern detection is limited by the size of individual cones.

FovVideoVDP: a visible difference predictor for wide field-of-view video

The heat-maps show the metric’s predictions for 4 types of distortions at three different eccentricities, and all types of artifacts are predicted to be much less noticeable when seen with peripheral vision at large eccentricities.

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

A new dataset of human perceptual similarity judgments is introduced and it is found that deep features outperform all previous metrics by large margins on this dataset, and suggests that perceptual similarity is an emergent property shared across deep visual representations.

Deepfovea: neural reconstruction for foveated rendering and video compression using learned natural video statistics

This research presents a novel approach to real-time estimation of gaze, enabling gaze-contingent rendering and compression methods that take advantage of this acuity falloff in VR displays.

A summary statistic representation in peripheral vision explains visual search.

It is argued that the key issue in search is the processing of image patches in the periphery, where visual representation is characterized by summary statistics computed over a sizable pooling region that predicts a set of classic search results, as well as peripheral discriminability of crowded patches such as those found in search displays.

Foveated 3D graphics

This work exploits the falloff of acuity in the visual periphery to accelerate graphics computation by a factor of 5-6 on a desktop HD display, and develops a general and efficient antialiasing algorithm easily retrofitted into existing graphics code to minimize "twinkling" artifacts in the lower-resolution layers.

Why Are Deep Representations Good Perceptual Quality Features?

It is demonstrated that the pre-trained CNN features which receive higher scores are better at predicting human quality judgment, and the possibility of using the method to select deep features to form a new loss function, which improves the image reconstruction quality for the well-known single-image super-resolution problem.

Supplementary Material for Foveated AR: Dynamically-Foveated Augmented Reality Display

This work presents the Supplementary Material for Foveated AR: Dynamically-Foveated Augmented Reality Display, a meta-modelling framework for dynamically-foveated augmented reality display, which aims to provide real-time information about the pixels of the eye in a 3D environment.

Metamers of neural networks reveal divergence from human perceptual systems

The results reveal discrepancies between model and human representations, but also show how metamers can help guide model refinement and elucidate model representations.

Measurements of contrast sensitivity for peripheral vision

Contrast detection thresholds were measured for sine-gratings with Gaussian patches for four mixed chromatic/achromatic with a varying share of the achromatic components and the results plotted as the contrast sensitivity function (CSF) follow the state-of-the-art CSF models.
...