• Corpus ID: 16165021

The local low-dimensionality of natural images

  title={The local low-dimensionality of natural images},
  author={Olivier J. H{\'e}naff and Johannes Ball{\'e} and Neil C. Rabinowitz and Eero P. Simoncelli},
We develop a new statistical model for photographic images, in which the local responses of a bank of linear filters are described as jointly Gaussian, with zero mean and a covariance that varies slowly over spatial position. We optimize sets of filters so as to minimize the nuclear norms of matrices of their local activations (i.e., the sum of the singular values), thus encouraging a flexible form of sparsity that is not tied to any particular dictionary or coordinate system. Filters optimized… 

Figures from this paper

Super-Resolution with Deep Convolutional Sufficient Statistics

This paper proposes to use as conditional model a Gibbs distribution, where its sufficient statistics are given by deep convolutional neural networks, and the features computed by the network are stable to local deformation, and have reduced variance when the input is a stationary texture.

On the Sparse Structure of Natural Sounds and Natural Images: Similarities, Differences, and Implications for Neural Coding

A sparse coding network with synaptically local plasticity rules learns different sparse features from speech data than are found by more conventional sparse coding algorithms, but the learned features are qualitatively the same for these models when trained on natural images.

Self-Supervised Learning of a Biologically-Inspired Visual Texture Model

A model for representing visual texture in a low-dimensional feature space, along with a novel self-supervised learning objective that is used to train it on an unlabeled database of texture images, exhibits stronger representational similarity to texture responses of neural populations recorded in primate V2 than pre-trained deep CNNs.

Statistical learning models of sensory processing and implications of biological constraints

This dissertation explores efficient coding and sparse coding models of the visual and auditory systems, the data these systems process, and how these models are affected by the constraints imposed by implementation in biological neural systems.

Neural Networks Optimally Compress the Sawbridge

This work precisely characterize the optimal entropy-distortion tradeoff for this source and shows numerically that it is achieved by neural-network-based compressors trained via stochastic gradient descent, and shows both analytically and experimentally that compressors based on the classical Karhunen-Loeve transform are highly suboptimal at high rates.

The Effects of Image Distribution and Task on Adversarial Robustness

In this paper, we propose an adaptation to the area under the curve (AUC) metric to measure the adversarial robustness of a model over a particular -interval [ 0, 1] (interval of adversarial

Do Neural Networks Compress Manifolds Optimally?

The optimal entropy-distortion tradeoffs for two low-dimensional manifolds with circular structure are determined and it is shown that state-of-the-art ANN-based compressors fail to optimally compress them.

Efficient and adaptive sensory codes

This work develops a theoretical framework that can account for the dynamics of adaptation from an information-processing perspective, and uses this framework to optimize and analyze adaptive sensory codes, and shows that codes optimized for stationary environments can suffer from prolonged periods of poor performance when the environment changes.

Minimalistic Unsupervised Learning with the Sparse Manifold Transform

Though there remains a small performance gap between the simple constructive model and SOTA methods, the evidence points to this as a promising direction for achieving a principled and white-box approach to unsupervised learning.



Nonlinear Extraction of Independent Components of Natural Images Using Radial Gaussianization

It is shown that distributions of spatially proximal bandpass filter responses are better described as elliptical than as linearly transformed independent sources, and it is demonstrated that the reduction in dependency achieved by applying RG to either nearby pairs or blocks of bandpass filters is significantly greater than that achieved by ICA.

Emergence of simple-cell receptive field properties by learning a sparse code for natural images

It is shown that a learning algorithm that attempts to find sparse linear codes for natural scenes will develop a complete family of localized, oriented, bandpass receptive fields, similar to those found in the primary visual cortex.

A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients

A universal statistical model for texture images in the context of an overcomplete complex wavelet transform is presented, demonstrating the necessity of subgroups of the parameter set by showing examples of texture synthesis that fail when those parameters are removed from the set.

The statistics of natural images

Recently there has been a resurgence of interest in the properties of natural images. Their statistics are important not only in image compression but also for the study of sensory processing in

Recovery of Sparse Translation-Invariant Signals With Continuous Basis Pursuit

This work develops two implementations of CBP for a one-dimensional translation-invariant source, one using a first-order Taylor approximation, and another using a form of trigonometric spline, and examines the tradeoff between sparsity and signal reconstruction accuracy in these methods.

The atoms of vision: Cartesian or polar?

It is shown that a more efficient representation can be obtained by a nonlinear encoding that yields a feature space with polar organization, and some striking similarities are pointed out between the polar representation in visual cortex and basic image-coding strategies pursued in shape-gain vector quantization schemes.

Emergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces

It is shown that the same principle of independence maximization can explain the emergence of phase- and shift-invariant features, similar to those found in complex cells, by maximizing the independence between norms of projections on linear subspaces.

Emergence of complex cell properties by learning to generalize in natural scenes

A model in which neural activity encodes the probability distribution most consistent with a given image is presented, which provides a new functional explanation for nonlinear effects in complex cells and offers insight into coding strategies in primary visual cortex (V1) and higher visual areas.

Independent component filters of natural images compared with simple cells in primary visual cortex

Properties of the receptive fields of simple cells in macaque cortex were compared with properties of independent component filters generated by independent component analysis on a large set of natural images: there is no match, however, in calculated and measured distributions for the peak of the spatial frequency response.