Landscape Learning for Neural Network Inversion

  title={Landscape Learning for Neural Network Inversion},
  author={Ruoshi Liu and Chen-Guang Mao and Purva Tendulkar and Hongya Wang and Carl Vondrick},
Many machine learning methods operate by inverting a neural network at inference time, which has become a popular technique for solving inverse problems in computer vision, robotics, and graphics. However, these methods often involve gradient descent through a highly non-convex loss landscape, causing the optimization process to be unstable and slow. We introduce a method that learns a loss landscape where gradient descent is efficient, bringing massive improvement and acceleration to the… 

Figures and Tables from this paper


Transforming and Projecting Images into Class-conditional Generative Networks
It is demonstrated that one can solve for image translation, scale, and global color transformation, during the projection optimization to address the object-center bias and color bias of a Generative Adversarial Network.
In-Domain GAN Inversion for Real Image Editing
An in-domain GAN inversion approach, which not only faithfully reconstructs the input image but also ensures the inverted code to be semantically meaningful for editing, which achieves satisfying real image reconstruction and facilitates various image editing tasks, significantly outperforming start-of-the-arts.
Invertible Conditional GANs for image editing
This work evaluates encoders to inverse the mapping of a cGAN, i.e., mapping a real image into a latent space and a conditional representation, which allows to reconstruct and modify real images of faces conditioning on arbitrary attributes.
Generative Adversarial Nets
We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a
Playing Atari with Deep Reinforcement Learning
This work presents the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning, which outperforms all previous approaches on six of the games and surpasses a human expert on three of them.
Generative Visual Manipulation on the Natural Image Manifold
This paper proposes to learn the natural image manifold directly from data using a generative adversarial neural network, and defines a class of image editing operations, and constrain their output to lie on that learned manifold at all times.
LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
This work proposes to amplify human effort through a partially automated labeling scheme, leveraging deep learning with humans in the loop, and constructs a new image dataset, LSUN, which contains around one million labeled images for each of 10 scene categories and 20 object categories.
Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining
An improved method for efficient black-box optimization is introduced, which performs the optimization in the low-dimensional, continuous latent manifold learned by a deep generative model, which can be easily implemented on top of existing methods.
On the "steerability" of generative adversarial networks
It is shown that although current GANs can fit standard datasets very well, they still fall short of being comprehensive models of the visual manifold, and it is hypothesized that the degree of distributional shift is related to the breadth of the training data distribution.
A Style-Based Generator Architecture for Generative Adversarial Networks
An alternative generator architecture for generative adversarial networks is proposed, borrowing from style transfer literature, that improves the state-of-the-art in terms of traditional distribution quality metrics, leads to demonstrably better interpolation properties, and also better disentangles the latent factors of variation.