# Neurally-Guided Procedural Models: Learning to Guide Procedural Models with Deep Neural Networks

@article{Ritchie2016NeurallyGuidedPM, title={Neurally-Guided Procedural Models: Learning to Guide Procedural Models with Deep Neural Networks}, author={Daniel Ritchie and Anna T. Thomas and Pat Hanrahan and Noah D. Goodman}, journal={ArXiv}, year={2016}, volume={abs/1603.06143} }

We present a deep learning approach for speeding up constrained procedural modeling. Probabilistic inference algorithms such as Sequential Monte Carlo (SMC) provide powerful tools for constraining procedural models, but they require many samples to produce desirable results. In this paper, we show how to create procedural models which learn how to satisfy constraints. We augment procedural models with neural networks: these networks control how the model makes random choices based on what…

## 10 Citations

### Inference Compilation and Universal Probabilistic Programming

- Computer ScienceAISTATS
- 2017

We introduce a method for using deep neural networks to amortize the cost of inference in models from the family induced by universal probabilistic programming languages, establishing a framework…

### Nested Variational Inference

- Computer ScienceNeurIPS
- 2021

NVI is developed, a family of methods that learn proposals for nested importance samplers by minimizing an forward or reverse KL divergence at each level of nesting, and it is observed that optimizing nested objectives leads to improved sample quality in terms of log average weight and effective sample size.

### OpenStreetMap Street Graph Preparation Road Network Raster GAN Training Noise Image Road Network Raster Post-Processing Vector Road Network Patches Offline Stage Online Stage Generator Network Weights : Generator Weights : Generator

- Computer Science
- 2017

A novel example-based approach for road network synthesis relying on Generative Adversarial Networks (GANs), a recently introduced deep learning technique that is able to automatically synthesize a multitude of arbitrary sized street networks that faithfully reproduce the style of the original patch.

### Partial Procedural Geometric Model Fitting for Point Clouds

- Computer ScienceArXiv
- 2016

Experimental results show that the PPGMF method based on the proposed metric is able to fit non-complete data, while the methodbased on other metrics is unable, and it is shown that the method can be accelerated by several times via early rejection.

### Particle swarm optimization-based automatic parameter selection for deep neural networks and its applications in large-scale and high-dimensional data

- Computer SciencePloS one
- 2017

The DNN classifiers trained by the network configurations expressed by the final solutions of the PSO algorithm, employed to construct an ensemble model and individual classifier, outperform the random approach in terms of the generalization performance.

### Parametric L-systems-based modeling self-reconfiguration of modular robots in obstacle environments

- Computer Science
- 2018

A modeling framework that makes it possible to simulate and visualize the interactions at the level of decentralized modules will be introduced, which extends the formalism of Lindenmayer systems (L-systems) with constructs needed to model robotic information exchanged between decentralized modules and their surrounding environments.

### L-systems driven self-reconfiguration of modular robots

- Computer Science
- 2016

A novel algorithm for distributed self-reconfiguration by combining cellular automata and L-systems is presented, extending the turtle interpretation to modular robotics to generate local predictions for distributed modules from global description.

### Modeling the fractal development of modular robots

- Computer Science
- 2017

This work extends L-systems to the self-reconfiguration process of modules robots, and shows how extended graphical interpretation of L- system symbols can generate module-level predictions about robotic global states.

## References

SHOWING 1-10 OF 43 REFERENCES

### Procedural Modeling Using Autoencoder Networks

- Computer ScienceUIST
- 2015

This work proposes a method to enable intuitive exploration of such high dimensional procedural modeling spaces within a lower dimensional space learned through autoencoder network training, and structuring the learned lowerdimensional space for continuous exploration with respect to shape features.

### Stochastic Backpropagation and Approximate Inference in Deep Generative Models

- Computer ScienceICML
- 2014

We marry ideas from deep neural networks and approximate Bayesian inference to derive a generalised class of deep, directed generative models, endowed with a new algorithm for scalable inference and…

### Black Box Variational Inference

- Computer ScienceAISTATS
- 2014

This paper presents a "black box" variational inference algorithm, one that can be quickly applied to many models with little additional derivation, based on a stochastic optimization of the variational objective where the noisy gradient is computed from Monte Carlo samples from the Variational distribution.

### Controlling procedural modeling programs with stochastically-ordered sequential Monte Carlo

- Computer ScienceACM Trans. Graph.
- 2015

To handle the many possible sequentializations of a structured, recursive procedural modeling program, this work develops and proves the correctness of a new SMC variant, Stochastically-Ordered Sequential Monte Carlo (SOSMC), and implements it for general-purpose programs using a new programming primitive: the stochastic future.

### Neural Variational Inference and Learning in Belief Networks

- Computer ScienceICML
- 2014

This work proposes a fast non-iterative approximate inference method that uses a feedforward network to implement efficient exact sampling from the variational posterior and shows that it outperforms the wake-sleep algorithm on MNIST and achieves state-of-the-art results on the Reuters RCV1 document dataset.

### Auto-Encoding Variational Bayes

- Computer ScienceICLR
- 2014

A stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case is introduced.

### Guided Procedural Modeling

- Computer ScienceComput. Graph. Forum
- 2011

This work presents guided procedural modeling, a new approach that allows a high level of top‐down control by breaking the system into smaller building blocks that communicate and generalizes the concept of the environment.

### Interactive design of probability density functions for shape grammars

- Computer ScienceACM Trans. Graph.
- 2015

A framework that enables a user to interactively design a probability density function over a procedural shape space and to sample models according to the designed pdf and modify the original grammars to generate models with a pdf proportional to the user preference scores is presented.

### Recurrent Models of Visual Attention

- Computer ScienceNIPS
- 2014

A novel recurrent neural network model that is capable of extracting information from an image or video by adaptively selecting a sequence of regions or locations and only processing the selected regions at high resolution is presented.

### Neural Adaptive Sequential Monte Carlo

- Computer ScienceNIPS
- 2015

A new method for automatically adapting the proposal using an approximation of the Kullback-Leibler divergence between the true posterior and the proposal distribution is presented, applicable to any parameterized proposal distribution and it supports online and batch variants.