# Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation

@inproceedings{Bengio2021FlowNB, title={Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation}, author={Emmanuel Bengio and Moksh Jain and Maksym Korablyov and Doina Precup and Yoshua Bengio}, booktitle={NeurIPS}, year={2021} }

This paper is about the problem of learning a stochastic policy for generating an object (like a molecular graph) from a sequence of actions, such that the probability of generating an object is proportional to a given positive reward for that object. Whereas standard return maximization tends to converge to a single return-maximizing sequence, there are cases where we would like to sample a diverse set of high-return solutions. These arise, for example, in black-box function optimization when…

## Figures from this paper

## 26 Citations

### GFlowNet Foundations

- Computer ScienceArXiv
- 2021

Additional theoretical properties of GFlowNets are shown, enabling the estimation of entropy and mutual information, sampling from a Pareto frontier, connections to reward-maximizing policies, and extensions to stochastic environments, continuous actions and modular energy functions.

### Bayesian Structure Learning with Generative Flow Networks

- Computer ScienceArXiv
- 2022

This work proposes to use a GFlowNet as an alternative to MCMC for approximating the posterior distribution over the structure of Bayesian networks, given a dataset of observations, and it compares favorably against other methods based on MCMC or variational inference.

### Generative Flow Networks for Discrete Probabilistic Modeling

- Computer ScienceICML
- 2022

This work presents energy-based generative networks (EB-GFN), a novel probabilistic modeling algorithm for high-dimensional discrete data, and proposes a framework to jointly train a GFlowNet with an energy function, so that the G FlowNet learns to sample from the energy distribution, while the energy learns with an approximate MLE objective with negative samples from the G flowNet.

### Trajectory Balance: Improved Credit Assignment in GFlowNets

- Computer ScienceArXiv
- 2022

It is proved that any global minimizer of the trajectory balance objective can define a policy that samples exactly from the target distribution, and the benefits of the trajectories balance objective for GFlowNet convergence, diversity of generated samples, and robustness to long action sequences and large action spaces are demonstrated.

### A Dataset Perspective on Offline Reinforcement Learning

- Computer Science
- 2021

It is found, that popular algorithms in Ofﬂine RL are strongly inﬁuenced by the characteristics of the dataset and the average performance across different datasets might not be enough for a fair comparison.

### Biological Sequence Design with GFlowNets

- Computer ScienceICML
- 2022

This work proposes an active learning algorithm leveraging epistemic uncertainty estimation and the recently proposed GFlowNets as a generator of diverse candidate solutions, with the objective to obtain a diverse batch of useful and informative candidates after each round.

### GEO: Enhancing Combinatorial Optimization with Classical and Quantum Generative Models

- Computer Science
- 2021

It is shown that TN-GEO can propose unseen candidates with lower cost function values than the candidates seen by classical solvers, the first demonstration of the generalization capabilities of quantum-inspired generative models that provide real value in the context of an industrial application.

### A SYMMETRY L EARNING FOR C OUNTERFACTUAL I NVARIANT C LASSIFICATION IN OOD T ASKS

- Computer Science
- 2022

This work argues that when the transformations in train T tr and test T te are (arbitrary) symmetry transformations induced by a collection of known m equivalence relations, the task of building a robust OOD classiﬁer can be deﬂned as deﷁned as the simplest causal model that creates a causal connection between the target labels and the symmetry transformations that are associated with label changes.

### Unifying Generative Models with GFlowNets

- Computer ScienceArXiv
- 2022

A short note on the connections between existing deep generative models and the GFlowNet framework is presented, shedding light on their overlapping traits and pro-viding a unifying viewpoint through the lens of learning with Markovian trajectories.

### Edge Rewiring Goes Neural: Boosting Network Resilience without Rich Features

- Computer Science
- 2021

Reforms resilience optimization as an MDP equipped with edge rewiring action space, and proposes a pure topology-oriented variant of GNN called FireGNN, which can learn from graphs without rich features, and achieves a near-optimal resilience gain on various graphs while balancing the utility.

## References

SHOWING 1-10 OF 50 REFERENCES

### Model-based reinforcement learning for biological sequence design

- Computer ScienceICLR
- 2020

A model-based variant of PPO, DyNA-PPO, is proposed to improve sample efficiency and performs significantly better than existing methods in settings in which modeling is feasible, while still not performing worse in situations in which a reliable model cannot be learned.

### Data Generation as Sequential Decision Making

- Computer ScienceNIPS
- 2015

This work forms data imputation as an MDP and develops models capable of representing effective policies for it, construct the models using neural networks and train them using a form of guided policy search.

### Approximate Inference in Discrete Distributions with Monte Carlo Tree Search and Value Functions

- Computer ScienceAISTATS
- 2020

The TreeSample algorithm is proposed, an adaptation of Monte Carlo Tree Search to approximate inference that caches all previous queries to the density oracle in an explicit search tree and dynamically allocates new queries based on a "best-first" heuristic for exploration, using existing upper confidence bound methods.

### MolGAN: An implicit generative model for small molecular graphs

- Computer ScienceArXiv
- 2018

MolGAN is introduced, an implicit, likelihood-free generative model for small molecular graphs that circumvents the need for expensive graph matching procedures or node ordering heuris-tics of previous likelihood-based methods.

### Amortized Bayesian Optimization over Discrete Spaces

- Computer ScienceUAI
- 2020

On several challenging discrete design problems, this method generally outperforms other methods at optimizing the inner acquisition function, resulting in more efficient optimization of the outer black-box objective.

### MolecularRNN: Generating realistic molecular graphs with optimized properties

- Computer ScienceArXiv
- 2019

MolecularRNN, the graph recurrent generative model for molecular structures, is presented, which generates diverse realistic molecular graphs after likelihood pretraining on a big database of molecules.

### Learning To Navigate The Synthetically Accessible Chemical Space Using Reinforcement Learning

- Computer ScienceICML
- 2020

A novel forward synthesis framework powered by reinforcement learning (RL) for de novo drug design, Policy Gradient for Forward Synthesis (PGFS), that addresses this challenge by embedding the concept of synthetic accessibility directly into the de noVO drug design system.

### Learning to Solve Network Flow Problems via Neural Decoding

- Computer Science
- 2020

This work interprets the output of the neural network as a noisy codeword, where the codebook is given by the optimization problem's KKT conditions, and proposes a feedforward decoding strategy that finds the optimal set of active constraints.

### Learning Discrete Energy-based Models via Auxiliary-variable Local Exploration

- Computer ScienceNeurIPS
- 2020

AlOE is proposed, a new algorithm for learning conditional and unconditional EBMs for discrete structured data, where parameter gradients are estimated using a learned sampler that mimics local search, and shows that learning local search leads to significant improvements in challenging application domains.

### Oops I Took A Gradient: Scalable Sampling for Discrete Distributions

- Computer ScienceICML
- 2021

This work proposes a general and scalable approximate sampling strategy for probabilistic models with discrete variables that outperforms variational auto-encoders and existing energy-based models and gives bounds showing that this approach is near-optimal in the class of samplers which propose local updates.