• Corpus ID: 235367990

Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation

@inproceedings{Bengio2021FlowNB,
  title={Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation},
  author={Emmanuel Bengio and Moksh Jain and Maksym Korablyov and Doina Precup and Yoshua Bengio},
  booktitle={NeurIPS},
  year={2021}
}
This paper is about the problem of learning a stochastic policy for generating an object (like a molecular graph) from a sequence of actions, such that the probability of generating an object is proportional to a given positive reward for that object. Whereas standard return maximization tends to converge to a single return-maximizing sequence, there are cases where we would like to sample a diverse set of high-return solutions. These arise, for example, in black-box function optimization when… 

GFlowNet Foundations

TLDR
Additional theoretical properties of GFlowNets are shown, enabling the estimation of entropy and mutual information, sampling from a Pareto frontier, connections to reward-maximizing policies, and extensions to stochastic environments, continuous actions and modular energy functions.

Bayesian Structure Learning with Generative Flow Networks

TLDR
This work proposes to use a GFlowNet as an alternative to MCMC for approximating the posterior distribution over the structure of Bayesian networks, given a dataset of observations, and it compares favorably against other methods based on MCMC or variational inference.

Generative Flow Networks for Discrete Probabilistic Modeling

TLDR
This work presents energy-based generative networks (EB-GFN), a novel probabilistic modeling algorithm for high-dimensional discrete data, and proposes a framework to jointly train a GFlowNet with an energy function, so that the G FlowNet learns to sample from the energy distribution, while the energy learns with an approximate MLE objective with negative samples from the G flowNet.

Trajectory Balance: Improved Credit Assignment in GFlowNets

TLDR
It is proved that any global minimizer of the trajectory balance objective can define a policy that samples exactly from the target distribution, and the benefits of the trajectories balance objective for GFlowNet convergence, diversity of generated samples, and robustness to long action sequences and large action spaces are demonstrated.

A Dataset Perspective on Offline Reinforcement Learning

TLDR
It is found, that popular algorithms in Offline RL are strongly infiuenced by the characteristics of the dataset and the average performance across different datasets might not be enough for a fair comparison.

Biological Sequence Design with GFlowNets

TLDR
This work proposes an active learning algorithm leveraging epistemic uncertainty estimation and the recently proposed GFlowNets as a generator of diverse candidate solutions, with the objective to obtain a diverse batch of useful and informative candidates after each round.

GEO: Enhancing Combinatorial Optimization with Classical and Quantum Generative Models

TLDR
It is shown that TN-GEO can propose unseen candidates with lower cost function values than the candidates seen by classical solvers, the first demonstration of the generalization capabilities of quantum-inspired generative models that provide real value in the context of an industrial application.

A SYMMETRY L EARNING FOR C OUNTERFACTUAL I NVARIANT C LASSIFICATION IN OOD T ASKS

TLDR
This work argues that when the transformations in train T tr and test T te are (arbitrary) symmetry transformations induced by a collection of known m equivalence relations, the task of building a robust OOD classifier can be deflned as deﷁned as the simplest causal model that creates a causal connection between the target labels and the symmetry transformations that are associated with label changes.

Unifying Generative Models with GFlowNets

TLDR
A short note on the connections between existing deep generative models and the GFlowNet framework is presented, shedding light on their overlapping traits and pro-viding a unifying viewpoint through the lens of learning with Markovian trajectories.

Edge Rewiring Goes Neural: Boosting Network Resilience without Rich Features

TLDR
Reforms resilience optimization as an MDP equipped with edge rewiring action space, and proposes a pure topology-oriented variant of GNN called FireGNN, which can learn from graphs without rich features, and achieves a near-optimal resilience gain on various graphs while balancing the utility.

References

SHOWING 1-10 OF 50 REFERENCES

Model-based reinforcement learning for biological sequence design

TLDR
A model-based variant of PPO, DyNA-PPO, is proposed to improve sample efficiency and performs significantly better than existing methods in settings in which modeling is feasible, while still not performing worse in situations in which a reliable model cannot be learned.

Data Generation as Sequential Decision Making

TLDR
This work forms data imputation as an MDP and develops models capable of representing effective policies for it, construct the models using neural networks and train them using a form of guided policy search.

Approximate Inference in Discrete Distributions with Monte Carlo Tree Search and Value Functions

TLDR
The TreeSample algorithm is proposed, an adaptation of Monte Carlo Tree Search to approximate inference that caches all previous queries to the density oracle in an explicit search tree and dynamically allocates new queries based on a "best-first" heuristic for exploration, using existing upper confidence bound methods.

MolGAN: An implicit generative model for small molecular graphs

TLDR
MolGAN is introduced, an implicit, likelihood-free generative model for small molecular graphs that circumvents the need for expensive graph matching procedures or node ordering heuris-tics of previous likelihood-based methods.

Amortized Bayesian Optimization over Discrete Spaces

TLDR
On several challenging discrete design problems, this method generally outperforms other methods at optimizing the inner acquisition function, resulting in more efficient optimization of the outer black-box objective.

MolecularRNN: Generating realistic molecular graphs with optimized properties

TLDR
MolecularRNN, the graph recurrent generative model for molecular structures, is presented, which generates diverse realistic molecular graphs after likelihood pretraining on a big database of molecules.

Learning To Navigate The Synthetically Accessible Chemical Space Using Reinforcement Learning

TLDR
A novel forward synthesis framework powered by reinforcement learning (RL) for de novo drug design, Policy Gradient for Forward Synthesis (PGFS), that addresses this challenge by embedding the concept of synthetic accessibility directly into the de noVO drug design system.

Learning to Solve Network Flow Problems via Neural Decoding

TLDR
This work interprets the output of the neural network as a noisy codeword, where the codebook is given by the optimization problem's KKT conditions, and proposes a feedforward decoding strategy that finds the optimal set of active constraints.

Learning Discrete Energy-based Models via Auxiliary-variable Local Exploration

TLDR
AlOE is proposed, a new algorithm for learning conditional and unconditional EBMs for discrete structured data, where parameter gradients are estimated using a learned sampler that mimics local search, and shows that learning local search leads to significant improvements in challenging application domains.

Oops I Took A Gradient: Scalable Sampling for Discrete Distributions

TLDR
This work proposes a general and scalable approximate sampling strategy for probabilistic models with discrete variables that outperforms variational auto-encoders and existing energy-based models and gives bounds showing that this approach is near-optimal in the class of samplers which propose local updates.