Molecular Design in Synthetically Accessible Chemical Space via Deep Reinforcement Learning

  title={Molecular Design in Synthetically Accessible Chemical Space via Deep Reinforcement Learning},
  author={Julien Horwood and Emmanuel Noutahi},
  journal={ACS Omega},
  pages={32984 - 32994}
The fundamental goal of generative drug design is to propose optimized molecules that meet predefined activity, selectivity, and pharmacokinetic criteria. Despite recent progress, we argue that existing generative methods are limited in their ability to favorably shift the distributions of molecular properties during optimization. We instead propose a novel Reinforcement Learning framework for molecular design in which an agent learns to directly optimize through a space of synthetically… 

Figures and Tables from this paper

RetroGNN: Approximating Retrosynthesis by Graph Neural Networks for De Novo Drug Design

This work investigates the feasibility of training deep graph neural networks to approximate the outputs of a retrosynthesis planning software, and their use to bias the search process, and finds molecules predicted to be more likely to be antibiotics while maintaining good drug-like properties and being easily synthesizable.

Defining and Exploring Chemical Spaces

Benchmarking Deep Graph Generative Models for Optimizing New Drug Molecules for COVID-19

The VAE method produced two novel molecules with similar structures to the antiretroviral protease inhibitor Indinavir that show potential binding affinity for the SARS-CoV-2 protein target 3-chymotrypsin-like protease (3CL-protease).

Barking up the right tree: an approach to search over molecule synthesis DAGs

This work proposes a deep generative model that better represents the real world process, by directly outputting molecule synthesis DAGs, and argues that this provides sensible inductive biases, ensuring that the model searches over the same chemical space that chemists would also have access to, as well as interpretability.

A practical guide to multi-objective reinforcement learning and planning

This paper identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems.

On modeling and utilizing chemical compound information with deep learning technologies: A task-oriented approach

A practical guide to multi-objective reinforcement learning and planning

Real-world sequential decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement

An efficient graph generative model for navigating ultra-large combinatorial synthesis libraries

The proposed CSLVAE method performs fast and parallel batch inference for ultra-large synthesis libraries, enabling a number of important applications in early-stage drug discovery, and can encode out-of-library compounds and search for in-library analogues.

Computer-Aided Multi-Objective Optimization in Small Molecule Discovery

This review describes pool-based and de novo generative approaches to multi-objective molecular discovery with a focus on Pareto optimization algorithms and shows how pool- based molecular discovery is a rel-atively direct extension of multi- objective Bayesian optimization.



Optimization of Molecules via Deep Reinforcement Learning

Inspired by problems faced during medicinal chemistry lead optimization, the MolDQN model is extended with multi-objective reinforcement learning, which maximizes drug-likeness while maintaining similarity to the original molecule.

The Synthesizability of Molecules Proposed by Generative Models

This analysis suggests that to improve the utility of state-of-the-art generative models in real discovery workflows, new algorithm development is warranted.

ChemBO: Bayesian Optimization of Small Organic Molecules with Synthesizable Recommendations

ChemBO, a Bayesian optimization framework for generating and optimizing organic molecules for desired molecular properties, is described and a novel optimal-transport based distance and kernel that accounts for graphical information explicitly is proposed.

Automated de novo molecular design by hybrid machine intelligence and rule-driven chemical synthesis

The capability of machine learning models to capture implicit chemical knowledge from chemical reaction data and suggest feasible syntheses of new chemical matter is demonstrated.

Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation

Graph Convolutional Policy Network (GCPN) is proposed, a general graph convolutional network based model for goal-directed graph generation through reinforcement learning that can achieve 61% improvement on chemical property optimization over state-of-the-art baselines while resembling known molecules, and achieve 184% improved on the constrained property optimization task.

A Model to Search for Synthesizable Molecules

This work proposes a new molecule generation model that can generate diverse, valid and unique molecules due to the useful inductive biases of modeling reactions, and allows chemists to interrogate not only the properties of the generated molecules but also the feasibility of the synthesis routes.

MolGAN: An implicit generative model for small molecular graphs

MolGAN is introduced, an implicit, likelihood-free generative model for small molecular graphs that circumvents the need for expensive graph matching procedures or node ordering heuris-tics of previous likelihood-based methods.

Junction Tree Variational Autoencoder for Molecular Graph Generation

The junction tree variational autoencoder generates molecular graphs in two phases, by first generating a tree-structured scaffold over chemical substructures, and then combining them into a molecule with a graph message passing network, which allows for incrementally expand molecules while maintaining chemical validity at every step.

Reaction-Based Enumeration, Active Learning, and Free Energy Calculations To Rapidly Explore Synthetically Tractable Chemical Space and Optimize Potency of Cyclin-Dependent Kinase 2 Inhibitors

A new computational technique, referred to as 'PathFinder', that uses retrosynthetic analysis followed by combinatorial synthesis to generate novel compounds in synthetically accessible chemical space to accelerate the discovery of novel chemical matter in drug discovery campaigns is reported.

Constrained Graph Variational Autoencoders for Molecule Design

A variational autoencoder model in which both encoder and decoder are graph-structured is proposed and it is shown that by using appropriate shaping of the latent space, this model allows us to design molecules that are (locally) optimal in desired properties.