# Reinforcement-Learning-Based Variational Quantum Circuits Optimization for Combinatorial Problems

@article{Khairy2019ReinforcementLearningBasedVQ, title={Reinforcement-Learning-Based Variational Quantum Circuits Optimization for Combinatorial Problems}, author={Sami Khairy and Ruslan Shaydulin and Lukasz Cincio and Yuri Alexeev and Prasanna Balaprakash}, journal={ArXiv}, year={2019}, volume={abs/1911.04574} }

Quantum computing exploits basic quantum phenomena such as state superposition and entanglement to perform computations. The Quantum Approximate Optimization Algorithm (QAOA) is arguably one of the leading quantum algorithms that can outperform classical state-of-the-art methods in the near term. QAOA is a hybrid quantum-classical algorithm that combines a parameterized quantum state evolution with a classical optimization routine to approximately solve combinatorial problems. The quality of…

## 21 Citations

### Learning to Optimize Variational Quantum Circuits to Solve Combinatorial Problems

- Computer ScienceAAAI
- 2020

Two machine-learning-based approaches are developed that reduce the optimality gap by factors up to 30.15 when compared with other commonly used off-the-shelf optimizers for Quantum Approximate Optimization Algorithm parameters.

### Iterative-Free Quantum Approximate Optimization Algorithm Using Neural Networks

- Computer Science
- 2022

A practical method that uses a simple, fully connected neural network that leverages previous executions of QAOA to create better initialization parameters tailored to a new given problem instance, and the parameters predicted by the neural network are shown to match very well with the fully optimized parameters.

### Optimizing quantum annealing schedules with Monte Carlo tree search enhanced with neural networks

- Computer ScienceNature Machine Intelligence
- 2022

A Monte Carlo tree search algorithm and its enhanced version boosted by neural networks are proposed to automate the design of annealing schedules in a hybrid quantum–classical framework and demonstrate in benchmark studies that MCTS and QZero perform more efficiently than other reinforcement learning algorithms in designing annealed schedules.

### Variational quantum compiling with double Q-learning

- Computer ScienceArXiv
- 2021

A variational quantum compiling (VQC) algorithm based on reinforcement learning is proposed in order to automatically design the structure of quantum circuit for VQC with no human intervention, and can reduce the errors of quantum algorithms due to decoherence process and gate noise in NISQ devices, and enable quantum algorithms especially for complex algorithms to be executed within coherence time.

### Graph neural network initialisation of quantum approximate optimisation

- Computer ScienceArXiv
- 2021

This work addresses two problems in the quantum approximate optimisation algorithm (QAOA), how to select initial parameters, and how to subsequently train the parameters to find an optimal solution, and demonstrates how the QAOA can be trained as an end-to-end differentiable pipeline.

### Efficient protocol for solving combinatorial graph problems on neutral-atom quantum processors

- Computer Science
- 2022

This work proposes a novel protocol for solving hard combinatorial graph problems that combines variational analog quantum computing and machine learning and shows that the proposed protocol can reduce dramatically the number of iterations to be run on the quantum device.

### Optimizing Quantum Annealing Schedules: From Monte Carlo Tree Search to QuantumZero

- Computer Science
- 2020

MCTS and QZero are found to be more efficient than many other leading reinforcement leanring algorithms for the task of desining annealing schedules and if there is a need to solve a large set of similar problems using a quantum annealer, QZero is the method of choice when the neural networks are first pre-trained with examples solved in the past.

### Noise-Robust End-to-End Quantum Control using Deep Autoregressive Policy Networks

- Computer ScienceMSML
- 2021

This work presents a hybrid policy gradient algorithm capable of simultaneously optimizing continuous and discrete degrees of freedom in an uncertainty-resilient way, modeled by a deep autoregressive neural network to capture causality.

### Predicting parameters for the Quantum Approximate Optimization Algorithm for MAX-CUT from the infinite-size limit

- Computer ScienceArXiv
- 2021

This work partially addresses issues for a specific combinatorial optimization problem: diluted spin models, with MAX-CUT as a notable special case, and provides good initial, if not nearly optimal, variational parameters for very small problem instances where the infinite-size limit assumption is clearly violated.

### Application of Quantum Machine Learning to VLSI Placement

- Computer Science2020 ACM/IEEE 2nd Workshop on Machine Learning for CAD (MLCAD)
- 2020

The Variational Quantum Eigensolver (VQE) is used to formulate a recursive Balanced Min-Cut (BMC) algorithm, and it is suggested that quantum machine learning techniques can lower error rates and allow for faster convergence to an optimal solution.

## References

SHOWING 1-10 OF 22 REFERENCES

### Improving Variational Quantum Optimization using CVaR

- PhysicsQuantum
- 2020

This paper empirically shows that the Conditional Value-at-Risk as an aggregation function leads to faster convergence to better solutions for all combinatorial optimization problems tested in this study.

### Performance of the Quantum Approximate Optimization Algorithm on the Maximum Cut Problem

- Computer Science
- 2018

It is found that QAOA can amortize the training cost by optimizing on batches of problems instances, and can exceed the performance of the classical polynomial time Goemans-Williamson algorithm with modest circuit depth, and that performance with fixed circuit depth is insensitive to problem size.

### Multistart Methods for Quantum Approximate optimization

- Computer Science2019 IEEE High Performance Extreme Computing Conference (HPEC)
- 2019

This paper studies the use of a multistart optimization approach within QAOA to improve the performance of quantum machines on important graph clustering problems and demonstrates that reusing the optimal parameters from similar problems can improve theperformance of classical optimization methods.

### Learning to learn with quantum neural networks via classical neural networks

- Computer ScienceArXiv
- 2019

This work trains classical recurrent neural networks to assist in the quantum learning process, also know as meta-learning, to rapidly find approximate optima in the parameter landscape for several classes of quantum variational algorithms.

### Optimizing Variational Quantum Algorithms using Pontryagin's Minimum Principle

- PhysicsArXiv
- 2016

It is shown that for a fixed computation time, the optimal evolution has a bang-bang (square pulse) form, both for closed and open quantum systems with Markovian decoherence, and support the choice of evolution ansatz in the recently proposed Quantum Approximate Optimization Algorithm.

### Quantum Approximate Optimization Algorithm: Performance, Mechanism, and Implementation on Near-Term Devices

- Computer Science, PhysicsPhysical Review X
- 2020

An in-depth study of the performance of QAOA on MaxCut problems is provided by developing an efficient parameter-optimization procedure and revealing its ability to exploit non-adiabatic operations, illustrating that optimization will be important only for problem sizes beyond numerical simulations, but accessible on near-term devices.

### The Quantum Approximation Optimization Algorithm for MaxCut: A Fermionic View

- Computer Science
- 2017

The parameter landscape is numerically investigated and it is shown that it is a simple one in the sense of having no local optima, which greatly simplifies numerical search for the optimal values of the parameters.

### Training A Quantum Optimizer

- Computer Science
- 2016

The goal is to find a quantum algorithm that, given an instance of MAX-2-SAT, will produce a state with high overlap with the optimal state, and the parameters that are found produce significantly larger overlap than the optimized annealing times of CFLLS.

### A Quantum Approximate Optimization Algorithm

- Computer Science, Mathematics
- 2014

A quantum algorithm that produces approximate solutions for combinatorial optimization problems that depends on a positive integer p and the quality of the approximation improves as p is increased, and is studied as applied to MaxCut on regular graphs.

### Strategies for quantum computing molecular energies using the unitary coupled cluster ansatz

- Computer ScienceQuantum Science and Technology
- 2018

The application of VQE to the simulation of molecular energies using the unitary coupled cluster (UCC) ansatz is studied and an analytical method to compute the energy gradient is proposed that reduces the sampling cost for gradient estimation by several orders of magnitude compared to numerical gradients.