# Variational Quantum Circuits for Deep Reinforcement Learning

@article{Chen2020VariationalQC, title={Variational Quantum Circuits for Deep Reinforcement Learning}, author={Samuel Yen-Chi Chen and Chao-Han Huck Yang and Jun Qi and Pin-Yu Chen and Xiaoli Ma and Hsi-Sheng Goan}, journal={IEEE Access}, year={2020}, volume={8}, pages={141007-141024} }

The state-of-the-art machine learning approaches are based on classical von Neumann computing architectures and have been widely used in many industrial and academic domains. With the recent development of quantum computing, researchers and tech-giants have attempted new quantum circuits for machine learning tasks. However, the existing quantum computing platforms are hard to simulate classical deep learning models or problems because of the intractability of deep quantum circuits. Thus, it is…

## Figures and Tables from this paper

## 99 Citations

### Reinforcement Learning with Quantum Variational Circuits

- Computer Science, PhysicsAAAI 2020
- 2020

Results indicate both hybrid and pure quantum variational circuit have the ability to solve reinforcement learning tasks with a smaller parameter space.

### Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning

- Computer Science
- 2021

A training method for parametrized quantum circuits (PQCs) that can be used to solve RL tasks for discrete and continuous state spaces based on the deep Q-learning algorithm and shows when recent separation results between classical and quantum agents for policy gradient RL can be extended to inferring optimal Q-values in restricted families of environments.

### Quantum Architecture Search via Deep Reinforcement Learning

- Computer Science, PhysicsArXiv
- 2021

A quantum architecture search framework with the power of deep reinforcement learning (DRL) to address the challenge of generation of quantum gate sequences for multiqubit GHZ states without encoding any knowledge of quantum physics in the agent.

### Quantum Architecture Search via Continual Reinforcement Learning

- Computer ScienceArXiv
- 2021

The Probabilistic Policy Reuse with deep Q-learning (PPR-DQL) framework is presented and it is demonstrated that the RL agent with PPR was able to find the quantum gate sequence to generate the two-qubit Bell state faster than the agent that was trained from scratch.

### Parametrized Quantum Policies for Reinforcement Learning

- Computer ScienceNeurIPS
- 2021

This work proposes a hybrid quantum-classical reinforcement learning model using very few qubits, which it is shown can be effectively trained to solve several standard benchmarking environments and formally proves the ability of parametrized quantum circuits to solve certain learning tasks that are intractable to classical models.

### Quantum Computing Aided Machine Learning Through Quantum State Fidelity

- Computer Science
- 2021

This work proposes a quantum deep learning architecture and demonstrates the quantum neural network architecture on tasks ranging from binary and multi-class classification to generative modelling and outperforms other quantum based GANs in the literature for up to 125.0% in terms of similarity between generated distributions and original data sets.

### Variational quantum policies for reinforcement learning

- Computer ScienceArXiv
- 2021

This work investigates how to construct and train reinforcement learning policies based on variational quantum circuits, and proposes and shows the existence of task environments with a provable separation in performance between quantum learning agents and any polynomial-time classical learner.

### Quantum Enhancements for Deep Reinforcement Learning in Large Spaces

- Computer Science
- 2020

This work studies the state-of-the-art neural-network approaches for reinforcement learning with quantum enhancements in mind and demonstrates the substantial learning advantage that models with a sampling bottleneck can provide over conventional neural network architectures in complex learning environments.

### Variational quantum reinforcement learning via evolutionary optimization

- Computer Science, PhysicsMach. Learn. Sci. Technol.
- 2022

A hybrid framework is proposed where the quantum RL agents are equipped with a hybrid tensor network-variational quantum circuit (TN-VQC) architecture to handle inputs of dimensions exceeding the number of qubits, enabling further quantum RL applications on noisy intermediate-scale quantum devices.

### Optimizing Quantum Variational Circuits with Deep Reinforcement Learning

- Computer ScienceArXiv
- 2021

It is found that reinforcement learning augmented optimizers consistently outperform gradient descent in noisy environments.

## References

SHOWING 1-10 OF 76 REFERENCES

### The Expressive Power of Parameterized Quantum Circuits

- Computer SciencePhysical Review Research
- 2020

It is proved that PQCs with a simple structure already outperform any classical neural network for generative tasks, unless the polynomial hierarchy collapses, and they are employed as an application for Bayesian learning.

### Quantum circuit learning

- Computer Science, PhysicsPhysical Review A
- 2018

A classical-quantum hybrid algorithm for machine learning on near-term quantum processors, which is hybridizing a low-depth quantum circuit and a classical computer for machinelearning, paves the way toward applications of near- term quantum devices for quantum machine learning.

### Deep reinforcement learning for quantum gate control

- Computer ScienceEPL (Europhysics Letters)
- 2019

A dueling double deep Q-learning neural network is constructed to find out the optimized time dependence of controllable parameters to implement two typical quantum gates: a single-qu bit Hadamard gate and a two-qubit CNOT gate.

### Evaluating analytic gradients on quantum hardware

- Computer SciencePhysical Review A
- 2019

This paper shows how gradients of expectation values of quantum measurements can be estimated using the same, or almost the same the architecture that executes the original circuit, and proposes recipes for the computation of gradients for continuous-variable circuits.

### Classification with Quantum Neural Networks on Near Term Processors

- Physics, Computer Science
- 2018

This work introduces a quantum neural network, QNN, that can represent labeled data, classical or quantum, and be trained by supervised learning, and shows through classical simulation that parameters can be found that allow the QNN to learn to correctly distinguish the two data sets.

### PennyLane: Automatic differentiation of hybrid quantum-classical computations

- Computer ScienceArXiv
- 2018

PennyLane's core feature is the ability to compute gradients of variational quantum circuits in a way that is compatible with classical techniques such as backpropagation, and it extends the automatic differentiation algorithms common in optimization and machine learning to include quantum and hybrid computations.

### Quantum Reinforcement Learning

- Computer ScienceIEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)
- 2008

The results demonstrate the effectiveness and superiority of the QRL algorithm for some complex problems and shows that this approach makes a good tradeoff between exploration and exploitation using the probability amplitude and can speedup learning through the quantum parallelism.

### Optimizing Quantum Error Correction Codes with Reinforcement Learning

- Computer ScienceQuantum
- 2019

This work considers a reinforcement learning agent tasked with modifying a family of surface code quantum memories until a desired logical error rate is reached, and demonstrates that agents trained on one setting are able to successfully transfer their experience to different settings.

### Advances in quantum reinforcement learning

- Computer Science2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
- 2017

It is shown that quantum enhancements can be achieved in a new setting: the setting of learning models which learn how to improve themselves — that is, those that metalearn, and the results show that also such models can be quantum-enhanced to make even better learners.

### The theory of variational hybrid quantum-classical algorithms

- Computer Science
- 2015

This work develops a variational adiabatic ansatz and explores unitary coupled cluster where it is shown how the use of modern derivative free optimization techniques can offer dramatic computational savings of up to three orders of magnitude over previously used optimization techniques.