# Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning

@inproceedings{Skolik2021QuantumAI, title={Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning}, author={Andrea Skolik and Sofi{\`e}ne Jerbi and Vedran Dunjko}, year={2021} }

Quantum machine learning (QML) has been identiﬁed as one of the key ﬁelds that could reap advantages from near-term quantum devices, next to optimization and quantum chemistry. Research in this area has focused primarily on variational quantum algorithms (VQAs), and several proposals to enhance supervised, unsupervised and reinforcement learning (RL) algorithms with VQAs have been put forward. Out of the three, RL is the least studied and it is still an open question whether VQAs can be…

## Figures and Tables from this paper

## 23 Citations

Uncovering Instabilities in Variational-Quantum Deep Q-Networks

- Computer ScienceArXiv
- 2022

It cannot be conclusively decided if known quantum approaches, even if simulated without physical imperfections, can provide an advantage as compared to classical approaches, and this work provides a robust, universal and well-tested implementation of VQ-DQN as a reproducible testbed for future experiments.

Parametrized Quantum Policies for Reinforcement Learning

- Computer ScienceNeurIPS
- 2021

This work proposes a hybrid quantum-classical reinforcement learning model using very few qubits, which it is shown can be effectively trained to solve several standard benchmarking environments and formally proves the ability of parametrized quantum circuits to solve certain learning tasks that are intractable to classical models.

Variational quantum reinforcement learning via evolutionary optimization

- Computer Science, PhysicsMachine Learning: Science and Technology
- 2021

A hybrid framework is proposed where the quantum RL agents are equipped with a hybrid tensor network-variational quantum circuit (TN-VQC) architecture to handle inputs of dimensions exceeding the number of qubits, enabling further quantum RL applications on noisy intermediate-scale quantum devices.

Variational Quantum Soft Actor-Critic

- Computer ScienceArXiv
- 2021

This work develops a quantum reinforcement learning algorithm based on soft actor-critic, a hybrid quantum-classical policy network consisting of a variational quantum circuit and a classical artificial neural network, and analyzes the effect of different hyper-parameters and policy network architectures.

Quantum Architecture Search via Continual Reinforcement Learning

- Computer ScienceArXiv
- 2021

The Probabilistic Policy Reuse with deep Q-learning (PPR-DQL) framework is presented and it is demonstrated that the RL agent with PPR was able to find the quantum gate sequence to generate the two-qubit Bell state faster than the agent that was trained from scratch.

Photonic Quantum Policy Learning in OpenAI Gym

- Computer Science2021 IEEE International Conference on Quantum Computing and Engineering (QCE)
- 2021

This work introduces proximal policy optimization for photonic variational quantum agents and the effect of the data re-uploading to solve a classical continuous control problem using continuous-variable quantum machine learning.

Playing Atari with Hybrid Quantum-Classical Reinforcement Learning

- Computer SciencePreregister@NeurIPS
- 2020

A neural network is proposed as a data encoder for quantum reinforcement learning that converts pixel input from Atari games to quantum data for a Quantum Variational Circuit (QVC) and is then used as a function approximator in the Double Deep Q Networks algorithm.

TensorFlow Quantum: A Software Framework for Quantum Machine Learning

- Computer Science, PhysicsArXiv
- 2020

This framework offers high-level abstractions for the design and training of both discriminative and generative quantum models under TensorFlow and supports high-performance quantum circuit simulators.

An Empirical Review of Optimization Techniques for Quantum Variational Circuits

- Computer Science, PhysicsArXiv
- 2022

A large number of problems and optimizers evaluated yields strong empirical guidance for choosing optimizers for QVCs that is currently lacking, and includes both classical and quantum data based optimization routines.

Quantum machine learning beyond kernel methods

- Computer ScienceArXiv
- 2021

This work identifies the first unifying framework that captures all standard models based on parametrized quantum circuits: that oflinear quantum models, and shows how data re-uploading circuits, a generalization of linear models, can be efficiently mapped into equivalent linear quantum models.

## References

SHOWING 1-10 OF 70 REFERENCES

Variational Quantum Circuits for Deep Reinforcement Learning

- Computer ScienceIEEE Access
- 2020

This work reshapes classical deep reinforcement learning algorithms like experience replay and target network into a representation of variational quantum circuits, and uses a quantum information encoding scheme to reduce the number of model parameters compared to classical neural networks.

Reinforcement Learning with Quantum Variational Circuits

- Computer Science, PhysicsAAAI 2020
- 2020

Results indicate both hybrid and pure quantum variational circuit have the ability to solve reinforcement learning tasks with a smaller parameter space.

TensorFlow Quantum: A Software Framework for Quantum Machine Learning

- Computer Science, PhysicsArXiv
- 2020

This framework offers high-level abstractions for the design and training of both discriminative and generative quantum models under TensorFlow and supports high-performance quantum circuit simulators.

Quantum reinforcement learning in continuous action space

- Computer ScienceArXiv
- 2020

This work proposes an alternative quantum circuit design that can solve RL problems in continuous action space without the dimensionality problem and demonstrates that quantum control tasks, including the eigenvalue problem and quantum state transfer, can be formulated as sequential decision problems and solved by this method.

Quantum-accessible reinforcement learning beyond strictly epochal environments

- Computer ScienceQuantum Mach. Intell.
- 2021

This work considers one of the first generalizations of quantum-accessible reinforcement learning, where the environment is not strictly episodic, which is mapped to an oracle identification setting with a changing oracle.

Barren plateaus in quantum neural network training landscapes

- Computer Science, PhysicsNature Communications
- 2018

It is shown that for a wide class of reasonable parameterized quantum circuits, the probability that the gradient along any reasonable direction is non-zero to some fixed precision is exponentially small as a function of the number of qubits.

Supervised learning with quantum-enhanced feature spaces

- Computer Science, PhysicsNature
- 2019

Two classification algorithms that use the quantum state space to produce feature maps are demonstrated on a superconducting processor, enabling the solution of problems when the feature space is large and the kernel functions are computationally expensive to estimate.

The Born supremacy: quantum advantage and training of an Ising Born machine

- Computer ScienceArXiv
- 2019

This work defines a subset of a class of quantum circuits known as Born machines based on Ising Hamiltonians and shows that the circuits encountered during gradient-based training cannot be efficiently sampled from classically up to multiplicative error in the worst case.

Quantum Wasserstein Generative Adversarial Networks

- Computer Science, PhysicsNeurIPS
- 2019

This work proposes the first design of quantum Wasserstein Generative Adversarial Networks (WGANs), which has been shown to improve the robustness and the scalability of the adversarial training of quantum generative models even on noisy quantum hardware.

Quantum Generative Adversarial Networks for learning and loading random distributions

- Computer Science, Physicsnpj Quantum Information
- 2019

This work uses quantum Generative Adversarial Networks (qGANs) to facilitate efficient learning and loading of generic probability distributions - implicitly given by data samples - into quantum states and can enable the use of potentially advantageous quantum algorithms, such as Quantum Amplitude Estimation.