# Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning

@article{Skolik2022QuantumAI, title={Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning}, author={Andrea Skolik and Sofi{\`e}ne Jerbi and Vedran Dunjko}, journal={Quantum}, year={2022}, volume={6}, pages={720} }

Quantum machine learning (QML) has been identiﬁed as one of the key ﬁelds that could reap advantages from near-term quantum devices, next to optimization and quantum chemistry. Research in this area has focused primarily on variational quantum algorithms (VQAs), and several proposals to enhance supervised, unsupervised and reinforcement learning (RL) algorithms with VQAs have been put forward. Out of the three, RL is the least studied and it is still an open question whether VQAs can be…

## Figures and Tables from this paper

## 32 Citations

### Uncovering Instabilities in Variational-Quantum Deep Q-Networks

- Computer ScienceJournal of the Franklin Institute
- 2022

### Parametrized Quantum Policies for Reinforcement Learning

- Computer ScienceNeurIPS
- 2021

This work proposes a hybrid quantum-classical reinforcement learning model using very few qubits, which it is shown can be effectively trained to solve several standard benchmarking environments and formally proves the ability of parametrized quantum circuits to solve certain learning tasks that are intractable to classical models.

### Quantum Architecture Search via Deep Reinforcement Learning

- Computer Science, PhysicsArXiv
- 2021

A quantum architecture search framework with the power of deep reinforcement learning (DRL) to address the challenge of generation of quantum gate sequences for multiqubit GHZ states without encoding any knowledge of quantum physics in the agent.

### Evolutionary quantum architecture search for parametrized quantum circuits

- Computer ScienceGECCO Companion
- 2022

EQAS-PQC is introduced, an evolutionary quantum architecture search framework for PQC-based models, which uses a population-based genetic algorithm to evolve PZC architectures by exploring the search space of quantum operations.

### Variational quantum reinforcement learning via evolutionary optimization

- Computer Science, PhysicsMach. Learn. Sci. Technol.
- 2022

A hybrid framework is proposed where the quantum RL agents are equipped with a hybrid tensor network-variational quantum circuit (TN-VQC) architecture to handle inputs of dimensions exceeding the number of qubits, enabling further quantum RL applications on noisy intermediate-scale quantum devices.

### Variational Quantum Soft Actor-Critic

- Computer ScienceArXiv
- 2021

This work develops a quantum reinforcement learning algorithm based on soft actor-critic, a hybrid quantum-classical policy network consisting of a variational quantum circuit and a classical artificial neural network, and analyzes the effect of different hyper-parameters and policy network architectures.

### Quantum Architecture Search via Continual Reinforcement Learning

- Computer ScienceArXiv
- 2021

The Probabilistic Policy Reuse with deep Q-learning (PPR-DQL) framework is presented and it is demonstrated that the RL agent with PPR was able to find the quantum gate sequence to generate the two-qubit Bell state faster than the agent that was trained from scratch.

### Photonic Quantum Policy Learning in OpenAI Gym

- Computer Science2021 IEEE International Conference on Quantum Computing and Engineering (QCE)
- 2021

This work introduces proximal policy optimization for photonic variational quantum agents and the effect of the data re-uploading to solve a classical continuous control problem using continuous-variable quantum machine learning.

### Learning capability of parametrized quantum circuits

- Computer Science
- 2022

This paper builds upon the work by Schuld et and compares popular ansätze for PQCs through the new measure of learning capability and proposes a data re-upload structure for dQNNs to increase their learning capability.

### Playing Atari with Hybrid Quantum-Classical Reinforcement Learning

- Computer SciencePreregister@NeurIPS
- 2020

A neural network is proposed as a data encoder for quantum reinforcement learning that converts pixel input from Atari games to quantum data for a Quantum Variational Circuit (QVC) and is then used as a function approximator in the Double Deep Q Networks algorithm.

## References

SHOWING 1-10 OF 64 REFERENCES

### Variational Quantum Circuits for Deep Reinforcement Learning

- Computer ScienceIEEE Access
- 2020

This work reshapes classical deep reinforcement learning algorithms like experience replay and target network into a representation of variational quantum circuits, and uses a quantum information encoding scheme to reduce the number of model parameters compared to classical neural networks.

### Parametrized Quantum Policies for Reinforcement Learning

- Computer ScienceNeurIPS
- 2021

This work proposes a hybrid quantum-classical reinforcement learning model using very few qubits, which it is shown can be effectively trained to solve several standard benchmarking environments and formally proves the ability of parametrized quantum circuits to solve certain learning tasks that are intractable to classical models.

### Quantum Enhancements for Deep Reinforcement Learning in Large Spaces

- Computer Science
- 2020

This work studies the state-of-the-art neural-network approaches for reinforcement learning with quantum enhancements in mind and demonstrates the substantial learning advantage that models with a sampling bottleneck can provide over conventional neural network architectures in complex learning environments.

### Variational quantum policies for reinforcement learning

- Computer ScienceArXiv
- 2021

This work investigates how to construct and train reinforcement learning policies based on variational quantum circuits, and proposes and shows the existence of task environments with a provable separation in performance between quantum learning agents and any polynomial-time classical learner.

### Exponential improvements for quantum-accessible reinforcement learning

- Computer Science
- 2017

The results suggest that quantum agents may perform well in certain game-playing scenarios, where the game has recursive structure, and the agent can learn by playing against itself.

### TensorFlow Quantum: A Software Framework for Quantum Machine Learning

- Computer Science, PhysicsArXiv
- 2020

This framework offers high-level abstractions for the design and training of both discriminative and generative quantum models under TensorFlow and supports high-performance quantum circuit simulators.

### Quantum reinforcement learning in continuous action space

- Computer ScienceArXiv
- 2020

This work proposes an alternative quantum circuit design that can solve RL problems in continuous action space without the dimensionality problem and demonstrates that quantum control tasks, including the eigenvalue problem and quantum state transfer, can be formulated as sequential decision problems and solved by this method.

### Quantum-accessible reinforcement learning beyond strictly epochal environments

- Computer ScienceQuantum Mach. Intell.
- 2021

This work considers one of the first generalizations of quantum-accessible reinforcement learning, where the environment is not strictly episodic, which is mapped to an oracle identification setting with a changing oracle.

### Adaptive pruning-based optimization of parameterized quantum circuits

- Computer Science
- 2020

PECT can enable optimizations of certain ansatze that were previously difficult to converge and more generally can improve the performance of variational algorithms by reducing the optimization runtime and/or the depth of circuits that encode the solution candidate(s).

### Classification with Quantum Neural Networks on Near Term Processors

- Physics, Computer Science
- 2018

This work introduces a quantum neural network, QNN, that can represent labeled data, classical or quantum, and be trained by supervised learning, and shows through classical simulation that parameters can be found that allow the QNN to learn to correctly distinguish the two data sets.