# Model-Free Quantum Control with Reinforcement Learning

@article{Sivak2021ModelFreeQC, title={Model-Free Quantum Control with Reinforcement Learning}, author={V. V. Sivak and A. Eickbusch and H. Liu and Baptiste Royer and Ioannis Tsioutsios and Michel H. Devoret}, journal={Physical Review X}, year={2021} }

Model bias is an inherent limitation of the current dominant approach to optimal quantum control, which relies on a system simulation for optimization of control policies. To overcome this limitation, we propose a circuit-based approach for training a reinforcement learning agent on quantum control tasks in a model-free way. Given a continuously parametrized control circuit, the agent learns its parameters through trial-and-error interaction with the quantum system, using measurement outcomes…

## 20 Citations

### Realizing a deep reinforcement learning agent discovering real-time feedback control strategies for a quantum system

- Computer Science
- 2022

This work implements a deep reinforcement learning agent in the form of a latency-optimized deep neural network on a FPGA that is used to initialize a superconducting qubit into a target state and uses model-free reinforcement learning that is based solely on measurement data.

### Gradient Ascent Pulse Engineering with Feedback

- Computer Science
- 2022

Efficient approaches to quantum control and feedback are essential for quantum technologies, from sensing to quantum computation. Pure control tasks have been successfully solved using optimization…

### Model-Free Deep Recurrent Q-Network Reinforcement Learning for Quantum Circuit Architectures Design

- Computer ScienceQuantum Reports
- 2022

This paper is the first attempt to solve the quantum circuit design problem from the recurrent reinforcement learning algorithm, while using discrete policy and simulation results suggest that the long short-term memory (LSTM)-based DRQN method is able to learn quantum circuits for entangled Bell–Greenberger–Horne–Zeilinger (Bell–GHZ) states.

### A tutorial on optimal control and reinforcement learning methods for quantum technologies

- Computer SciencePhysics Letters A
- 2022

### Variational Quantum Pulse Learning

- Computer Science2022 IEEE International Conference on Quantum Computing and Engineering (QCE)
- 2022

Inspired by the promising performance of VQC, this paper proposes variational quantum pulses (VQP), a novel paradigm to directly train quantum pulses for learning tasks and maintains the robustness to noise on Noisy Intermediate-Scale Quantum (NISQ) computers.

### Deep reinforcement learning for preparation of thermal and prethermal quantum states

- Physics
- 2022

We propose a method based on deep reinforcement learning that eﬃciently prepares a quantum many-body pure state in thermal or prethermal equilibrium. The main physical intuition underlying the method…

### Learning Quantum Systems

- Physics
- 2022

Quantum technologies hold the promise to revolutionise our society with ground-breaking applications in secure communication, high-performance computing and ultra-precise sensing. One of the main…

### Reinforcement learning-enhanced protocols for coherent population-transfer in three-level quantum systems

- Computer ScienceNew Journal of Physics
- 2021

This work deploys a combination of reinforcement learning-based approaches and more traditional optimization techniques to identify optimal protocols for population transfer in a multi-level system and reveals the existence of efficient protocols that, remarkably, differ from (and can be superior to) standard Raman, stimulated Raman adiabatic passage or other adiABatic schemes.

### Fast universal control of an oscillator with weak dispersive coupling to a qubit

- PhysicsNature Physics
- 2022

and Full manipulation of a quantum system requires controlled evolution generated by nonlinear interactions, which is coherent when the rate of nonlinearity is large compared with the rate of…

### Optimizing measurement-based cooling by reinforcement learning

- PhysicsPhysical Review A
- 2022

Conditional cooling-by-measurement holds a signiﬁcant advantage over its unconditional (nons-elective) counterpart in the average-population-reduction rate. However, it has a clear weakness with…

## References

SHOWING 1-10 OF 107 REFERENCES

### Measurement-Based Feedback Quantum Control with Deep Reinforcement Learning for a Double-Well Nonlinear Potential.

- Computer SciencePhysical review letters
- 2021

This work applies deep reinforcement learning (DRL), where an artificial neural agent explores and learns to control the quantum evolution of a highly nonlinear system (double well), driving the system toward the ground state with high fidelity.

### Reinforcement Learning in Different Phases of Quantum Control

- Physics, Computer SciencePhysical Review X
- 2018

This work implements cutting-edge Reinforcement Learning techniques and shows that their performance is comparable to optimal control methods in the task of finding short, high-fidelity driving protocol from an initial to a target state in non-integrable many-body quantum systems of interacting qubits.

### Quantum Observables for continuous control of the Quantum Approximate Optimization Algorithm via Reinforcement Learning

- Computer ScienceArXiv
- 2019

This work presents a classical control mechanism for Quantum devices using Reinforcement Learning that provides optimal control of the Quantum device following a reformulation of QAOA as an environment where an autonomous classical agent interacts and performs actions to achieve higher rewards.

### Universal quantum control through deep reinforcement learning

- Computer Sciencenpj Quantum Information
- 2019

This work improves the control robustness of a broad family of two-qubit unitary gates that are important for quantum simulation of many-electron systems by adding control noise into training environments for reinforcement learning agents trained with trusted-region-policy-optimization.

### Optimizing Quantum Error Correction Codes with Reinforcement Learning

- Computer ScienceQuantum
- 2019

This work considers a reinforcement learning agent tasked with modifying a family of surface code quantum memories until a desired logical error rate is reached, and demonstrates that agents trained on one setting are able to successfully transfer their experience to different settings.

### Reinforcement Learning with Neural Networks for Quantum Feedback

- Computer SciencePhysical Review X
- 2018

This work shows how a network-based "agent" can discover complete quantum-error-correction strategies, protecting a collection of qubits against noise, and develops two ideas: two-stage learning with teacher/student networks and a reward quantifying the capability to recover the quantum information stored in a multi-qubit system.

### When does reinforcement learning stand out in quantum control? A comparative study on state preparation

- Computer Sciencenpj Quantum Information
- 2019

A comparative study on the efficacy of three reinforcement learning algorithms: tabular Q- learning, deep Q-learning, and policy gradient, as well as two non-machine-learning methods: stochastic gradient descent and Krotov algorithms, in the problem of preparing a desired quantum state is performed.

### Reinforcement-learning-assisted quantum optimization

- Computer Science
- 2020

It is shown that the proposed RL scheme finds a policy converging to the optimal adiabatic solution for QAOA found by Mbeng et al. arXiv:1906.08948 for the translationally invariant quantum Ising chain.

### Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator

- PhysicsPhysical Review B
- 2018

I demonstrate the potential of reinforcement learning (RL) to prepare quantum states of strongly periodically driven non-linear single-particle models. The ability of Q-Learning to control systems…

### Quantum optimal control of multilevel dissipative quantum systems with reinforcement learning

- Computer Science
- 2021

This work proposes a multilevel dissipative quantum control framework and shows that deep reinforcement learning provides an efficient way to identify the optimal strategies with restricted control parameters of the complex quantum system.