# Model-Free Quantum Control with Reinforcement Learning

@article{Sivak2022ModelFreeQC, title={Model-Free Quantum Control with Reinforcement Learning}, author={V. V. Sivak and A. Eickbusch and H. Liu and Baptiste Royer and Ioannis Tsioutsios and Michel H. Devoret}, journal={Physical Review X}, year={2022} }

Model bias is an inherent limitation of the current dominant approach to optimal quantum control, which relies on a system simulation for optimization of control policies. To overcome this limitation, we propose a circuit-based approach for training a reinforcement learning agent on quantum control tasks in a model-free way. Given a continuously parameterized control circuit, the agent learns its parameters through trial-and-error interaction with the quantum system, using measurement outcomes…

## Figures from this paper

## 14 Citations

### Gradient Ascent Pulse Engineering with Feedback

- Computer Science
- 2022

Efficient approaches to quantum control and feedback are essential for quantum technologies, from sensing to quantum computation. Pure control tasks have been successfully solved using optimization…

### Deep Reinforcement Learning for Quantum State Preparation with Weak Nonlinear Measurements

- PhysicsQuantum
- 2022

Quantum control has been of increasing interest in recent years, e.g. for tasks like state initialization and stabilization. Feedback-based strategies are particularly powerful, but also hard to…

### A tutorial on optimal control and reinforcement learning methods for quantum technologies

- Computer SciencePhysics Letters A
- 2022

### Variational quantum reinforcement learning via evolutionary optimization

- Computer Science, PhysicsMach. Learn. Sci. Technol.
- 2022

A hybrid framework is proposed where the quantum RL agents are equipped with a hybrid tensor network-variational quantum circuit (TN-VQC) architecture to handle inputs of dimensions exceeding the number of qubits, enabling further quantum RL applications on noisy intermediate-scale quantum devices.

### Variational Quantum Pulse Learning

- Computer Science
- 2022

Inspired by the promising performance of VQC, this paper proposes variational quantum pulses (VQP), a novel paradigm to directly train quantum pulses for learning tasks and maintains the robustness to noise on Noisy Intermediate-Scale Quantum (NISQ) computers.

### Learning Quantum Systems

- Physics
- 2022

Quantum technologies hold the promise to revolutionise our society with ground-breaking applications in secure communication, high-performance computing and ultra-precise sensing. One of the main…

### Reinforcement learning-enhanced protocols for coherent population-transfer in three-level quantum systems

- Computer ScienceNew Journal of Physics
- 2021

This work deploys a combination of reinforcement learning-based approaches and more traditional optimization techniques to identify optimal protocols for population transfer in a multi-level system and reveals the existence of efficient protocols that, remarkably, differ from (and can be superior to) standard Raman, stimulated Raman adiabatic passage or other adiABatic schemes.

### Deep reinforcement learning for preparation of thermal and prethermal quantum states

- Physics
- 2022

We propose a method based on deep reinforcement learning that eﬃciently prepares a quantum many-body pure state in thermal or prethermal equilibrium. The main physical intuition underlying the method…

### Fast Universal Control of an Oscillator with Weak Dispersive Coupling to a Qubit

- Physics
- 2021

A controlled evolution generated by nonlinear interactions is required to perform full manipulation of a quantum system, and such control is only coherent when the rate of nonlinearity is large…

### Quantum optimal control in quantum technologies. Strategic report on current status, visions and goals for research in Europe

- PhysicsEPJ Quantum Technology
- 2022

Quantum optimal control, a toolbox for devising and implementing the shapes of external fields that accomplish given tasks in the operation of a quantum device in the best way possible, has evolved…

## References

SHOWING 1-10 OF 96 REFERENCES

### Reinforcement Learning in Different Phases of Quantum Control

- Physics, Computer SciencePhysical Review X
- 2018

This work implements cutting-edge Reinforcement Learning techniques and shows that their performance is comparable to optimal control methods in the task of finding short, high-fidelity driving protocol from an initial to a target state in non-integrable many-body quantum systems of interacting qubits.

### Quantum Observables for continuous control of the Quantum Approximate Optimization Algorithm via Reinforcement Learning

- Computer ScienceArXiv
- 2019

This work presents a classical control mechanism for Quantum devices using Reinforcement Learning that provides optimal control of the Quantum device following a reformulation of QAOA as an environment where an autonomous classical agent interacts and performs actions to achieve higher rewards.

### Universal quantum control through deep reinforcement learning

- Computer Sciencenpj Quantum Information
- 2019

This work improves the control robustness of a broad family of two-qubit unitary gates that are important for quantum simulation of many-electron systems by adding control noise into training environments for reinforcement learning agents trained with trusted-region-policy-optimization.

### Reinforcement Learning with Neural Networks for Quantum Feedback

- Computer SciencePhysical Review X
- 2018

This work shows how a network-based "agent" can discover complete quantum-error-correction strategies, protecting a collection of qubits against noise, and develops two ideas: two-stage learning with teacher/student networks and a reward quantifying the capability to recover the quantum information stored in a multi-qubit system.

### Optimizing Quantum Error Correction Codes with Reinforcement Learning

- Computer ScienceQuantum
- 2019

This work considers a reinforcement learning agent tasked with modifying a family of surface code quantum memories until a desired logical error rate is reached, and demonstrates that agents trained on one setting are able to successfully transfer their experience to different settings.

### Deep reinforcement learning for quantum gate control

- Computer ScienceEPL (Europhysics Letters)
- 2019

A dueling double deep Q-learning neural network is constructed to find out the optimized time dependence of controllable parameters to implement two typical quantum gates: a single-qu bit Hadamard gate and a two-qubit CNOT gate.

### When does reinforcement learning stand out in quantum control? A comparative study on state preparation

- Computer Sciencenpj Quantum Information
- 2019

A comparative study on the efficacy of three reinforcement learning algorithms: tabular Q- learning, deep Q-learning, and policy gradient, as well as two non-machine-learning methods: stochastic gradient descent and Krotov algorithms, in the problem of preparing a desired quantum state is performed.

### Experimental Deep Reinforcement Learning for Error-Robust Gate-Set Design on a Superconducting Quantum Computer

- Computer SciencePRX Quantum
- 2021

This work experimentally demonstrates that a fully autonomous deep reinforcement learning agent can design single qubit gates up to 3× faster than default DRAG operations without additional leakage error, and shows robustness against calibration drifts over weeks, and benchmark the performance ofDeep reinforcement learning derived gates against other black box optimization techniques, showing that deep reinforcementLearning can achieve comparable or marginally superior performance, even with limited hardware access.

### Reinforcement learning for autonomous preparation of Floquet-engineered states: Inverting the quantum Kapitza oscillator

- PhysicsPhysical Review B
- 2018

I demonstrate the potential of reinforcement learning (RL) to prepare quantum states of strongly periodically driven non-linear single-particle models. The ability of Q-Learning to control systems…

### Generalizable control for quantum parameter estimation through reinforcement learning

- Computer Sciencenpj Quantum Information
- 2019

It is demonstrated that reinforcement learning provides an efficient way to identify the controls that can be employed to improve the precision and is highly generalizable, namely the neural network trained under one particular value of the parameter can work for different values within a broad range.