Finding the ground state of spin Hamiltonians with reinforcement learning

@article{Mills2020FindingTG,
  title={Finding the ground state of spin Hamiltonians with reinforcement learning},
  author={Kyle Mills and Pooya Ronagh and Isaac Tamblyn},
  journal={Nature Machine Intelligence},
  year={2020},
  pages={1-9}
}
Reinforcement learning (RL) has become a proven method for optimizing a procedure for which success has been defined, but the specific actions needed to achieve it have not. Using a method we call ‘controlled online optimization learning’ (COOL), we apply the so-called ‘black box’ method of RL to simulated annealing (SA), demonstrating that an RL agent based on proximal policy optimization can, through experience alone, arrive at a temperature schedule that surpasses the performance of standard… 

Figures from this paper

Variational Neural Annealing
TLDR
It is shown that by generalizing the target distribution with a parameterized model, an analogous annealing framework based on the variational principle can be used to search for groundstate solutions in the asymptotic limit.
Machine learning for quantum matter
ABSTRACT Quantum matter, the research field studying phases of matter whose properties are intrinsically quantum mechanical, draws from areas as diverse as hard condensed matter physics, materials
Suppressing amplitude inhomogeneity enhances computational performance of Ising machines: why the choice of nonlinearity matters
TLDR
This work investigates if the shape of the system’s nonlinear transfer function can mitigate amplitude inhomogeneity and improve computational performance, and reports order-of-magnitude improvements in the time-to-solution compared to conventional polynomial models.
Deep Learning of Quantum Many-Body Dynamics via Random Driving
Machine learning based on neural networks (NN) [1, 2] has revealed a remarkable potential in solving a wide variety of complex problems. More recently, a number of first applications to quantum
Balancing Information with Observation Costs in Deep Reinforcement Learning
TLDR
This work makes the measurement costs explicit in the form of a costed reward and proposes the active-measure with costs framework that enables off-the-shelf deep RL algorithms to learn a policy for both selecting actions and determining whether or not to measure the state of the system at each time step.
CITS: Coherent Ising Tree Search Algorithm Towards Solving Combinatorial Optimization Problems
TLDR
This paper proposes a heuristic search algorithm by expanding search space from a Markov chain to a recursive depth limited tree based on SA, where the parent and child nodes represent the current and future spin states and shows that this high-level tree search strategy is able to provide solutions within fewer epochs for Ising formulated NP-optimization problems.
Combinatorial Optimization with Physics-Inspired Graph Neural Networks
TLDR
The graph neural network optimizer performs on par or outperforms existing solvers, with the ability to scale beyond the state of the art to problems with millions of variables.
Deep learning and crystal plasticity: A preconditioning approach for accurate orientation evolution prediction
Deep learning enhanced Rydberg multifrequency microwave recognition
Recognition of multifrequency microwave (MW) electric fields is challenging because of the complex interference of multifrequency fields in practical applications. Rydberg atom-based measurements for
Machine Learning in Materials Chemistry: An Invitation
...
1
2
...

References

SHOWING 1-10 OF 101 REFERENCES
Solving the Rubik’s cube with deep reinforcement learning and search
TLDR
A new deep learning based search heuristic performs well on the iconic Rubik’s cube and can also generalize to puzzles in which optimal solvers are intractable.
Reinforcement Learning: An Introduction
TLDR
This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.
Quantum annealing with manufactured spins
TLDR
This programmable artificial spin network bridges the gap between the theoretical study of ideal isolated spin networks and the experimental investigation of bulk magnetic samples, and may provide a practical physical means to implement a quantum algorithm, possibly allowing more-effective approaches to solving certain classes of hard combinatorial optimization problems.
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
TLDR
This paper generalizes the AlphaZero approach into a single AlphaZero algorithm that can achieve superhuman performance in many challenging games, and convincingly defeated a world champion program in the games of chess and shogi (Japanese chess), as well as Go.
Mastering the game of Go with deep neural networks and tree search
TLDR
Using this search algorithm, the program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0.5, the first time that a computer program has defeated a human professional player in the full-sized game of Go.
Trust Region Policy Optimization
TLDR
A method for optimizing control policies, with guaranteed monotonic improvement, by making several approximations to the theoretically-justified scheme, called Trust Region Policy Optimization (TRPO).
Grandmaster level in StarCraft II using multi-agent reinforcement learning
TLDR
The agent, AlphaStar, is evaluated, which uses a multi-agent reinforcement learning algorithm and has reached Grandmaster level, ranking among the top 0.2% of human players for the real-time strategy game StarCraft II.
Hierarchical Reinforcement Learning for Multi-agent MOBA Game
TLDR
A novel hierarchical reinforcement learning model for mastering Multiplayer Online Battle Arena (MOBA) games, a sub-genre of RTS games, and designing a dense reward function for multi-agent cooperation in the absence of game engine or Application Programming Interface (API).
Temperature schedules for simulated annealing
TLDR
In this paper, a dynamic programming approach is used to find the temperature schedule which is optimal for a simple minimization problem, and the optimal schedule is compared with certain standard non-optimal choices.
Quantum versus classical annealing of Ising spin glasses
TLDR
The question of when quantum speedup may be expected is revisited and it is shown that the results depend on how the problem is described and how the optimization routine is implemented, which implies that care must be taken when using QMC simulations to assess the potential for quantum speed up.
...
1
2
3
4
5
...