Acquisition of chess knowledge in AlphaZero

@article{McGrath2022AcquisitionOC,
  title={Acquisition of chess knowledge in AlphaZero},
  author={Thomas McGrath and Andrei Kapishnikov and Nenad Toma{\vs}ev and Adam Pearce and Demis Hassabis and Been Kim and Ulrich Paquet and Vladimir Kramnik},
  journal={Proceedings of the National Academy of Sciences of the United States of America},
  year={2022},
  volume={119}
}
Significance Seventy years ago, Alan Turing conjectured that a chess-playing machine could be built that would self-learn and continuously profit from its own experience. The AlphaZero system—a neural network-powered reinforcement learner—passed this milestone. In this paper, we ask the following questions. How did it do it? What did it learn from its experience, and how did it encode it? Did it learn anything like a human understanding of chess, in spite of having never seen a human game… 

Where, When & Which Concepts Does AlphaZero Learn? Lessons from the Game of Hex

This work investigates AlphaZero’s representations in Hex using both model probing and behavioral tests and finds that the MCTS search initially finds important concepts, and then the neural network learns to encode these concepts.

Evaluation Beyond Task Performance: Analyzing Concepts in AlphaZero in Hex

This work investigates AlphaZero’s internal representations in the game of Hex using two evaluation techniques from natural language processing (NLP): model probing and behavioral tests, and finds that MCTS discovers concepts before the neural network learns to encode them.

Learning Chess With Language Models and Transformers

BERT models are applied to the simple Nim game to analyze its performance in the presence of noise in a setup of a few-shot learning architecture and it is shown that model practically learns the rules of the chess game and can survive games against Stockfish at a category-A rating level.

L EARNING C HESS AND N IM WITH T RANSFORMERS

This study first applied the BERT model to the simple combinatorial Nim game to analyze BERT’s performance in the varying presence of noise, and applied it to the game of chess through a large set of high ELO stockfish games with exhaustive encyclopedia openings.

Reinforcement Learning in an Adaptable Chess Environment for Detecting Human-understandable Concepts

This work demonstrates a method for probing which concepts self-learning agents internalise in the course of their training and presents results for different chess board sizes, discussing the concepts learned by self-play learning chess agents in light of human domain knowledge.

Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task

Although the network has no a priori knowledge of the game or its rules, evidence is uncovered of an emergent nonlinear internal representation of the board state that can be used to control the output of the network and create “latent saliency maps” that help explain predictions.

Advancing Human Understanding with Deep Learning Go AI Engines

This work investigates the epistemological problem of using AI data for human understanding and suggests an active approach based on the scientific method that may provide solutions in other fields of knowledge, too, where AI technology outperforms humans.

Proceedings of the Annual Meeting of the Cognitive Science Society Comparing Machine and Human Learning in a Planning Task of Intermediate Complexity

Using a board game that strikes a balance between complexity and tractability, it is found that AlphaZero agents improve in value function quality and planning depth through learning, similar to human in previous modeling work.

L EARNING C HESS WITH L ANGUAGE M ODELS AND T RANSFORMERS

BERT models are applied to the simple Nim game to analyze its performance in the presence of noise in a setup of a few-shot learning architecture and it is shown that model practically learns the rules of the chess game and can survive games against Stockfish at a category-A rating level.

Understanding Game-Playing Agents with Natural Language Annotations

A new dataset containing 10K human-annotated games of Go is presented and it is found these game concepts are nontrivially encoded in two distinct policy networks, one trained via imitation learning and another trained via reinforcement learning.

References

SHOWING 1-10 OF 102 REFERENCES

Chess, a Drosophila of reasoning

The recent world chess championship saw Magnus Carlsen defend his title against Fabiano Caruana. But it was not a contest between the two strongest chess players on the planet, only the strongest

Where, When & Which Concepts Does AlphaZero Learn? Lessons from the Game of Hex

This work investigates AlphaZero’s representations in Hex using both model probing and behavioral tests and finds that the MCTS search initially finds important concepts, and then the neural network learns to encode these concepts.

Learning Personalized Models of Human Behavior in Chess

This work can significantly improve prediction of a particular player's moves by applying a series of fine-tuning adjustments, and can accurately perform stylometry -- predicting who made a given set of actions -- indicating that the authors' personalized models capture human decision-making at an individual level.

Aligning Superhuman AI with Human Behavior: Chess as a Model System

Maia is developed and introduced, a customized version of AlphaZero trained on human chess games, that predicts human moves at a much higher accuracy than existing engines, and can achieve maximum accuracy when predicting decisions made by players at a specific skill level in a tuneable way.

Learning to Play the Chess Variant Crazyhouse Above World Champion Level With Deep Neural Networks and Human Data

Improvements include modifications in the neural network design and training configuration, the introduction of a data normalization step and a more sample efficient Monte-Carlo tree search which has a lower chance to blunder.

A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

This paper generalizes the AlphaZero approach into a single AlphaZero algorithm that can achieve superhuman performance in many challenging games, and convincingly defeated a world champion program in the games of chess and shogi (Japanese chess), as well as Go.

Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess

An analytic comparison show that pieces are valued differently between variants, and that some variants are more decisive than classical chess, demonstrate the rich possibilities that lie beyond the rules of modern chess.

SentiMATE: Learning to play Chess through Natural Language Processing

The results exhibit strong evidence to support the initial hypothesis - "Can Natural Language Processing be used to train a novel and sample efficient evaluation function in Chess Engines?" - as the authors integrate the evaluation function into modern Chess engines and play against agents with traditional Chess move evaluation functions.

Chess Q & A : Question Answering on Chess Games

This work introduces a new dataset1 for the evaluation of models addressing reasoning tasks, and provides question and answer pairs, an image of the board, and the sequence of moves up to that position in a chess game.

Learning to Generate Move-by-Move Commentary for Chess Games from Large-Scale Social Forum Data

This paper introduces a new large-scale chess commentary dataset and proposes an end-to-end trainable neural model which takes into account multiple pragmatic aspects of the game state that may be commented upon to describe a given chess move.
...