# Reinforcement Learning in Video Games Using Nearest Neighbor Interpolation and Metric Learning

@article{Emigh2016ReinforcementLI, title={Reinforcement Learning in Video Games Using Nearest Neighbor Interpolation and Metric Learning}, author={Matthew S. Emigh and Evan Kriminger and Austin J. Brockmeier and Jos{\'e} Carlos Pr{\'i}ncipe and Panos M. Pardalos}, journal={IEEE Transactions on Computational Intelligence and AI in Games}, year={2016}, volume={8}, pages={56-66} }

Reinforcement learning (RL) has had mixed success when applied to games. Large state spaces and the curse of dimensionality have limited the ability for RL techniques to learn to play complex games in a reasonable length of time. We discuss a modification of Q-learning to use nearest neighbor states to exploit previous experience in the early stages of learning. A weighting on the state features is learned using metric learning techniques, such that neighboring states represent similar game…

## 21 Citations

### Perceptron Q-learning Applied to Super Smash Bros Melee

- Computer Science
- 2018

A reinforcement learning strategy for Super Smash Bros Melee, a video gamewith a continuous state and action space by applying global approximation in the Q-learning algorithm is discussed.

### Modular Reinforcement Learning Framework for Learners and Educators

- Computer ScienceFDG
- 2021

The EasyGameRL framework is proposed, a novel approach to the education of reinforcement learning in games using modular visual design patterns, and its software implementation in Unreal Engine are modular, reusable, and applicable to multiple game scenarios.

### Theoretically Principled Deep RL Acceleration via Nearest Neighbor Function Approximation

- Computer ScienceAAAI
- 2021

This work presents Nearest Neighbor Actor-Critic (NNAC), an online policy gradient algorithm that demonstrates the practicality of combining function approximation with deep RL, and a plug-and-play NN update module that aids the training of existing deep RL methods.

### Improving Reinforcement Learning Results with Qualitative Spatial Representation

- Computer Science2017 Brazilian Conference on Intelligent Systems (BRACIS)
- 2017

A hybrid method that uses a qualitative formalism with reinforcement learning, named QRL, and is able to get better results than traditional methods is applied in the robot soccer domain and the results show that the agent can learn optimal policies and perform more goals than quantitative representation.

### Learning to Navigate Through Complex Dynamic Environment With Modular Deep Reinforcement Learning

- Computer ScienceIEEE Transactions on Games
- 2018

The proposed end-to-end modular reinforcement learning architecture for a navigation task in complex dynamic environments with rapidly moving obstacles can efficiently avoid moving obstacles and complete the navigation task at a high success rate.

### Zombies Arena: fusion of reinforcement learning with augmented reality on NPC

- Computer ScienceCluster Computing
- 2017

The objective is to develop AR based first person shooter game empowering reinforcement learning that will act as a building block to capacitate users to interact with the physical environment.

### Zombies Arena: fusion of reinforcement learning with augmented reality on NPC

- Computer ScienceCluster Computing
- 2017

The objective is to develop AR based first person shooter game empowering reinforcement learning that will act as a building block to capacitate users to interact with the physical environment.

### Analysis of Agent Expertise in Ms. Pac-Man Using Value-of-Information-Based Policies

- Computer ScienceIEEE Transactions on Games
- 2019

An information-theoretic cost function for performing constrained stochastic searches that promote the formation of risk-averse to risk-favoring behaviors is considered, which implies that the value-of-information theory is appropriate for framing the exploitation–exploration tradeoff in reinforcement learning.

### Let's Do the Time Warp Again: Human Action Assistance for Reinforcement Learning Agents

- Computer ScienceICAART
- 2021

A system called Time Warp is introduced that allows a human teacher to provide action selection assistance to the agent during critical moments of the training for the RL agent and rivals the performance of computer teaching agents.

### Narrow Artificial Intelligence with Machine Learning for Real-Time Estimation of a Mobile Agents Location Using Hidden Markov Models

- Computer ScienceInt. J. Comput. Games Technol.
- 2017

This work proposes to use a supervised machine learning technique to track the location of a mobile agent in real time and uses the BaumźWelch algorithm as a statistical learning tool to gain knowledge of the mobile target.

## References

SHOWING 1-10 OF 39 REFERENCES

### Playing Atari with Deep Reinforcement Learning

- Computer ScienceArXiv
- 2013

This work presents the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning, which outperforms all previous approaches on six of the games and surpasses a human expert on three of them.

### Metric Learning for Invariant Feature Generation in Reinforcement Learning

- Computer Science
- 2013

This work proposes a method to automatically map states to feature vectors, which are not only sufficiently descriptive of the environment, but are invariant to information not relevant to the agent’s goal.

### Kernel-Based Reinforcement Learning

- Computer ScienceEncyclopedia of Machine Learning and Data Mining
- 2017

A kernel-based approach to reinforcement learning that overcomes the stability problems of temporal-difference learning in continuous state-spaces and shows that the limiting distribution of the value function estimate is a Gaussian process.

### CBR for State Value Function Approximation in Reinforcement Learning

- Computer ScienceICCBR
- 2005

This paper investigates the use of case-based methods to realise the task of approximating a function over high-dimensional, continuous spaces and examines the approach taken in robotic soccer simulation.

### Learning to Play Chess Using Temporal Differences

- Computer ScienceMachine Learning
- 2004

TDLEAF(λ), a variation on the TD(λ) algorithm that enables it to be used in conjunction with game-tree search, is presented and it is investigated whether it can yield better results in the domain of backgammon, where TD( ε) has previously yielded striking success.

### Reinforcement Learning: An Introduction

- Computer ScienceIEEE Transactions on Neural Networks
- 2005

This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.

### Using Imagery to Simplify Perceptual Abstraction in Reinforcement Learning Agents

- PsychologyAAAI
- 2010

This paper considers the problem of reinforcement learning in spatial tasks, and shows that this integration with an imagery system broadens the class of perceptual abstraction methods that can be used while preserving the underlying problem.

### Covert Perceptual Capability Development

- Computer Science
- 2005

A model to develop robots’ covert perceptual capability using reinforcement learning, where covert perceptual behavior is treated as action selected by a motivational system, and K Nearest-Neighbor strategy is adopted to reduce training time complexity.

### TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play

- Computer ScienceNeural Computation
- 1994

The latest version of TD-Gammon is now estimated to play at a strong master level that is extremely close to the world's best human players.