Reinforcement Learning in Video Games Using Nearest Neighbor Interpolation and Metric Learning

@article{Emigh2016ReinforcementLI,
  title={Reinforcement Learning in Video Games Using Nearest Neighbor Interpolation and Metric Learning},
  author={Matthew S. Emigh and Evan Kriminger and Austin J. Brockmeier and Jos{\'e} Carlos Pr{\'i}ncipe and Panos M. Pardalos},
  journal={IEEE Transactions on Computational Intelligence and AI in Games},
  year={2016},
  volume={8},
  pages={56-66}
}
Reinforcement learning (RL) has had mixed success when applied to games. Large state spaces and the curse of dimensionality have limited the ability for RL techniques to learn to play complex games in a reasonable length of time. We discuss a modification of Q-learning to use nearest neighbor states to exploit previous experience in the early stages of learning. A weighting on the state features is learned using metric learning techniques, such that neighboring states represent similar game… 
View on IEEE
cnel.ufl.edu
21 Citations
Background Citations
6
Methods Citations
9

21 Citations

Perceptron Q-learning Applied to Super Smash Bros Melee

A reinforcement learning strategy for Super Smash Bros Melee, a video gamewith a continuous state and action space by applying global approximation in the Q-learning algorithm is discussed.

Modular Reinforcement Learning Framework for Learners and Educators

The EasyGameRL framework is proposed, a novel approach to the education of reinforcement learning in games using modular visual design patterns, and its software implementation in Unreal Engine are modular, reusable, and applicable to multiple game scenarios.

Theoretically Principled Deep RL Acceleration via Nearest Neighbor Function Approximation

This work presents Nearest Neighbor Actor-Critic (NNAC), an online policy gradient algorithm that demonstrates the practicality of combining function approximation with deep RL, and a plug-and-play NN update module that aids the training of existing deep RL methods.

Improving Reinforcement Learning Results with Qualitative Spatial Representation

A hybrid method that uses a qualitative formalism with reinforcement learning, named QRL, and is able to get better results than traditional methods is applied in the robot soccer domain and the results show that the agent can learn optimal policies and perform more goals than quantitative representation.

Learning to Navigate Through Complex Dynamic Environment With Modular Deep Reinforcement Learning

The proposed end-to-end modular reinforcement learning architecture for a navigation task in complex dynamic environments with rapidly moving obstacles can efficiently avoid moving obstacles and complete the navigation task at a high success rate.

Zombies Arena: fusion of reinforcement learning with augmented reality on NPC

The objective is to develop AR based first person shooter game empowering reinforcement learning that will act as a building block to capacitate users to interact with the physical environment.

Zombies Arena: fusion of reinforcement learning with augmented reality on NPC

The objective is to develop AR based first person shooter game empowering reinforcement learning that will act as a building block to capacitate users to interact with the physical environment.

Analysis of Agent Expertise in Ms. Pac-Man Using Value-of-Information-Based Policies

An information-theoretic cost function for performing constrained stochastic searches that promote the formation of risk-averse to risk-favoring behaviors is considered, which implies that the value-of-information theory is appropriate for framing the exploitation–exploration tradeoff in reinforcement learning.

Let's Do the Time Warp Again: Human Action Assistance for Reinforcement Learning Agents

A system called Time Warp is introduced that allows a human teacher to provide action selection assistance to the agent during critical moments of the training for the RL agent and rivals the performance of computer teaching agents.

Narrow Artificial Intelligence with Machine Learning for Real-Time Estimation of a Mobile Agents Location Using Hidden Markov Models

This work proposes to use a supervised machine learning technique to track the location of a mobile agent in real time and uses the BaumźWelch algorithm as a statistical learning tool to gain knowledge of the mobile target.

References

SHOWING 1-10 OF 39 REFERENCES

Playing Atari with Deep Reinforcement Learning

This work presents the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning, which outperforms all previous approaches on six of the games and surpasses a human expert on three of them.

Metric Learning for Invariant Feature Generation in Reinforcement Learning

This work proposes a method to automatically map states to feature vectors, which are not only sufficiently descriptive of the environment, but are invariant to information not relevant to the agent’s goal.

Stable Function Approximation in Dynamic Programming

Kernel-Based Reinforcement Learning

A kernel-based approach to reinforcement learning that overcomes the stability problems of temporal-difference learning in continuous state-spaces and shows that the limiting distribution of the value function estimate is a Gaussian process.

CBR for State Value Function Approximation in Reinforcement Learning

This paper investigates the use of case-based methods to realise the task of approximating a function over high-dimensional, continuous spaces and examines the approach taken in robotic soccer simulation.

Learning to Play Chess Using Temporal Differences

TDLEAF(λ), a variation on the TD(λ) algorithm that enables it to be used in conjunction with game-tree search, is presented and it is investigated whether it can yield better results in the domain of backgammon, where TD( ε) has previously yielded striking success.

Reinforcement Learning: An Introduction

This book provides a clear and simple account of the key ideas and algorithms of reinforcement learning, which ranges from the history of the field's intellectual foundations to the most recent developments and applications.

Using Imagery to Simplify Perceptual Abstraction in Reinforcement Learning Agents

This paper considers the problem of reinforcement learning in spatial tasks, and shows that this integration with an imagery system broadens the class of perceptual abstraction methods that can be used while preserving the underlying problem.

Covert Perceptual Capability Development

A model to develop robots’ covert perceptual capability using reinforcement learning, where covert perceptual behavior is treated as action selected by a motivational system, and K Nearest-Neighbor strategy is adopted to reduce training time complexity.

TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play

The latest version of TD-Gammon is now estimated to play at a strong master level that is extremely close to the world's best human players.