# Learning classifier systems from a reinforcement learning perspective

@article{Lanzi2002LearningCS, title={Learning classifier systems from a reinforcement learning perspective}, author={Pier Luca Lanzi}, journal={Soft Computing}, year={2002}, volume={6}, pages={162-170} }

Abstract We analyze learning classifier systems in the light of tabular reinforcement learning. We note that although genetic algorithms are the most distinctive feature of learning classifier systems, it is not clear whether genetic algorithms are important to learning classifiers systems. In fact, there are models which are strongly based on evolutionary computation (e.g., Wilson's XCS) and others which do not exploit evolutionary computation at all (e.g., Stolzmann's ACS). To find some…

## Topics from this paper

## 89 Citations

Rule-based evolutionary online learning systems: learning bounds, classification, and prediction

- Computer Science
- 2004

The quantitative analysis of XCS shows that the interactive, evolutionary-based online learning mechanism works machine learning competitively yielding a low-order polynomial learning complexity.

Learning classifier system with average reward reinforcement learning

- Computer ScienceKnowl. Based Syst.
- 2013

R-learning is used as the reinforcement learning employed by XCS, to replace Q-learning, and produces uniformly spaced payoff levels, which can support long action chains and thus effectively prevent the occurrence of overgeneralization.

Combining latent learning with dynamic programming in the modular anticipatory classifier system

- Computer ScienceEur. J. Oper. Res.
- 2005

This paper describes how the model can be used to perform active exploration and how this exploration may be aggregated with the policy resulting from the reinforcement learning process, and proposes a new system, called Modular Anticipatory Classifier System (MACS), which allows the latent learning process to take advantage of new regularities.

Robot Learning Using Learning Classifier Systems Approach

- Computer Science
- 2010

The aim of the present contribution is to describe the state-of the-art of LCSs, emphasizing recent developments, and focusing more on the application of LCS for Robotics domain.

Learning classifier systems: a survey

- Computer ScienceSoft Comput.
- 2007

The present contribution is to describe the state-of- the-art of LCSs, emphasizing recent developments, and focusing more on the sequential decision domain than on automatic classification.

A comparison between ATNoSFERES and Learning Classifier Systems on non-Markov problems

- Computer ScienceInf. Sci.
- 2008

This paper presents a model based on an indirect encoding Genetic Algorithm which builds Augmented Transition Network controllers, and shows that ATNoSFERES is endowed with enough expressive power to represent the mechanisms used by the other two systems to deal with perceptual aliasing problems.

The anticipatory classifier system and genetic generalization

- Computer ScienceNatural Computing
- 2004

It is shown that it is possible to form a symbiosis between a directed specialization and a genetic generalization mechanism achieving a learning mechanism that evolves a complete, accurate, and compact description of theceived environment.

XCS with opponent modelling for concurrent reinforcement learners

- Computer ScienceNeurocomputing
- 2020

An algorithm X-OMQ(λ) that integrates eXtended Classifier System (XCS) with opponent modelling for concurrent reinforcement learners in zero-sum Markov Games is proposed that can learn general, accurate, and interpretable action selection rules and allow policy optimization using the genetic algorithm (GA).

Gradient descent methods in learning classifier systems: improving XCS performance in multistep problems

- Mathematics, Computer ScienceIEEE Transactions on Evolutionary Computation
- 2005

The extension of XCS to gradient-based update methods results in a classifier system that is more robust and more parameter independent, solving large and difficult maze problems reliably.

Learning classifier systems: New models, successful applications

- Computer ScienceInf. Process. Lett.
- 2002

This paper overviews the recent developments in learning classifier systems research, the new models, and the most interesting applications, suggesting some of the most relevant future research directions.

## References

SHOWING 1-10 OF 19 REFERENCES

Introducing a Genetic Generalization Pressure to the Anticipatory Classifier System - Part 1: Theoretical approach

- Computer ScienceGECCO
- 2000

This paper shows how a genetic algorithm can be used to overcome the present pressure of over-specialization in the ACS mechanism with a genetic generalization pressure.

Classifier Fitness Based on Accuracy

- Mathematics, Computer ScienceEvolutionary Computation
- 1995

A classifier system, XCS, is investigated, in which each classifier maintains a prediction of expected payoff, but the classifier's fitness is given by a measure of the prediction's accuracy, making it suitable for a wide range of reinforcement learning situations where generalization over states is desirable.

Evolutionary Algorithms for Reinforcement Learning

- Computer ScienceJ. Artif. Intell. Res.
- 1999

Strengths and weaknesses of the evolutionary approach to reinforcement learning are presented, along with a survey of representative applications.

Do We Really Need to Estimate Rule Utilities in Classifier Systems?

- Computer ScienceLearning Classifier Systems
- 1999

Preliminary tests of this classifier system on the multiplexor problem show that it performs as well as utility-based classifier systems such as XCS.

Technical Note: Q-Learning

- Computer ScienceMachine Learning
- 2004

A convergence theorem is presented and proves that Q -learning converges to the optimum action-values with probability 1 so long as all actions are repeatedly sampled in all states and the action- values are represented discretely.

An Introduction to Anticipatory Classifier Systems

- Computer ScienceLearning Classifier Systems
- 1999

Two extensions are discussed that enable an ACS to learn in any deterministic multistep environment and to deal with a special kind of non-Markov state in a deterministic multi-step environment.

Latent Learning and Action Planning in Robots with Anticipatory Classifier Systems

- Computer ScienceLearning Classifier Systems
- 1999

A simulation of an experiment about latent learning in rats with a mobile robot shows that an ACS is able to learn latently, i.e. in the absence of environmental reward and that ACS can do action planning.

A Roadmap to the Last Decade of Learning Classifier System Research

- Psychology, Computer ScienceLearning Classifier Systems
- 1999

This paper reviews the subsequent ten years of learning classifier systems research discussing the main achievements and the major research directions pursued in those years.

Genetic Algorithms in Search Optimization and Machine Learning

- Computer Science
- 1988

This book brings together the computer techniques, mathematical tools, and research results that will enable both students and practitioners to apply genetic algorithms to problems in many fields.

The Fighter Aircraft LCS: A Case of Different LCS Goals and Techniques

- Computer ScienceLearning Classifier Systems
- 1999

The authors believe the fighter aircraft LCS's success has three primary origins: differences in credit assignment, differences in action encoding, and (possibly most importantly) a difference in system goals.