Learning classifier systems from a reinforcement learning perspective

@article{Lanzi2002LearningCS,
  title={Learning classifier systems from a reinforcement learning perspective},
  author={Pier Luca Lanzi},
  journal={Soft Computing},
  year={2002},
  volume={6},
  pages={162-170}
}
  • P. Lanzi
  • Published 1 June 2002
  • Computer Science
  • Soft Computing
Abstract We analyze learning classifier systems in the light of tabular reinforcement learning. We note that although genetic algorithms are the most distinctive feature of learning classifier systems, it is not clear whether genetic algorithms are important to learning classifiers systems. In fact, there are models which are strongly based on evolutionary computation (e.g., Wilson's XCS) and others which do not exploit evolutionary computation at all (e.g., Stolzmann's ACS). To find some… 
Rule-based evolutionary online learning systems: learning bounds, classification, and prediction
TLDR
The quantitative analysis of XCS shows that the interactive, evolutionary-based online learning mechanism works machine learning competitively yielding a low-order polynomial learning complexity.
Learning classifier system with average reward reinforcement learning
TLDR
R-learning is used as the reinforcement learning employed by XCS, to replace Q-learning, and produces uniformly spaced payoff levels, which can support long action chains and thus effectively prevent the occurrence of overgeneralization.
Combining latent learning with dynamic programming in the modular anticipatory classifier system
TLDR
This paper describes how the model can be used to perform active exploration and how this exploration may be aggregated with the policy resulting from the reinforcement learning process, and proposes a new system, called Modular Anticipatory Classifier System (MACS), which allows the latent learning process to take advantage of new regularities.
Robot Learning Using Learning Classifier Systems Approach
TLDR
The aim of the present contribution is to describe the state-of the-art of LCSs, emphasizing recent developments, and focusing more on the application of LCS for Robotics domain.
Learning classifier systems: a survey
TLDR
The present contribution is to describe the state-of- the-art of LCSs, emphasizing recent developments, and focusing more on the sequential decision domain than on automatic classification.
A comparison between ATNoSFERES and Learning Classifier Systems on non-Markov problems
TLDR
This paper presents a model based on an indirect encoding Genetic Algorithm which builds Augmented Transition Network controllers, and shows that ATNoSFERES is endowed with enough expressive power to represent the mechanisms used by the other two systems to deal with perceptual aliasing problems.
The anticipatory classifier system and genetic generalization
TLDR
It is shown that it is possible to form a symbiosis between a directed specialization and a genetic generalization mechanism achieving a learning mechanism that evolves a complete, accurate, and compact description of theceived environment.
XCS with opponent modelling for concurrent reinforcement learners
TLDR
An algorithm X-OMQ(λ) that integrates eXtended Classifier System (XCS) with opponent modelling for concurrent reinforcement learners in zero-sum Markov Games is proposed that can learn general, accurate, and interpretable action selection rules and allow policy optimization using the genetic algorithm (GA).
Gradient descent methods in learning classifier systems: improving XCS performance in multistep problems
TLDR
The extension of XCS to gradient-based update methods results in a classifier system that is more robust and more parameter independent, solving large and difficult maze problems reliably.
Learning classifier systems: New models, successful applications
TLDR
This paper overviews the recent developments in learning classifier systems research, the new models, and the most interesting applications, suggesting some of the most relevant future research directions.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 19 REFERENCES
Introducing a Genetic Generalization Pressure to the Anticipatory Classifier System - Part 1: Theoretical approach
TLDR
This paper shows how a genetic algorithm can be used to overcome the present pressure of over-specialization in the ACS mechanism with a genetic generalization pressure.
Classifier Fitness Based on Accuracy
TLDR
A classifier system, XCS, is investigated, in which each classifier maintains a prediction of expected payoff, but the classifier's fitness is given by a measure of the prediction's accuracy, making it suitable for a wide range of reinforcement learning situations where generalization over states is desirable.
Evolutionary Algorithms for Reinforcement Learning
TLDR
Strengths and weaknesses of the evolutionary approach to reinforcement learning are presented, along with a survey of representative applications.
Do We Really Need to Estimate Rule Utilities in Classifier Systems?
  • L. Booker
  • Computer Science
    Learning Classifier Systems
  • 1999
TLDR
Preliminary tests of this classifier system on the multiplexor problem show that it performs as well as utility-based classifier systems such as XCS.
Technical Note: Q-Learning
TLDR
A convergence theorem is presented and proves that Q -learning converges to the optimum action-values with probability 1 so long as all actions are repeatedly sampled in all states and the action- values are represented discretely.
An Introduction to Anticipatory Classifier Systems
  • W. Stolzmann
  • Computer Science
    Learning Classifier Systems
  • 1999
TLDR
Two extensions are discussed that enable an ACS to learn in any deterministic multistep environment and to deal with a special kind of non-Markov state in a deterministic multi-step environment.
Latent Learning and Action Planning in Robots with Anticipatory Classifier Systems
TLDR
A simulation of an experiment about latent learning in rats with a mobile robot shows that an ACS is able to learn latently, i.e. in the absence of environmental reward and that ACS can do action planning.
A Roadmap to the Last Decade of Learning Classifier System Research
TLDR
This paper reviews the subsequent ten years of learning classifier systems research discussing the main achievements and the major research directions pursued in those years.
Genetic Algorithms in Search Optimization and Machine Learning
TLDR
This book brings together the computer techniques, mathematical tools, and research results that will enable both students and practitioners to apply genetic algorithms to problems in many fields.
The Fighter Aircraft LCS: A Case of Different LCS Goals and Techniques
TLDR
The authors believe the fighter aircraft LCS's success has three primary origins: differences in credit assignment, differences in action encoding, and (possibly most importantly) a difference in system goals.
...
1
2
...