# Online Learning with Random Representations

@inproceedings{Sutton1993OnlineLW, title={Online Learning with Random Representations}, author={R. Sutton and S. Whitehead}, booktitle={ICML}, year={1993} }

We consider the requirements of online learning|learning which must be done incrementally and in realtime, with the results of learning available soon after each new example is acquired. [...] Key Result Our results suggest that randomness has a useful role to play in online supervised learning and constructive induction. 1. Online Learning Applications of supervised learning can be divided into two types: online and oine. Expand

#### 133 Citations

Representation Search through Generate and Test

- Computer Science
- AAAI Workshop: Learning Rich Representations from Low-Level Sensors
- 2013

This work views the problem of representation learning as one of learning features such that performance of the underlying base system continually improves, and develops new representation-search methods and shows that the generate-and-test approach can be utilized in a simple and effective way for learning representations. Expand

Online Extreme Evolutionary Learning Machines

- Computer Science
- 2014

It is demonstrated that a Darwinian neurodynamic approach of feature replication can improve performance beyond selection alone, and may offer a path towards effective learning of predictive models in robotic agents. Expand

Position Paper: Representation Search through Generate and Test

- Computer Science
- SARA
- 2013

This work views the problem of representation learning as one of learning features such that performance of the underlying base system continually improves, and develops new representation-search methods and shows that the generate-and-test approach can be utilized in a simple and effective way for continually improving representations. Expand

The Interplay of Search and Gradient Descent in Semi-stationary Learning Problems

- Mathematics
- 2020

We explore the interplay of generate-and-test and gradient-descent techniques for solving online supervised learning problems. The task in supervised learning is to learn a function using samples of… Expand

Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding

- Computer Science
- NIPS
- 1995

It is concluded that reinforcement learning can work robustly in conjunction with function approximators, and that there is little justification at present for avoiding the case of general λ. Expand

Online Discovery of Feature Dependencies

- Mathematics, Computer Science
- ICML
- 2011

Online representational expansion techniques have improved the learning speed of existing reinforcement learning (RL) algorithms in low dimensional domains, yet existing online expansion methods do… Expand

Online Representation Search and Its Interactions with Unsupervised Learning

- 2012

We consider the problem of finding good hidden units, or features, for use in multilayer neural networks. Solution methods that generate candidate features, evaluate them, and retain the most useful… Expand

A survey of reinforcement learning in relational domains

- Computer Science
- 2005

The aim is to give a complete survey of the available literature, of the underlying motivations and of the implications if the new methods for learning in large, relational and probabilistic environments. Expand

A Bayesian Approach to Finding Compact Representations for Reinforcement Learning

- 2012

Feature-based function approximation methods have been applied to reinforcement learning to learn policies in a data-efficient way, even when the learner may not have visited all states during… Expand

A factorization perspective for learning representations in reinforcement learning

- 2014

Reinforcement learning is a general formalism for sequential decision-making, with recent algorithm development focusing on function approximation to handle large state spaces and high-dimensional,… Expand

#### References

SHOWING 1-10 OF 56 REFERENCES

Fast Learning in Networks of Locally-Tuned Processing Units

- Computer Science
- Neural Computation
- 1989

We propose a network architecture which uses a single internal layer of locally-tuned processing units to learn both classification tasks and real-valued function approximations (Moody and Darken… Expand

Feature discovery by competitive learning

- Computer Science
- 1985

This paper shows how a set of feature detectors which capture important aspects of the set of stimulus input patterns are discovered and how these feature detectors form the basis of a multilayer system that serves to learn categorizations of stimulus sets which are not linearly separable. Expand

Connectionist Learning Procedures

- Computer Science, Mathematics
- Artif. Intell.
- 1989

These relatively simple, gradient-descent learning procedures work well for small tasks and the new challenge is to find ways of improving their convergence rate and their generalization abilities so that they can be applied to larger, more realistic tasks. Expand

Sparse Distributed Memory

- Computer Science
- 1988

Pentti Kanerva's Sparse Distributed Memory presents a mathematically elegant theory of human long term memory that resembles the cortex of the cerebellum, and provides an overall perspective on neural systems. Expand

Practical Issues in Temporal Difference Learning

- 1992

This paper examines whether temporal difference methods for training connectionist networks, such as Sutton's TD(λ) algorithm, can be successfully applied to complex real-world problems. A number of… Expand

Enhancing Transfer in Reinforcement Learning by Building Stochastic Models of Robot Actions

- Computer Science
- ML
- 1992

Experiments on simulated and real robots show that useful action models can be learned from a 12 by 12 scrolling certainty grid sensor and on the simulator these models are sufficiently rich to enable significant transfer within and across two tasks, box pushing and wall following. Expand

Optimal Hidden Units for Two-Layer nonlinear Feedforward Neural Networks

- Mathematics, Computer Science
- Int. J. Pattern Recognit. Artif. Intell.
- 1991

A definition is proposed for optimal nonlinear features, and a constructive method, which has an iterative implementation, is derived for finding them. Expand

Predicting the Future: Advantages of Semilocal Units

- Mathematics, Medicine
- Neural Computation
- 1991

It is postulate that an interesing behavior displayed by gaussian bar functions under gradient descent dynamics, which is called automatic connection pruning, is an important factor in the success of this representation. Expand

The self-organizing map

- Computer Science
- 1990

The self-organizing map, an architecture suggested for artificial neural networks, is explained by presenting simulation experiments and practical applications, and an algorithm which order responses spatially is reviewed, focusing on best matching cell selection and adaptation of the weight vectors. Expand

An Evolutionary Pattern Recognition Network

- Mathematics
- 1969

A pattern recognition network with two types of adaptation has been investigated. The network output is a weighted sum of the outputs of elements which compute real functions of the discrete network… Expand