Learning Multilevel Distributed Representations for High-Dimensional Sequences

Abstract

We describe a new family of non-linear sequence models that are substantially more powerful than hidden Markov models or linear dynamical systems. Our models have simple approximate inference and learning procedures that work well in practice. Multilevel representations of sequential data can be learned one hidden layer at a time, and adding extra hidden layers improves the resulting generative models. The models can be trained with very high-dimensional, very non-linear data such as raw pixel sequences. Their performance is demonstrated using synthetic video sequences of two balls bouncing in a box.

Extracted Key Phrases

3 Figures and Tables

02040'06'07'08'09'10'11'12'13'14'15'16'17
Citations per Year

181 Citations

Semantic Scholar estimates that this publication has 181 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@inproceedings{Sutskever2007LearningMD, title={Learning Multilevel Distributed Representations for High-Dimensional Sequences}, author={Ilya Sutskever and Geoffrey E. Hinton}, booktitle={AISTATS}, year={2007} }