A Cortex-inspired Associative Memory with O(1) Time Complexity Learning, Recall, and Recognition of Sequences

Abstract

A cortex-inspired associative memory model possessing O(1) time complexity for both storage (learning) and retrieval (recall, recognition) of sequences is described. It learns sequences, specifically, binary spatiotemporal patterns, with single trials. Results are given demonstrating O(1) retrieval of: a) a sequence’s remaining items when prompted with its initial item, i.e., episodic recall; and b) the most similar stored sequence when presented with a novel sequence, i.e., recognition/categorization. The hidden (representation) layer, L2, is organized into winner-take-all competitive modules (CMs) hypothesized to be analogous to cortical minicolumns. Representations consist of one active unit per CM. The heart of the model is a matching algorithm that, given the current input moment, σ, i.e., the current input item in the context of the sequence thus far: a) finds the stored representation, ∆σ*, of that previously experienced moment, σ*, out of all previously experienced moments, which is spatiotemporally most similar to σ; and b) returns a normalized measure, G, of that similarity. When in recall or recognition mode, the model simply reactivates ∆σ* since it is the most likely hypothesis given the model’s history and the current input. When in learning mode, the model injects an amount of noise, inversely proportional G, into the process of choosing the cells to represent σ. This yields the property that the size of the intersection between representations is an increasing function of the spatiotemporal similarity of the moments that they represent. Thus, the higher-order statistics (spatiotemporal similarity structure) of the set of learned sequences is reflected directly in the patterns of intersections over the set of representations. This property, in conjunction with the use of the binary sparse representation, makes the O(1) recall and recognition (i.e., inference) possible.

12 Figures and Tables

Cite this paper

@inproceedings{Rinkus2006ACA, title={A Cortex-inspired Associative Memory with O(1) Time Complexity Learning, Recall, and Recognition of Sequences}, author={Gerard J. Rinkus}, year={2006} }