Corpus ID: 237532316

Associative Memories via Predictive Coding

  title={Associative Memories via Predictive Coding},
  author={Tommaso Salvatori and Yuhang Song and Yujian Hong and Simon Frieder and Lei Sha and Zhenghua Xu and Rafał Bogacz and Thomas Lukasiewicz},
Associative memories in the brain receive and store patterns of activity registered by the sensory neurons, and are able to retrieve them when necessary. Due to their importance in human intelligence, computational models of associative memories have been developed for several decades now. They include autoassociative memories, which allow for storing data points and retrieving a stored data point s when provided with a noisy or partial variant of s, and heteroassociative memories, able to… Expand


Dense Associative Memory for Pattern Recognition
The proposed duality makes it possible to apply energy-based intuition from associative memory to analyze computational properties of neural networks with unusual activation functions - the higher rectified polynomials which until now have not been used in deep learning. Expand
Deep associative neural network for associative memory based on unsupervised representation learning
The DANN is able to achieve many machine learning problems, including not only classification, but also depicting the data given a label and recovering corrupted images, and is optimized by a modified contrastive divergence algorithm with a novel iterated sampling process. Expand
Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory.
The account presented here suggests that memories are first stored via synaptic changes in the hippocampal system, that these changes support reinstatement of recent memories in the neocortex, that neocortical synapses change a little on each reinstatement, and that remote memory is based on accumulated neocorticals changes. Expand
Cortical Representations Are Reinstated by the Hippocampus during Memory Retrieval
Functional evidence that the hippocampus reactivates specific memory representations during retrieval is provided, as it selectively prevented neurons that were active during learning from being reactivated during retrieval. Expand
Generative Predictive Codes by Multiplexed Hippocampal Neuronal Tuplets
It is proposed that flexible multiplexing of neuronal tuplets into repertoires of extended sequences vastly expands the capacity of hippocampal predictive codes, which could initiate top-down hierarchical cortical loops for spatial and mental navigation and rapid learning. Expand
On a Model of Associative Memory with Huge Storage Capacity
With this interaction, Krotov and Hopfield's generalized version of the well-known Hopfield model of associative memory has an exponential storage capacity in the number of neurons, yet the basins of attraction are almost as large as in the standard Hopfields. Expand
An Approximation of the Error Backpropagation Algorithm in a Predictive Coding Network with Local Hebbian Synaptic Plasticity
This work shows that a network developed in the predictive coding framework can efficiently perform supervised learning fully autonomously, employing only simple local Hebbian plasticity. Expand
Predictive Coding Can Do Exact Backpropagation on Convolutional and Recurrent Neural Networks
This work is the first to show that a biologically plausible algorithm is able to exactly replicate the accuracy of BP on such complex architectures, bridging the existing gap between IL and BP, and setting an unprecedented performance for PCNs, which can now be considered as efficient alternatives to BP. Expand
Hippocampal conjunctive encoding, storage, and recall: Avoiding a trade‐off
This analysis is focused on the feedforward pathways from the entorhinal cortex to the dentate gyrus (DG) and region CA3 and finds that Hebbian synaptic modification facilitates completion but reduces separation, unless the strengths of synapses from inactive presynaptic units to active postsynaptic units are reduced (LTD). Expand
Direct Reactivation of a Coherent Neocortical Memory of Context
A c-fos-based genetic tagging system is used to selectively express the channelrhodopsin variant, ChEF, and optogenetically reactivate a specific neural ensemble in retrosplenial cortex engaged by context fear conditioning, suggesting that higher-order cortical framework can independently subserve learned behavior, even shortly after learning. Expand