Willshaw model

Associative memory with sparse coding and low firing rates

D. Golomb, N. Rubin, H. Sompolinsky

Research output: Contribution to journalArticle

Abstract

The Willshaw model of associative memory, implemented in a fully connected network with stochastic asynchronous dynamics, is studied. In addition to Willshaw's learning rule, the network contains uniform synaptic inhibition, of relative strength K, and negative neural threshold -,>0. The P stored memories are sparsely coded. The total number of on bits in each memory is Nf, where f is much smaller than 1 but much larger than lnN/N. Mean-field theory of the system is solved in the limit where C==exp(-f2P) is finite. Memory states are stable (at zero temperature), as long as C>h0==K-1+ and h0>0. When C<h0 or h0<0, P retrieval phases, highly correlated with the memory states, exist. These phases are only partially frozen at low temperature, so that the full memories can be retrieved from them by averaging over the dynamic fluctuations of the neural activity. In particular, when h0<0 the retrieval phases at low temperatures correspond to freezing of most of the population in a quiescent state while the rest are active with a time average that can be significantly smaller than the saturation level. These features resemble, to some extent, the observed patterns of neural activity in the cortex, in experiments of short-term memory tasks. The maximal value of P for which stable retrieval phases exist, scales as f-3/lnf for f1/lnN, and as f-2ln(Nf/lnf) for f1/lnN. Numerical simulations of the model with N=1000 and f=0.04 are presented. We also discuss the possible realization of the model in a biologically plausible architecture, where the inhibition is provided by special inhibitory neurons.

Original languageEnglish (US)
Pages (from-to)1843-1854
Number of pages12
JournalPhysical Review A
Volume41
Issue number4
DOIs
StatePublished - 1990

Fingerprint

associative memory
coding
retrieval
cortexes
neurons
freezing
learning
saturation
thresholds
simulation

ASJC Scopus subject areas

  • Physics and Astronomy(all)
  • Atomic and Molecular Physics, and Optics

Cite this

Willshaw model : Associative memory with sparse coding and low firing rates. / Golomb, D.; Rubin, N.; Sompolinsky, H.

In: Physical Review A, Vol. 41, No. 4, 1990, p. 1843-1854.

Research output: Contribution to journalArticle

Golomb, D. ; Rubin, N. ; Sompolinsky, H. / Willshaw model : Associative memory with sparse coding and low firing rates. In: Physical Review A. 1990 ; Vol. 41, No. 4. pp. 1843-1854.
@article{ead6c1d3446d4d9992e148a61a438226,
title = "Willshaw model: Associative memory with sparse coding and low firing rates",
abstract = "The Willshaw model of associative memory, implemented in a fully connected network with stochastic asynchronous dynamics, is studied. In addition to Willshaw's learning rule, the network contains uniform synaptic inhibition, of relative strength K, and negative neural threshold -,>0. The P stored memories are sparsely coded. The total number of on bits in each memory is Nf, where f is much smaller than 1 but much larger than lnN/N. Mean-field theory of the system is solved in the limit where C==exp(-f2P) is finite. Memory states are stable (at zero temperature), as long as C>h0==K-1+ and h0>0. When C",
author = "D. Golomb and N. Rubin and H. Sompolinsky",
year = "1990",
doi = "10.1103/PhysRevA.41.1843",
language = "English (US)",
volume = "41",
pages = "1843--1854",
journal = "Physical Review A",
issn = "2469-9926",
publisher = "American Physical Society",
number = "4",

}

TY - JOUR

T1 - Willshaw model

T2 - Associative memory with sparse coding and low firing rates

AU - Golomb, D.

AU - Rubin, N.

AU - Sompolinsky, H.

PY - 1990

Y1 - 1990

N2 - The Willshaw model of associative memory, implemented in a fully connected network with stochastic asynchronous dynamics, is studied. In addition to Willshaw's learning rule, the network contains uniform synaptic inhibition, of relative strength K, and negative neural threshold -,>0. The P stored memories are sparsely coded. The total number of on bits in each memory is Nf, where f is much smaller than 1 but much larger than lnN/N. Mean-field theory of the system is solved in the limit where C==exp(-f2P) is finite. Memory states are stable (at zero temperature), as long as C>h0==K-1+ and h0>0. When C

AB - The Willshaw model of associative memory, implemented in a fully connected network with stochastic asynchronous dynamics, is studied. In addition to Willshaw's learning rule, the network contains uniform synaptic inhibition, of relative strength K, and negative neural threshold -,>0. The P stored memories are sparsely coded. The total number of on bits in each memory is Nf, where f is much smaller than 1 but much larger than lnN/N. Mean-field theory of the system is solved in the limit where C==exp(-f2P) is finite. Memory states are stable (at zero temperature), as long as C>h0==K-1+ and h0>0. When C

UR - http://www.scopus.com/inward/record.url?scp=0001670336&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0001670336&partnerID=8YFLogxK

U2 - 10.1103/PhysRevA.41.1843

DO - 10.1103/PhysRevA.41.1843

M3 - Article

VL - 41

SP - 1843

EP - 1854

JO - Physical Review A

JF - Physical Review A

SN - 2469-9926

IS - 4

ER -