### Abstract

The Willshaw model of associative memory, implemented in a fully connected network with stochastic asynchronous dynamics, is studied. In addition to Willshaw's learning rule, the network contains uniform synaptic inhibition, of relative strength K, and negative neural threshold -,>0. The P stored memories are sparsely coded. The total number of on bits in each memory is Nf, where f is much smaller than 1 but much larger than lnN/N. Mean-field theory of the system is solved in the limit where C==exp(-f2P) is finite. Memory states are stable (at zero temperature), as long as C>h0==K-1+ and h0>0. When C<h0 or h0<0, P retrieval phases, highly correlated with the memory states, exist. These phases are only partially frozen at low temperature, so that the full memories can be retrieved from them by averaging over the dynamic fluctuations of the neural activity. In particular, when h0<0 the retrieval phases at low temperatures correspond to freezing of most of the population in a quiescent state while the rest are active with a time average that can be significantly smaller than the saturation level. These features resemble, to some extent, the observed patterns of neural activity in the cortex, in experiments of short-term memory tasks. The maximal value of P for which stable retrieval phases exist, scales as f-3/lnf for f1/lnN, and as f-2ln(Nf/lnf) for f1/lnN. Numerical simulations of the model with N=1000 and f=0.04 are presented. We also discuss the possible realization of the model in a biologically plausible architecture, where the inhibition is provided by special inhibitory neurons.

Original language | English (US) |
---|---|

Pages (from-to) | 1843-1854 |

Number of pages | 12 |

Journal | Physical Review A |

Volume | 41 |

Issue number | 4 |

DOIs | |

State | Published - 1990 |

### Fingerprint

### ASJC Scopus subject areas

- Physics and Astronomy(all)
- Atomic and Molecular Physics, and Optics

### Cite this

*Physical Review A*,

*41*(4), 1843-1854. https://doi.org/10.1103/PhysRevA.41.1843

**Willshaw model : Associative memory with sparse coding and low firing rates.** / Golomb, D.; Rubin, N.; Sompolinsky, H.

Research output: Contribution to journal › Article

*Physical Review A*, vol. 41, no. 4, pp. 1843-1854. https://doi.org/10.1103/PhysRevA.41.1843

}

TY - JOUR

T1 - Willshaw model

T2 - Associative memory with sparse coding and low firing rates

AU - Golomb, D.

AU - Rubin, N.

AU - Sompolinsky, H.

PY - 1990

Y1 - 1990

N2 - The Willshaw model of associative memory, implemented in a fully connected network with stochastic asynchronous dynamics, is studied. In addition to Willshaw's learning rule, the network contains uniform synaptic inhibition, of relative strength K, and negative neural threshold -,>0. The P stored memories are sparsely coded. The total number of on bits in each memory is Nf, where f is much smaller than 1 but much larger than lnN/N. Mean-field theory of the system is solved in the limit where C==exp(-f2P) is finite. Memory states are stable (at zero temperature), as long as C>h0==K-1+ and h0>0. When C

AB - The Willshaw model of associative memory, implemented in a fully connected network with stochastic asynchronous dynamics, is studied. In addition to Willshaw's learning rule, the network contains uniform synaptic inhibition, of relative strength K, and negative neural threshold -,>0. The P stored memories are sparsely coded. The total number of on bits in each memory is Nf, where f is much smaller than 1 but much larger than lnN/N. Mean-field theory of the system is solved in the limit where C==exp(-f2P) is finite. Memory states are stable (at zero temperature), as long as C>h0==K-1+ and h0>0. When C

UR - http://www.scopus.com/inward/record.url?scp=0001670336&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0001670336&partnerID=8YFLogxK

U2 - 10.1103/PhysRevA.41.1843

DO - 10.1103/PhysRevA.41.1843

M3 - Article

VL - 41

SP - 1843

EP - 1854

JO - Physical Review A

JF - Physical Review A

SN - 2469-9926

IS - 4

ER -