Seminario del 2024
Maggio
del 15/05/2024
Federico Ricci-Tersenghi
Daydreaming Hopfield Networks and their surprising effectiveness on correlated data
Seminario di fisica matematica, interdisciplinare
To improve the storage capacity of the Hopfield model, we develop a version of the dreaming
algorithm that is perpetually exposed to data and therefore called Daydreaming. Daydreaming
is not destructive and converges asymptotically to a stationary coupling matrix. When trained
on random uncorrelated examples, the model shows optimal performance in terms of the size of
the basins of attraction of stored examples and the quality of reconstruction. We also train the
Daydreaming algorithm on correlated data obtained via the random-features model and argue
that it spontaneously exploits the correlations thus increasing even further the storage capacity
and the size of the basins of attraction. Moreover, the Daydreaming algorithm is also able to
stabilize the features hidden in the data. Finally, we test Daydreaming on the MNIST dataset and
show that it still works surprisingly well, producing attractors that are close to unseen examples
and class prototypes.