Direct/Inverse Hopfield model and Restricted Boltzmann Machines

Mean-field methods fail to reconstruct the parameters of the model when the dataset is clusterized. This situation is found at low temperatures because of the emergence of multiple thermodynamic states. The paradigmatic Hopfield model is considered in a teacher student scenario as a problem of unsupervised learning with Restricted Boltzmann Machines (RBM). For different choices of the priors on units and weights, the replica symmetric phase diagram of random RBM’s is analyzed and in particular the paramagnetic phase boundary is presented as directly related to the optimal size of the training set necessary for a good generalization. The connection between the direct and inverse problem is pointed out by showing that inference can be efficiently performed by suitably adapting both standard learning technique and standard approaches to the direct problem.