Optimal errors and phase transitions in high-dimensional generalized linear models

Generalized linear models arise in high-dimensional machine learning, statistics, communications and signal processing. In this talk we review such models in a teacher-student setting of supervised learning, and when the data matrix is random, as relevant in benchmark models of neural networks. Predictions for the mutual information and Bayes-optimal generalization errors have existed since a long time for special cases, e.g. for the perceptron or the committee machine, in the field of statistical physics based on spin-glass methods. We will explain recently developed mathematical techniques rigorously establishing those old conjectures and bring forward their algorithmic interpretation in terms of performance of message-passing algorithms. For many learning problems, we will illustrate regions of parameters for which message passing algorithms achieve the optimal performance, and locate the associated sharp phase transitions separating learnable and non-learnable regions. These rigorous results can serve as a challenging benchmark for multi-purpose algorithms.