Lecture 3: Dynamics and landscapes in high dimension

This series of lectures aims to review the research landscape on high-dimensional dynamical systems, a topic that arises in multiple disciplines. In physics, understanding the many-body dynamics of complex systems is crucial for characterizing their equilibration properties—or the lack thereof. Beyond physics, the study of dynamics is equally vital in optimization and computer science, particularly when dealing with high-dimensional, non-convex problems—such as those encountered in the training dynamics of artificial neural networks. In neuroscience, these systems also model recurrent neural networks, where fixed points, chaos, and control mechanisms shape their behavior. The course has a twofold purpose: first, to survey current knowledge of high-dimensional dynamical systems across various contexts; and second, to explore dynamical mean field theory, the primary toolkit for tackling these challenges. Outline: 1. Stochastic differential equations, Langevin dynamics, Fokker-Planck equation and stationary measures. 2. Equilibrium dynamics and Fluctuation-Dissipation relations. 3. Dynamical mean field theory: Dynamical cavity method 4. Dynamics of mean field spin glasses. High temperature phase and relaxation to equilibrium; Low temperature phase, aging. Landscape interpretation. 5. High-dimensional inference: Langevin/Gradient Descent algorithms and their suboptimality with respect to Approximate Message Passing. 6. Training dynamics of artificial neural networks. Neural tangent kernel and feature learning theory. Separation of timescales between feature learning and overfitting. 7. Recurrent neural networks. Transition to and chaos in high dimension. Maximal Lypaunov exponent. 8. Learning algorithms and optimal control of high-d chaotic dynamics. 9. From Neural ODEs to LLMs and generative models 10. Open problems and perspectives