Speaker: Dr. Molei Tao from Georgia Institute of Technology
Date and Time: Friday, November 19, 2021, 11am-12pm via Zoom. Please contact Qingning Zhou to obtain the Zoom link.
Title: Examples of interactions between dynamics and machine learning
Abstract: This talk will report some of our progress in showing how dynamics can be a useful mathematical tool for machine learning. Three demonstrations will be given, namely, how dynamics help design (and analyze) optimization algorithms, how dynamics help quantitatively understand nontrivial observations in deep learning practices, and how deep learning can in turn help dynamics (or more broadly put, AI for sciences). More precisely, in part 1 (dynamics for algorithm): I will talk about how to add momentum to gradient descent on a class of manifolds known as Lie groups. The treatment will be based on geometric mechanics and dynamics in continuous and discrete time, and it will lead to accelerated optimization. Part 2 (dynamics for understanding deep learning) will be on how large learning rates could deterministically lead to escapes from local minima, which is an alternative mechanism to commonly known noisy escapes due to stochastic gradients. If time permits, I will also talk about another example, on an implicit regularization effect of large learning rates (which we term as `balancing’). Part 3 (AI for sciences) will be on data-driven prediction of mechanical dynamics, for which I will demonstrate one strong benefit of having physics hard-wired into deep learning models (more precisely, how to obtain symplectic predictions, and how that generically enables accurate long-time predictions).