Online Mathematics seminar by Prof Molei Tao (Georgia Institute of Technology), held on 10 March 2022.

Title: Machine learning meets mechanics: manifold optimization with momentum and learning mechanical dynamics

Abstract: This talk will report some of our efforts in showing how dynamics and machine learning can help each other. Primarily two demonstrations will be given, namely, how dynamics help design (and analyze) optimization algorithms, and how deep learning can in turn help dynamics (or more broadly put, AI for sciences). More precisely, in part 1 (dynamics for algorithm) I will talk about how to add momentum to gradient descent on Riemannian manifolds and Lie groups. The treatment will be based on geometric mechanics and an interplay between continuous and discrete time dynamics. It will lead to accelerated optimization. Part 2 (AI for sciences) will be on data-driven prediction of mechanical dynamics, for which I will demonstrate one strong benefit of having physics hard-wired into deep learning models; more precisely, how to make exactly symplectic predictions, and how that provably improves the accuracy of long-time predictions. If time permits, I will also briefly describe how dynamics can help quantitatively understand deep learning, in the aspect of how large learning rate can (3a) provide a local minimum escape mechanism and (3b) implicitly bias toward flatter minimum.

Bio: Prof. Molei Tao received B.S. in Mathematics and Physics (Academic Talent Program) in 2006 from Tsinghua University, China, and Ph.D. in Control & Dynamical Systems with a minor in Physics in 2011 from California Institute of Technology (Caltech; advisor: Houman Owhadi, co-advisor: Jerry Marsden). Afterwards, he worked as a postdoctoral researcher in Computing & Mathematical Sciences at Caltech from 2011 to 2012, and then as a Courant Instructor at New York University from 2012 to 2014. From 2014 on, he has been working as an assistant, and then associate professor in School of Mathematics at Georgia Institute of Technology. He is a recipient of W.P. Carey Ph.D. Prize in Applied Mathematics (2011), American Control Conference Best Student Paper Finalist (2013), the NSF CAREER Award (2019), AISTATS best paper award (2020), IEEE EFTF-IFCS Best Student Paper Finalist (2021), and Cullen-Peck Scholar Award (2022).