Colloquium: Prof. Hans De Sterck
Event Description:
https://unm.zoom.us/j/91525130869
Title:
Convergence Acceleration for Nonlinear Fixed-Point Methods
Abstract:
Empirical results show that nonlinear convergence acceleration methods such as Anderson Acceleration (AA) or the Nonlinear Generalized Minimal Residual (NGMRES) method may often dramatically speed up the convergence of fixed-point algorithms that are widely used in scientific computing and optimization. However, little is known theoretically that can help us to understand and quantify the asymptotic convergence improvement. We present new results that shed light on this open problem, by considering optimal stationary versions of the acceleration methods that allow us to quantify the convergence improvement using spectral properties of the Jacobian of the fixed-point iteration function, viewed as a nonlinear preconditioner. We illustrate these findings for nonlinear acceleration of the Alternating Least Squares (ALS) method for tensor decomposition, and the Alternating Direction Method of Multipliers (ADMM) for optimization problems in machine learning.