Academy of Mathematics and Systems Science, CAS
Colloquia & Seminars

Speaker:	张小群教授，上海交通大学
Inviter:	陈冲副研究员
Title:	Stochastic splitting algorithms for nonconvex problems in imaging and data sciences
Time & Venue:	2022.07.05 10:20-11:05 腾讯会议: 371-565-071 会议密码：311311
Abstract:	Splitting algorithms are largely adopted for composited optimization problems arising in imaging and data sciences. In this talk, I will present stochastic variants of composited optimization algorithms in nonconvex settings and their applications. The first class of algorithms is based on Alternating direction method of multipliers (ADMM) for nonconvex composite problems. In particular, we study the ADMM method combined with a class of variance reduction gradient estimators and established the global convergence of the sequence and convergence rate under the assumption of Kurdyka-Lojasiewicz (KL) function. The efficiency of the algorithms is verified through statistical learning examples and L0 based sparse regularization for 3D image reconstruction. The second class of stochastic algorithm is proposed for a type of three-block alternating minimization arising in training quantized neural networks. We develop a convergence theory for the stochastic three-block algorithm (STAM) and obtain an $\epsilon$-stationary point with optimal convergence rate $\mathcal{O}(\epsilon^{-4})$. The experiments on training quantized DNNs are carried out on different network structures on CIFAR-10 and CIFAR-100 datasets. The test accuracy indicates the effectiveness of STAM algorithm for training binary quantization DNNs.