In applications, a substantial number of problems can be formulated as nonlinear least squares problems over smooth varieties. Unlike the classical least squares problem over a Euclidean space, the nonlinear least squares problem over a variety can be challenging to solve and analyze even if the variety itself is simple. Geometrically, this problem is equivalent to projecting a point in the ambient Euclidean space onto the image of the given variety under a nonlinear map. It is the singularities of the image that make both the computation and the analysis difficult. In this paper, we prove that, under some mild assumptions, these troublesome singularities can always be avoided. This enables us to establish that linear convergence rate can be generic for iterative sequences generated by algorithms satisfying some standard assumptions. We apply our general results to the low-rank partially orthogonal tensor approximation problem. As a consequence, we obtain the linear convergence rate for a classical alternating polar decomposition-alternating least squares method applied to a generic tensor without any further assumptions.

Publication:

MATHEMATICS OF OPERATIONS RESEARCH

http://dx.doi.org/10.1287/moor.2025.1060

Author:

Hu, Shenglong (corresponding author)

Natl Univ Def Technol, Coll Sci, Changsha 410072, Hunan, Peoples R China

Email address: hushenglong@nudt.edu.cn

Ye, Ke

Chinese Acad Sci, Acad Math & Syst Sci, State Key Lab Math Sci, Beijing 100190, Peoples R China

Email address: keyk@amss.ac.cn