Academy of Mathematics and Systems Science, CAS
Colloquia & Seminars

Speaker:	Chao Ma, Princeton University
Inviter:	刘歆副研究员
Title:	A-priori estimates of population risks for neural networks models
Time & Venue:	2019.7.25 10:00-11:00 Z311
Abstract:	Optimal a priori estimates are derived for the generalization error of regularized two-layer network and deep residual network models. For two-layer neural networks, the path norm is used as the regularization term. For residual networks, we define and use the "weighted path norm", which treats the skip connections and the nonlinearities differently so that paths with more nonlinearities are regularized by larger weights. The error estimates are a priori in the sense that the estimates depend only on the target function, not on the parameters obtained in the training process. The estimates are optimal, in a high dimensional setting, in the sense that both the bounds for approximation and estimation errors are comparable to the Monte Carlo error rates. Comparisons are made with existing norm-based generalization error bounds.