SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation

Stochastic Gradient Descent (SGD) is being used routinely for optimizing non-convex functions. Yet, the standard convergence theory for SGD in the smooth non-convex setting gives a slow sublinear convergence to a stationary point. In this work, we provide several convergence theorems for SGD showing...

Disgrifiad llawn

Manylion Llyfryddiaeth

Prif Awduron:	Gower, Robert M., Sebbouh, Othmane, Loizou, Nicolas
Fformat:	Erthygl
Cyhoeddwyd:	2020
Pynciau:	Optimization and Control Machine Learning
Mynediad Ar-lein:	https://arxiv.org/abs/2006.10311 http://arxiv.org/abs/2006.10311
Tagiau:	Ychwanegu Tag Dim Tagiau, Byddwch y cyntaf i dagio'r cofnod hwn!