Global Convergence in Deep Networks via the Second Law of Thermodynamics
Existing convergence analyses of finite-width overparameterized deep neural networks under standard parameterization often rely on parameters remaining close to their initialization and typically neglect the role of noise, despite empirical evidence that stochasticity can improve optimization and generalization. To bridge this gap, we study $L_2$-regularized continuous-time Langevin dynamics (CLD) with isotropic noise, for training feedforward networks with smooth activations. We introduce a change-of-measure inequality, inspired by the second law of thermodynamics, that enables control of key properties of the training dynamics under the time-evolving parameter distribution. Leveraging a...
| Authors | Matan Tsipory, Ofir Gaash, Itamar Harel et al. |
|---|---|
| Primary category | cs.LG |
| Published | 17 October 2026 |
| Last updated at source | 17 October 2026 |
Popularity, last 5 days
0 now. Counts as reported by the source, snapshotted daily by Silytics.
Silytics shows public metadata from arXiv and links to the original. Information may be incomplete or out of date. Last refreshed 24 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.