Méthodes à base de Réseaux de Neurones pour les Problèmes Inverses: Algorithmes et Garanties

Le résumé fourni par la source

This manuscript is devoted to the analysis of neural networks when trained in an unsupervised way to solve inverse problems in finite dimension. While these methods have become popular and heavily developed in the last years, leading to some qualitatively impressive results, they are lacking a thorough theoretical understanding, in particular of their recovery guarantees. In this thesis, our goal is to partly close that gap. For this, the key idea is to exploit the implicit regularization induced by the dynamic of the optimization method. Therefore, we study the trajectories of neural networks parameters under different optimization methods, and show how this leads to various inverse problem related recovery guarantees. We first study optimization through continuous gradient-flow, and its discrete counterpart gradient descent, for general sufficiently smooth loss functions that obey the Kurdyka-Lojasiewicz inequality. We show that under a non-degenerate initialization condition, the neural network will converge to a zero empirical risk solution with a rate that depends explicitly on the desingularizing function of the loss. We also provide an early-stopping bound to avoid the overfitting of the noise. We then show that with an additional restricted injectivity constraint, a recovery bound of the original object (e.g. signal/image, etc.) can be obtained. Second, we extend the above results when training with the mean square error loss using an inertial dynamic combining viscous and geometric Hessian-driven damping, and show that faster convergence and recovery guarantees can be obtained with a wise choice of dynamic parameters at the cost of more subtle initialization conditions. An inertial/momentum algorithm is then derived as a discretization of the continuous dynamic, it is then studied and its guarantees are established. For all these optimization methods, we also give an overparametrization bound under which a two-layer deep inverse prior network can benefit from the above guarantees with high probability. We numerically verify our results on a large ensemble of experiments, and we also exemplify our findings on two applications, for instance on multi-view shape-from-shading.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.