Double descent without digital computation

Le résumé fourni par la source

The success of large-scale AI relies on the phenomenon of double descent, where overfitting first increases and then counterintuitively decreases as the network grows relative to the amount of training data and becomes overparameterized. In this overparameterized regime it is easy to find good solutions that do not overfit. However, double descent has not previously been observed in physical neural networks, which are attracting increasing attention but remain small-scale. Here we demonstrate double descent in a decentralized analog network of self-adjusting resistive elements. This system trains itself and performs tasks without a digital processor, offering potential gains in energy efficiency and speed—but must inevitably confront component nonidealities. We find that standard training protocols fail to yield double descent, but a modified protocol that accommodates generic imperfections succeeds, qualitatively matching digital neural network simulations. Our findings show that analog physical neural networks, if appropriately trained, can reproduce important facets of overparameterized digital learning. This result is critical if such networks are to be scaled up to useful sizes.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.