text:   and Wide Neural Networks",7,"['We’re thrilled to announce our new paper, in which we derive and test a ✨first-principles theory of generalization in deep learning!✨ We affectionately call it the Theory of Eigenlearning. 🧵⤵\n\n<LINK>', 'Recent breakthroughs have shown that ∞-width nets are actually very simple: their evolution’s described by a “neural tangent kernel” (NTK), and their final learned functions (when trained on MSE loss) are given by kernel regression with that NTK. (2/n)', 'We show that wide net learning is easily understood in the eigenbasis of the NTK: the network more easily learns high-eigenvalue eigenfunctions, which typically correspond to low spatial freq, quantifying the well-known but vague notion of “spectral bias” to low freqs. (3/n) https://t.co/Imki19Nzll', 'We derive simple expressions for the generalization of NTK regression, then show that they accurately predict generalization *even in finite nets!* We focus on a simple new measure of generalization we call ""learnability""; in Fig, exp (dots) agrees v well w/theory (curves) (4/n) https://t.co/zPP9omyrEb', 'We also prove a fundamental conservation law governing the inductive bias of wide nets: all NTKs have the same fixed budget of “learnability” that they must divvy up among their eigenmodes. Fig shows how the sum of these learnabilities is always the trainset size. (5/n) https://t.co/5U6T9Snpw6', 'This work was done by lab members Jamie Simon and Maddie Dickens. It builds on this pioneering work from @blake__bordelon, @canatar_a, and @CPehlevan, which you should also check out! https://t.co/QeDKdT6U7E', 'If you have any questions, drop us a tweet or an email! (7/7)']",21,10,1552