text:   Variational Spline Theory",3,"['What kinds of functions do deep neural networks learn?  Now we know!\n<LINK>\nOur paper presents a new representer theorem for deep ReLU networks and provides theoretical insights into  weight decay, sparsity, skip connections, and low-rank weight matrices.', '@roydanroy The main result says that a solution to a variational problem in a certain Banach space (compositional Radon BV) is a deep ReLU network having widths proportional to sample size, and that this optimization is equivalent to one over NNs that is min loss+ridge penalty on weights.', '@roydanroy It really falls together seamlessly.  I think these are the “right” spaces to be working with, at least for multilayer relu nets.']",21,05,683