text: 303,20,1143916572924190721,452384386,Sebastien Bubeck,"New paper <LINK> on black-box complexity of *parallel convex optimization*. We uncover yet again a quadratic acceleration phenomenon: while serial gradient descent is optimal up to depth D (=dim of the problem), for parallel it's true only up to depth sqrt(D)!!! <LINK> Prescient work by Nemirovski from 1994 (rediscovered recently by Balkanski-Singer <LINK>) essentially showed optimality of GD up to //-depth D^{1/3}: a key contribution of our work is to improve this to the optimal D^{1/2} by a new ``Wall function"" construction. From the algo side, Duchi-Bartlett-Wainwright already showed that after D^{1/2} depth one can improve the 1/eps^2 rate of GD to D^{1/4}/eps. We also improve this regime, and leverage our highly smooth acceleration paper <LINK> to obtain rate D^{1/3}/eps^{2/3}. Conjecture: D^{1/3}/eps^{2/3} is optimal in depth range [D^{1/2}, D]. Seems like fun & difficult convex geometry question!  Both highly smooth and parallel acceleration papers are joint work with Qijia Jiang (Stanford grad student), Yin Tat Lee, Yuanzhi Li, and Aaron Sidford. @roydanroy @aminkarbasi I disagree so strongly with that last comment I don't even know where to begin... @aminkarbasi @roydanroy Dan's. I happen to midly (much much more midly) disagree with yours too, just in the sense that I think there *is* a quite significant gain for small accuracies. To be concrete, say you are in d=10^9, and you want to reach accuracy eps=10^-6, then our // alg is 100x improvement.",https://arxiv.org/abs/1906.10655,"A landmark result of non-smooth convex optimization is that gradient descent is an optimal algorithm whenever the number of computed gradients is smaller than the dimension $d$. In this paper we study the extension of this result to the parallel optimization setting. Namely we consider optimization algorithms interacting with a highly parallel gradient oracle, that is one that can answer $\mathrm{poly}(d)$ gradient queries in parallel. We show that in this case gradient descent is optimal only up to $\tilde{O}(\sqrt{d})$ rounds of interactions with the oracle. The lower bound improves upon a decades old construction by Nemirovski which proves optimality only up to $d^{1/3}$ rounds (as recently observed by Balkanski and Singer), and the suboptimality of gradient descent after $\sqrt{d}$ rounds was already observed by Duchi, Bartlett and Wainwright. In the latter regime we propose a new method with improved complexity, which we conjecture to be optimal. The analysis of this new method is based upon a generalized version of the recent results on optimal acceleration for highly smooth convex optimization. ",Complexity of Highly Parallel Non-Smooth Convex Optimization,6,"[""New paper <LINK> on black-box complexity of *parallel convex optimization*. We uncover yet again a quadratic acceleration phenomenon: while serial gradient descent is optimal up to depth D (=dim of the problem), for parallel it's true only up to depth sqrt(D)!!! <LINK>"", 'Prescient work by Nemirovski from 1994 (rediscovered recently by Balkanski-Singer https://t.co/yq1VxQScqK) essentially showed optimality of GD up to //-depth D^{1/3}: a key contribution of our work is to improve this to the optimal D^{1/2} by a new ``Wall function"" construction.', 'From the algo side, Duchi-Bartlett-Wainwright already showed that after D^{1/2} depth one can improve the 1/eps^2 rate of GD to D^{1/4}/eps. We also improve this regime, and leverage our highly smooth acceleration paper https://t.co/lU18Ee0d6h to obtain rate D^{1/3}/eps^{2/3}.', 'Conjecture: D^{1/3}/eps^{2/3} is optimal in depth range [D^{1/2}, D]. Seems like fun &amp; difficult convex geometry question! \n\nBoth highly smooth and parallel acceleration papers are joint work with  Qijia Jiang (Stanford grad student), Yin Tat Lee, Yuanzhi Li, and Aaron  Sidford.', ""@roydanroy @aminkarbasi I disagree so strongly with that last comment I don't even know where to begin..."", ""@aminkarbasi @roydanroy Dan's.\n\nI happen to midly (much much more midly) disagree with yours too, just in the sense that I think there *is* a quite significant gain for small accuracies. To be concrete, say you are in d=10^9, and you want to reach accuracy eps=10^-6, then our // alg is 100x improvement.""]",19,06,1480