text:   Quadratic Regulator",10,"['New paper with @stephenltu and @beenwrekt on analyzing the sample complexity of policy iteration on the linear quadratic regulator. <LINK>', ""This paper spans ideas from control theory, reinforcement learning, and nonasymptotic statistics. It might be quite hard to parse if you haven't encountered some of those ideas before. In case you're having a hard time understanding it here's a tl;dr instead."", ""There's recently been a lot of interest in analyzing how many samples RL algorithms need to perform well. Unfortunately we don't have the mathematical tools to analyze these algorithms in full generality. So instead we study specialized problems."", 'One such problem is called the linear quadratic regulator (LQR) which is a classic continuous controls problem with a known optimal solution. The continuous part is very important since continuous RL has seen much less analysis than its discrete counterpart.', ""In our paper we study policy iteration (PI) when applied to LQR. Lucky for us specializing PI to LQR just involves taking this paper: https://t.co/AU0jCMmF0s and making a few variable substitutions. Most of the work is in proving bounds on the algorithm's sample complexity."", ""That's where ideas from statistics come into play. Thanks to recently published results we're able to provide a non-asymptotic analysis that assumes very little of our problem."", ""So what's the result of our analysis? Well first of all we show that the number of samples we need for PI has a quadratic (up to logarithmic factors) dependence on how close we want to get to the optimal policy."", ""Second we show that when we run PI and compare it against the optimal policy we won't be too worse off in terms of cost. In the language of multi-armed bandits we say that our algorithm exhibits sublinear regret."", 'In particular we show that the regret grows at a rate of at most T^(2/3) where T is the number of iterations we run for. This is in contrast to model-based methods that are able to achieve a regret of T^(1/2).', ""However our work doesn't definitively say that PI can't achieve a rate of T^(1/2). We only provide an upper bound on regret. It'd be very exciting if someone derived a corresponding lower bound to answer this question!""]",19,05,2193