text: 270,97,1491439202885861376,109603566,Stephen James,"New paper! Not all reinforcement learning problems are suited for a Gaussian policy parameterization!🤯 Plan to use 3D rotation or 6D pose as part of an action space? Consider the Bingham distribution! 🧵1/5 Paper: <LINK> Code: <LINK> w/@pabbeel <LINK> Quaternions are often used as the output rotation representation when using deep networks, but due to their antipodal symmetric property, sampling a quaternion from a Gaussian doesn't seem appropriate. Here comes the Bingham distribution to the rescue! 🧵2/5 <LINK> The Bingham distribution is super cool! It's parameterized by an orthogonal 4x4 matrix M, and a diag matrix Z. Intuitively, M holds information regarding the *direction* (akin to mean of Gaussian), while Z controls the *spread* (akin to variance) along the direction. 🧵3/5 <LINK> Although unexplored for RL, the Bingham distribution has had success in supervised learning! Two notable works are by @valentinp @domo_mr_roboto (<LINK>) and @igilitschenski (<LINK>). 🧵4/5 The gist of our paper is to show *how* we can leverage the power of Bingham for RL! 💪 When evaluating our approach on the Wahba problem and a set of vision-based robot manipulation tasks from RLBench, we achieve superior performance over a Gaussian parameterization! 🧵5/5 <LINK> @rndmcnlly @pabbeel I'm not familiar with the *matrix* variant you spoke of, but (correct me if I'm wrong), von Mises-Fisher does not model the antipodally symmetric property of Quaternions; I imagine the *matrix* variant would also have this issue. If true, Bingham seems the better option here.",https://arxiv.org/abs/2202.03957,"We propose a new policy parameterization for representing 3D rotations during reinforcement learning. Today in the continuous control reinforcement learning literature, many stochastic policy parameterizations are Gaussian. We argue that universally applying a Gaussian policy parameterization is not always desirable for all environments. One such case in particular where this is true are tasks that involve predicting a 3D rotation output, either in isolation, or coupled with translation as part of a full 6D pose output. Our proposed Bingham Policy Parameterization (BPP) models the Bingham distribution and allows for better rotation (quaternion) prediction over a Gaussian policy parameterization in a range of reinforcement learning tasks. We evaluate BPP on the rotation Wahba problem task, as well as a set of vision-based next-best pose robot manipulation tasks from RLBench. We hope that this paper encourages more research into developing other policy parameterization that are more suited for particular environments, rather than always assuming Gaussian. ","Bingham Policy Parameterization for 3D Rotations in Reinforcement