text: 354,71,1320753970256744451,3377714115,Karl Pertsch,"How can we use large offline datasets for accelerating the learning of new tasks? We can transfer skills! Check out our #CoRL2020 paper on efficient skill transfer with learned skill priors! 📄Paper: <LINK> 💻Website & Code: <LINK> Thread👇(1/8) <LINK> Learning reusable skills is great: it does not need rewards (vs offline RL) and can be done fully offline without a training task distribution (vs meta-RL). But: with larger datasets, the number of learned skills grows. Exploring all skills on the target task is infeasible! (2/8) <LINK> Intuitively, not all skills should be explored with equal probability: when holding onto the handle of a kettle, it is more promising to attempt a pickup than a sweeping skill. We can learn such _skill priors_ from our experience solving other tasks! (3/8) <LINK> We introduce SPiRL (Skill-Prior RL), an approach for jointly learning an embedding space of skills and a prior over skills from offline data. For learning downstream tasks, we modify the SAC objective to use the learned prior as guidance for a hierarchical policy. (4/8) <LINK> We test SPiRL on three environments: maze navigation, block stacking and kitchen manipulation. For each we collect a large offline dataset and then test transfer to new tasks. (5/8) <LINK> On all environments, we show that SPiRL’s skill prior leads to better target task performance than using a “flat” prior over primitive actions or training a hierarchical policy without prior guidance. (6/8) <LINK> Only SPiRL is able to learn the most challenging target tasks — a hierarchical SAC baseline without prior guidance can solve some subtasks, but struggles to effectively explore the large set of learned skills. (7/8) <LINK> For more details, code and videos, check out our website: <LINK> joint work w/ @YoungwoonLee and @JosephLim_AI (8/8) Shoutout to @abhishekunique7 for publishing the kitchen environment with his RPL paper, and to Aviral Kumar and Justin Fu for packaging env+data nicely in the D4RL benchmark! :)",https://arxiv.org/abs/2010.11944,"Intelligent agents rely heavily on prior experience when learning a new task, yet most modern reinforcement learning (RL) approaches learn every task from scratch. One approach for leveraging prior knowledge is to transfer skills learned on prior tasks to the new task. However, as the amount of prior experience increases, the number of transferable skills grows too, making it challenging to explore the full set of available skills during downstream learning. Yet, intuitively, not all skills should be explored with equal probability; for example information about the current state can hint which skills are promising to explore. In this work, we propose to implement this intuition by learning a prior over skills. We propose a deep latent variable model that jointly learns an embedding space of skills and the skill prior from offline agent experience. We then extend common maximum-entropy RL approaches to use skill priors to guide downstream learning. We validate our approach, SPiRL (Skill-Prior RL), on complex navigation and robotic manipulation tasks and show that learned skill priors are essential for effective skill transfer from rich datasets. Videos and code are available at this https URL ",Accelerating Reinforcement Learning with Learned Skill Priors,9,"['How can we use large offline datasets for accelerating the learning of new tasks? We can transfer skills!\nCheck out our #CoRL2020 paper on efficient skill transfer with learned skill priors!\n📄Paper: <LINK>\n💻Website &amp; Code: <LINK>\n\nThread👇(1/8) <LINK>', 'Learning reusable skills is great: it does not need rewards (vs offline RL) and can be done fully offline without a training task distribution (vs meta-RL).\nBut: with larger datasets, the number of learned skills grows. Exploring all skills on the target task is infeasible!\n(2/8) https://t.co/XjecYmfYf1', 'Intuitively, not all skills should be explored with equal probability: when holding onto the handle of a kettle, it is more promising to attempt a pickup than a sweeping skill.\n\nWe can learn such _skill priors_ from our experience solving other tasks!\n(3/8) https://t.co/7rMtmlnP5x', 'We introduce SPiRL (Skill-Prior RL), an approach for jointly learning an embedding space of skills and a prior over skills from offline data. For learning downstream tasks, we modify the SAC objective to use the learned prior as guidance for a hierarchical policy.\n(4/8) https://t.co/eHlPwYbCGd', 'We test SPiRL on three environments: maze navigation, block stacking and kitchen manipulation. For each we collect a large offline dataset and then test transfer to new tasks.\n(5/8) https://t.co/eREcaEsTvi', 'On all environments, we show that SPiRL’s skill prior leads to better target task performance than using a “flat” prior over primitive actions or training a hierarchical policy without prior guidance.\n(6/8) https://t.co/IxxugJCbY0', 'Only SPiRL is able to learn the most challenging target tasks — a hierarchical SAC baseline without prior guidance can solve some subtasks, but struggles to effectively explore the large set of learned skills.\n(7/8) https://t.co/3Dhb2HkAXG', 'For more details, code and videos, check out our website: https://t.co/9fyZ1NfP9g\n\njoint work w/ @YoungwoonLee and @JosephLim_AI\n\n(8/8)', 'Shoutout to @abhishekunique7 for publishing the kitchen environment with his RPL paper, and to Aviral Kumar and Justin Fu for packaging env+data nicely in the D4RL benchmark! :)']",20,10,2000