text:   Instruction Following",9,"['New NLP+robotics+vision paper: Few-shot Object Grounding and Mapping for Natural Language Robot Instruction Following @ CoRL 2020\n<LINK>\nWork done by Valts Blukis\nThree core contributions make this happen, let’s upack them ... <LINK>', 'First contribution: a general few-shot language-conditioned object segmentation method. We align references to objects in language to their observations in the world via a database of exemplars. Need to reason about new objects? Add exemplars to the database! https://t.co/ZZYmO1dxkK', 'Training requires quite a bit of data to generalize well to unseen objects, *but* the nice part: the whole thing is trained from automatically generated augmented reality data using synthetic ShapeNet objects https://t.co/kDAf7aXzLU', 'Second contribution: given the alignment, we formalize and build a learned object context map. The map masks over object identities and easily handles new objects. It explicitly encodes text context information into positions in the spatial environment. https://t.co/qjlpE7g29g', 'Third contribution: integrating the map into a complete policy (natural language instructions, raw observations) -&gt; continuous velocity control. The policy generates plans on top of our context maps as visitation distributions. https://t.co/I8wb6Pfhin', 'First stage mapping is through a series of projections and using our LingUNet architecture to predict the visitation distributions on top of the map (we first introduced this method in our CoRL 2018 paper) https://t.co/UWYuHvvRJs', 'Agent never saw all these objects during learning, just a few exemplars. Bonus 1: modeling of concepts like objects and trajectories is strong inductive bias —&gt; outperform systems that learn with hundreds of demonstrations with new objects.  Let’s look at the complete video again https://t.co/pBZ8qYakTV', 'Bonus2: interpretability via explicit alignments! Another example ... https://t.co/M1jzdfz3Wz', 'There’s a much longer talk with more details https://t.co/41Isbg4JEt from the recent SpLU workshop and also a brief overview from CoRL https://t.co/fk8Fw7seXN']",20,11,1920