text: 18,336,1319628325380378624,182730982,Andreas Rücklé,"Check out our paper “AdapterDrop""!  We find that Adapters can train 60% faster than full fine-tuning. With AdapterDrop we increase inference speed by up to 36% for 8 parallel tasks.  GGeigle @Maxxx216 @devnull90 @PfeiffJo NReimers IGurevych @AdapterHub <LINK> <LINK> With our *robust* AdapterDrop layers, we improve the efficiency of adapter models during training and inference. By removing adapters from lower layers, based on available computational resources, we enable a dynamic trade-off between inference speed and task performance. <LINK> Interestingly, we also find that we can share the adapter weights across *all* layers without losing much performance. This drastically reduces the storage space required for each task even more. <LINK> We also reveal significant potential for boosting the efficiency of AdapterFusion. We can drop the majority of adapters after transfer learning from the Fusion model, while maintaining task performance entirely. <LINK>",https://arxiv.org/abs/2010.11918,"Massively pre-trained transformer models are computationally expensive to fine-tune, slow for inference, and have large storage requirements. Recent approaches tackle these shortcomings by training smaller models, dynamically reducing the model size, and by training light-weight adapters. In this paper, we propose AdapterDrop, removing adapters from lower transformer layers during training and inference, which incorporates concepts from all three directions. We show that AdapterDrop can dynamically reduce the computational overhead when performing inference over multiple tasks simultaneously, with minimal decrease in task performances. We further prune adapters from AdapterFusion, which improves the inference efficiency while maintaining the task performances entirely. ",AdapterDrop: On the Efficiency of Adapters in Transformers,4,"['Check out our paper “AdapterDrop""!\n \nWe find that Adapters can train 60% faster than full fine-tuning. With AdapterDrop we increase inference speed by up to 36% for 8 parallel tasks.\n \nGGeigle @Maxxx216 @devnull90 @PfeiffJo NReimers IGurevych @AdapterHub\n<LINK> <LINK>', 'With our *robust* AdapterDrop layers, we improve the efficiency of adapter models during training and inference. By removing adapters from lower layers, based on available computational resources, we enable a dynamic trade-off between inference speed and task performance. https://t.co/oTiMSmEPpn', 'Interestingly, we also find that we can share the adapter weights across *all* layers without losing much performance. This drastically reduces the storage space required for each task even more. https://t.co/7cmEk6jRt1', 'We also reveal significant potential for boosting the efficiency of AdapterFusion. We can drop the majority of adapters after transfer learning from the Fusion model, while maintaining task performance entirely. https://t.co/b4eHQTrs5G']",20,10,968