text:   Transformers",5,"['New paper <LINK> 🧵 : How to train your ViT? It is common to train vision transformers on ImageNet-1k (~1.3m images) for 300 epochs. We show that you are better off investing the same compute budget for training on ImageNet-21k (~13m images) for 30 epochs. <LINK>', 'But this is just one out of many highlights. We also conduct extremely thorough study on the interplay of model size, dataset size, regularizations and augmentations. Plus extensive transfer learning experiments. Check out our paper if you want to learn how to train ViT models. https://t.co/fYX1M2XIny', ""We are sure there are lots of interesting insights to be drawn from our collection of trained ViTs. So we release them all: &gt;50'000 models!\n\nJAX repo: https://t.co/9SocJQaQtb.\nJAX colab: https://t.co/NwIS8nPXHD\n\nOr also available in pytorch through timm: https://t.co/HZie2tyfhQ."", 'Also check out this great thread with the summary of our insights: https://t.co/wrGNnZFuab', 'Work done with stellar Andreas Steiner, @XiaohuaZhai, @wightmanr, @kyosu and @giffmana.']",21,06,940