Improve model card: Update pipeline tag, add library name, and update news section

This PR improves the model card by:

- Correcting the `pipeline_tag` to `text-generation`, which accurately reflects the model's functionality.
- Adding the `library_name` tag as `transformers` for better model loading.
- Updating the news section with the latest information from the Github README, including the ICLR 2025 acceptance.

These changes ensure the model card is more informative and facilitates easier model usage and discovery.

README.md

@@ -1,37 +1,88 @@
 ---
 license: apache-2.0
-pipeline_tag: image-text-to-text
+pipeline_tag: text-generation
+library_name: transformers
 ---
 
 <h2 align="center"> <a href="https://arxiv.org/abs/2405.14297">Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models</a></h2>
 <h5 align="center"> If our project helps you, please give us a star ⭐ on <a href="https://github.com/LINs-lab/DynMoE">GitHub</a> and cite our paper!</h2>
 <h5 align="center">
 
-## 📰 News
+[![hf_space](https://img.shields.io/badge/🤗-Paper%20In%20HF-red.svg)](https://huggingface.co/papers/2405.14297)
+[![arxiv](https://img.shields.io/badge/Arxiv-2405.14297-b31b1b.svg?logo=arXiv)](https://arxiv.org/abs/2405.14297)
+[![visitor](https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fgithub.com%2FLINs-lab%2FDynMoE&count_bg=%2379C83D&title_bg=%23454343&icon=&icon_color=%23E7E7E7&title=visitor&edge_flat=false)](https://hits.seeyoufarm.com)
 
-- **[2024.05.31]** 🔥 Our [code](https://github.com/LINs-lab/DynMoE/) is released!
-- **[2024.05.25]** 🔥 Our **checkpoints** are available now!
-- **[2024.05.23]** 🔥 Our [paper](https://arxiv.org/abs/2405.14297) is released!
+## News
+- **[2025.01.23]**: 🎉 Our paper is accepted to ICLR 2025!
+- **[2024.05.25]** Our [checkpoints](https://huggingface.co/collections/LINs-lab/dynmoe-family-665ed5a331a7e84463cab01a) are available now!
+- **[2024.05.23]** Our [paper](https://arxiv.org/abs/2405.14297) is released!
 
-## 😎 What's Interesting?
+## Why Do We Need DynMoE?
 
-**Dynamic Mixture of Experts (DynMoE)** incorporates (1) a novel gating method that enables each token to automatically determine the number of experts to activate. (2) An adaptive process automatically adjusts the number of experts during training.
+Sparse MoE (SMoE) has an unavoidable drawback: *the performance of SMoE heavily relies on the choice of hyper-parameters, such as the number of activated experts per token (top-k) and the number of experts.*
 
-### Top-Any Gating
+Also, *identifying the optimal hyper-parameter without a sufficient number of ablation studies is challenging.* As the size of the models continues to grow, this limitation could result in a significant waste of computational resources, and in turn, could hinder the efficiency of training MoE-based models in practice.
 
-<video controls src="https://i.imgur.com/bLgNaoH.mp4" title="Top-Any Gating"></video>
+Now, our **DynMoE** addresses these challenges through the two components introduced in [Dynamic Mixture of Experts (DynMoE)](#dynamic-mixture-of-experts-dynmoe).
 
-### Adaptive Training Process
+## Dynamic Mixture of Experts (DynMoE)
 
-![](https://cdn.jsdelivr.net/gh/QAQdev/Pics@master/uPic/adaptive.png)
+## Top-Any Gating
 
-## 💡 Model Details
+![hh](./assets/moe-overview.gif)
 
-- 🤔 DynMoE-StableLM is a MoE model with **dynamic top-k gating**, finetuned on [LanguageBind/MoE-LLaVA-StableLM-Stage2](https://huggingface.co/LanguageBind/MoE-LLaVA-StableLM-Stage2).
-- 🚀 Our DynMoE-StableLM-1.6B has totally 2.9B parameters, but **only 1.8B are activated!** (average top-k = 1.25)
-- ⌛ With the DynMoE tuning stage, we can complete training on 8 A100 GPUs **within 40 hours.**
+We first introduce a novel gating method that enables each token to **automatically determine the number of experts to activate**.
 
-## 👍 Acknowledgement
+## Adaptive Training Process
+
+![adaptive-training](https://cdn.jsdelivr.net/gh/QAQdev/Pics@master/uPic/adaptive.png)
+
+Our method also includes an adaptive process **automatically adjusts the number of experts** during training.
+
+## Can We Trust DynMoE? Yes!
+
+- On language tasks, **DynMoE surpasses the average performance among various MoE settings.**
+- **Effectiveness of DynMoE remains consistent** in both Vision and Vision-Language tasks.
+- Although sparsity is not enforced in DynMoE, it **maintains efficiency by activating even less parameters!**
+
+## Model Zoo
+
+| Model | Activated Params / Total Params| Transformers(HF) |
+| ----- | --------------- | ---------------- |
+| DynMoE-StableLM-1.6B | 1.8B / 2.9B | [LINs-lab/DynMoE-StableLM-1.6B](https://huggingface.co/LINs-lab/DynMoE-StableLM-1.6B)
+| DynMoE-Qwen-1.8B | 2.2B / 3.1B | [LINs-lab/DynMoE-Qwen-1.8B](https://huggingface.co/LINs-lab/DynMoE-Qwen-1.8B)
+| DynMoE-Phi-2-2.7B | 3.4B / 5.3B| [LINs-lab/DynMoE-Phi-2-2.7B](https://huggingface.co/LINs-lab/DynMoE-Phi-2-2.7B)
+
+##  Directory Specification
+
+### Experiment Code
+
+- `EMoE/` contains experiments on language and vision tasks, which uses tutel-based DynMoE.
+- `MoE-LLaVA/` contains experiments on language-vision tasks, which uses deepspeed-0.9.5-based DynMoE.
+
+### DynMoE Implementations
+
+- `Deepspeed/` provides DynMoE-Deepspeed implementation. **(Recommend)**
+- `EMoE/tutel/` provides DynMoE-Tutel implementation.
+
+## Environment Setup
+
+Please refer to instructions under `EMoE/` and `MoE-LLaVA/`.
+
+## Usage
+
+### Tutel Examples
+
+Please refer to `EMoE/Language/README.md` and `EMoE/Language/Vision.md`.
+
+### DeepSpeed Examples (Recommend)
+
+We give a minimal example to train DynMoE-ViT on ImageNet-1K from scratch at `Examples/DeepSpeed-MoE`.
+
+- Check `Examples/DeepSpeed-MoE/dynmoe_vit.py` for how to use DynMoE in model implementation.
+- Check `Examples/DeepSpeed-MoE/train.py` for how to train model with DynMoE.
+
+## Acknowledgement
 
 We are grateful for the following awesome projects:
 
@@ -42,19 +93,21 @@ We are grateful for the following awesome projects:
 - [MoE-LLaVA](https://github.com/PKU-YuanGroup/MoE-LLaVA)
 - [GLUE-X](https://github.com/YangLinyi/GLUE-X)
 
-## 🔒 License
-
-This project is released under the Apache-2.0 license as found in the [LICENSE](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) file.
+## Citation
 
-## ✏️ Citation
+If you find this project helpful, please consider citing our work:
 
-```tex
-@misc{guo2024dynamic,
-      title={Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models}, 
-      author={Yongxin Guo and Zhenglin Cheng and Xiaoying Tang and Tao Lin},
-      year={2024},
-      eprint={2405.14297},
-      archivePrefix={arXiv},
-      primaryClass={cs.LG}
+```bibtex
+@article{guo2024dynamic,
+  title={Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models},
+  author={Guo, Yongxin and Cheng, Zhenglin and Tang, Xiaoying and Lin, Tao},
+  journal={arXiv preprint arXiv:2405.14297},
+  year={2024}
 }
-```
+```
+
+## Star History
+
+[![Star History Chart](https://api.star-history.com/svg?repos=LINs-lab/DynMoE&type=Date)](https://star-history.com/#LINs-lab/DynMoE&Date)
+
+Code: https://github.com/LINs-lab/DynMoE