Update

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+images/drop-upcycling.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -2,6 +2,19 @@
 license: apache-2.0
 ---
 
+<h1 align="center">
+    <img alt="Drop-Upcycling" src="images/drop-upcycling.png"></a><br>
+<b>Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization</b><br>
+</h1>
+
+<p align="center">
+  📄 <a href="https://openreview.net/forum?id=gx1wHnf5Vp">[Paper]</a> |
+  🤗 <a href="https://huggingface.co/collections/llm-jp/drop-upcycling-674dc5be7bbb45e12a476b80">[Hugging Face]</a>
+  📁 <a href="https://gitlab.llm-jp.nii.ac.jp/datasets/llm-jp-corpus-v3">[Dataset]</a>
+  💻 <a href="https://github.com/Taishi-N324/Drop-Upcycling">[Code]</a> |
+  📊 <a href="https://wandb.ai/taishi-nakamura/Drop-Upcycling">[Log]</a>
+</p>
+
 # Model Index
 
 We provide model checkpoints for all experiments to ensure reproducibility of the results presented in Tables 1 and 2.
@@ -45,3 +58,18 @@ We provide model checkpoints for all experiments to ensure reproducibility of th
 |Japanese expert 1.5B| [Link](https://huggingface.co/llm-jp/Dense-btx-japanese-expert-1.5B) |
 |English expert 1.5B| [Link](https://huggingface.co/llm-jp/Dense-btx-english-expert-1.5B) |
 |Code expert 1.5B| [Link](https://huggingface.co/llm-jp/Dense-btx-code-expert-1.5B) |
+
+## How to cite
+
+If you find our work helpful, please feel free to cite these papers.
+
+```
+@inproceedings{
+    nakamura2025dropupcycling,
+    title={Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization},
+    author={Taishi Nakamura and Takuya Akiba and Kazuki Fujii and Yusuke Oda and Rio Yokota and Jun Suzuki},
+    booktitle={The Thirteenth International Conference on Learning Representations},
+    year={2025},
+    url={https://openreview.net/forum?id=gx1wHnf5Vp}
+}
+```