yixinsong commited on
Commit
55e4499
·
verified ·
1 Parent(s): 6f58649

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -4
README.md CHANGED
@@ -43,10 +43,13 @@ We pretrain the model on 100 billion tokens, including:
43
  Please kindly cite using the following BibTeX:
44
 
45
  ```bibtex
46
- @misc{sparsellm,
47
- title={Sparse Large Language Models with ReLU Activation},
48
- author={SpaseLLM Team},
49
- year={2023}
 
 
 
50
  }
51
  ```
52
 
 
43
  Please kindly cite using the following BibTeX:
44
 
45
  ```bibtex
46
+ @misc{zhang2024relu2,
47
+ title={ReLU$^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs},
48
+ author={Zhengyan Zhang and Yixin Song and Guanghui Yu and Xu Han and Yankai Lin and Chaojun Xiao and Chenyang Song and Zhiyuan Liu and Zeyu Mi and Maosong Sun},
49
+ year={2024},
50
+ eprint={2402.03804},
51
+ archivePrefix={arXiv},
52
+ primaryClass={cs.LG}
53
  }
54
  ```
55