Transformers
Safetensors
English
yoniebans commited on
Commit
51935b5
·
verified ·
1 Parent(s): c6894e6

Update README.md

Browse files

added loss vs steps graph and minor text updates

Files changed (1) hide show
  1. README.md +7 -8
README.md CHANGED
@@ -141,7 +141,7 @@ The model was trained on a dataset consisting of pairs of natural language instr
141
 
142
  #### Training Hyperparameters
143
 
144
- - Number of train epochs: 3
145
  - Double quant: true
146
  - Quant type: nf4
147
  - Bits: 4
@@ -218,6 +218,8 @@ The model was trained on a dataset consisting of pairs of natural language instr
218
  | 29.07 | 0.138704 | 0.0353 | 9800 |
219
  | 29.66 | 0.216177 | 0.0312 | 10000 |
220
 
 
 
221
  ### Results
222
 
223
  #### Summary
@@ -235,7 +237,7 @@ Initial evaluations show promising results in generating syntactically correct S
235
  ## Citation [optional]
236
 
237
  **BibTeX:**
238
-
239
  @misc{lozhkov2024starcoder,
240
  title={StarCoder 2 and The Stack v2: The Next Generation},
241
  author={Anton Lozhkov and Raymond Li and Loubna Ben Allal and Federico Cassano and Joel Lamy-Poirier and Nouamane Tazi and Ao Tang and Dmytro Pykhtar and Jiawei Liu and Yuxiang Wei and Tianyang Liu and Max Tian and Denis Kocetkov and Arthur Zucker and Younes Belkada and Zijian Wang and Qian Liu and Dmitry Abulkhanov and Indraneil Paul and Zhuang Li and Wen-Ding Li and Megan Risdal and Jia Li and Jian Zhu and Terry Yue Zhuo and Evgenii Zheltonozhskii and Nii Osae Osae Dade and Wenhao Yu and Lucas Krauß and Naman Jain and Yixuan Su and Xuanli He and Manan Dey and Edoardo Abati and Yekun Chai and Niklas Muennighoff and Xiangru Tang and Muhtasham Oblokulov and Christopher Akiki and Marc Marone and Chenghao Mou and Mayank Mishra and Alex Gu and Binyuan Hui and Tri Dao and Armel Zebaze and Olivier Dehaene and Nicolas Patry and Canwen Xu and Julian McAuley and Han Hu and Torsten Scholak and Sebastien Paquet and Jennifer Robinson and Carolyn Jane Anderson and Nicolas Chapados and Mostofa Patwary and Nima Tajbakhsh and Yacine Jernite and Carlos Muñoz Ferrandis and Lingming Zhang and Sean Hughes and Thomas Wolf and Arjun Guha and Leandro von Werra and Harm de Vries},
@@ -244,16 +246,13 @@ Initial evaluations show promising results in generating syntactically correct S
244
  archivePrefix={arXiv},
245
  primaryClass={cs.SE}
246
  }
 
247
 
248
  **APA:**
249
 
250
- Starcoder2-3B:
251
- Author(s). (Year). Title [Software]. Available from URL
252
- BigCode Team. (2024). Starcoder2-3B [Software]. Available from https://huggingface.co/bigcode/starcoder2-3b
253
 
254
- AlfredPros Smart Contracts Instructions:
255
- Author(s). (Year). Title [Data set]. Available from URL
256
- AlfredPros. (2023). Smart Contracts Instructions [Data set]. Available from https://huggingface.co/datasets/AlfredPros/smart-contracts-instructions
257
 
258
  ## Model Card Contact
259
 
 
141
 
142
  #### Training Hyperparameters
143
 
144
+ - Number of train epochs: 30
145
  - Double quant: true
146
  - Quant type: nf4
147
  - Bits: 4
 
218
  | 29.07 | 0.138704 | 0.0353 | 9800 |
219
  | 29.66 | 0.216177 | 0.0312 | 10000 |
220
 
221
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/65e20784f70c00af9691d07b/kcg17I-kDF3j49eLo-g5B.png)
222
+
223
  ### Results
224
 
225
  #### Summary
 
237
  ## Citation [optional]
238
 
239
  **BibTeX:**
240
+ ```
241
  @misc{lozhkov2024starcoder,
242
  title={StarCoder 2 and The Stack v2: The Next Generation},
243
  author={Anton Lozhkov and Raymond Li and Loubna Ben Allal and Federico Cassano and Joel Lamy-Poirier and Nouamane Tazi and Ao Tang and Dmytro Pykhtar and Jiawei Liu and Yuxiang Wei and Tianyang Liu and Max Tian and Denis Kocetkov and Arthur Zucker and Younes Belkada and Zijian Wang and Qian Liu and Dmitry Abulkhanov and Indraneil Paul and Zhuang Li and Wen-Ding Li and Megan Risdal and Jia Li and Jian Zhu and Terry Yue Zhuo and Evgenii Zheltonozhskii and Nii Osae Osae Dade and Wenhao Yu and Lucas Krauß and Naman Jain and Yixuan Su and Xuanli He and Manan Dey and Edoardo Abati and Yekun Chai and Niklas Muennighoff and Xiangru Tang and Muhtasham Oblokulov and Christopher Akiki and Marc Marone and Chenghao Mou and Mayank Mishra and Alex Gu and Binyuan Hui and Tri Dao and Armel Zebaze and Olivier Dehaene and Nicolas Patry and Canwen Xu and Julian McAuley and Han Hu and Torsten Scholak and Sebastien Paquet and Jennifer Robinson and Carolyn Jane Anderson and Nicolas Chapados and Mostofa Patwary and Nima Tajbakhsh and Yacine Jernite and Carlos Muñoz Ferrandis and Lingming Zhang and Sean Hughes and Thomas Wolf and Arjun Guha and Leandro von Werra and Harm de Vries},
 
246
  archivePrefix={arXiv},
247
  primaryClass={cs.SE}
248
  }
249
+ ```
250
 
251
  **APA:**
252
 
253
+ Starcoder2-3B: BigCode Team. (2024). Starcoder2-3B [Software]. Available from https://huggingface.co/bigcode/starcoder2-3b
 
 
254
 
255
+ AlfredPros Smart Contracts Instructions: AlfredPros. (2023). Smart Contracts Instructions [Data set]. Available from https://huggingface.co/datasets/AlfredPros/smart-contracts-instructions
 
 
256
 
257
  ## Model Card Contact
258