encounter some inconsistency when i try to reproduce the result #aime #loss curve
#3
by
sqatwork
- opened
i take exactly the same settings as Bespoke-Stratos-32B, my dataset_info.json is "Sky-T1": {
"hf_hub_url": "NovaSky-AI/Sky-T1_data_17k",
"formatting": "sharegpt",
"columns": {
"messages": "conversations",
"system": "system"
},
"tags": {
"role_tag": "from",
"content_tag": "value",
"user_tag": "user",
"assistant_tag": "assistant"
}
}, but we have totally different loss curves; my traninig log is here, https://wandb.ai/shuqiatwork-minimax/huggingface?nw=nwusershuqiatwork; the loss curves are different from the one you posted. and the final result of aime is 0.26 Results:
{'aime': 0.2667}.
sqatwork
changed discussion status to
closed