Spaces:
Running
on
CPU Upgrade
Same eval error?
Hi, I'm here with same reason like previously https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard/discussions/1089
All my mistral 7b v0.2 models failed like previous.(problem started a month ago, even model that previously was evaluated was getting error with new try)
https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/icefog72/Ice0.88-07.02-RP-dpo-merged_16bit-4_eval_request_False_float16_Original.json
https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/icefog72/Ice0.89-13.02-RP_eval_request_False_bfloat16_Original.json
https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/icefog72/Ice0.90-14.02-RP_eval_request_False_bfloat16_Original.json
https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/icefog72/Ice0.91-14.02-RP_eval_request_False_bfloat16_Original.json
Thx for looking at it.
Tried icefog72/Ice0.91-14.02-RP and it failed again.
Maybe this is a problem?
Tried eval with max_position_embeddings 4096 and still fail
Already evaluated model failed(I don't need re eval it again, it's just to show that eval is broken).
Hi @icefog72 ,
I've resubmitted all the models you mentioned. The logs show a different error than the one we discussed here. I'll review the evaluation process for your models and, if the issue is on our side, I hope you'll receive the results by the end of the week
I'm closing this issue but I'll get back to you here as soon as I get the results