Question About Benchmark Version in README
#9
by
lianghsun
- opened
Hello, cool new model—really! I wanted to ask about the benchmark on the README page. It seems to be from the pretraining version rather than the instruction version. Is this a mistake? Thanks!
Hi @lianghsun ,
The benchmarks in the README usually represent the pre-trained model's performance. Instruction-tuned models might not have separate benchmarks listed, likely because their performance can vary based on specific tasks or due to the lack of standardized evaluation metrics for instruction-following abilities.
Thank you.