Post
1031
You are happy that
@Meta
has open-sourced Llama 3 😃...
So you jump on @HuggingFace Hub to download the new shiny Llama 3 model only to see a few quintillion Llama 3's! 🦙✨
Which one should you use? 🤔
Not all Llamas are created equal! 🦙⚖️
An absolutely crazy comparison experiment by Wolfram Ravenwolf ( @Wolfram ) might answer your question! 🧪🧙♂️
- Comprehensive assessment of Llama 3 Instruct 70B and 8B models. 📊
- Tested 20 versions across HF, GGUF, and EXL2 formats. 🔄
- Methodology: The process tested translation capabilities and cross-language understanding, using deterministic generation settings to minimize random factors. Used German data protection training exams to evaluate cross-language understanding. 🌐📝
- Best performance from EXL2 4.5bpw quant, scoring perfect in all tests. 🏆✅
- GGUF 8-bit to 4-bit quants also performed exceptionally. 🌟
- Llama 3 8B unquantized is best in its size class but not as good as 70B quants. 📏🔍
- 1-bit quantizations showed significant quality drops. ⚠️⬇️
Best models:
- turboderp/Llama-3-70B-Instruct-exl2
- casperhansen/llama-3-70b-instruct-awq
Blog: https://huggingface.co/blog/wolfram/llm-comparison-test-llama-3
So you jump on @HuggingFace Hub to download the new shiny Llama 3 model only to see a few quintillion Llama 3's! 🦙✨
Which one should you use? 🤔
Not all Llamas are created equal! 🦙⚖️
An absolutely crazy comparison experiment by Wolfram Ravenwolf ( @Wolfram ) might answer your question! 🧪🧙♂️
- Comprehensive assessment of Llama 3 Instruct 70B and 8B models. 📊
- Tested 20 versions across HF, GGUF, and EXL2 formats. 🔄
- Methodology: The process tested translation capabilities and cross-language understanding, using deterministic generation settings to minimize random factors. Used German data protection training exams to evaluate cross-language understanding. 🌐📝
- Best performance from EXL2 4.5bpw quant, scoring perfect in all tests. 🏆✅
- GGUF 8-bit to 4-bit quants also performed exceptionally. 🌟
- Llama 3 8B unquantized is best in its size class but not as good as 70B quants. 📏🔍
- 1-bit quantizations showed significant quality drops. ⚠️⬇️
Best models:
- turboderp/Llama-3-70B-Instruct-exl2
- casperhansen/llama-3-70b-instruct-awq
Blog: https://huggingface.co/blog/wolfram/llm-comparison-test-llama-3