File size: 2,352 Bytes
fb4aa40 80d42fb 414547b 80d42fb f825f50 80d42fb |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
---
license: other
license_name: nexusflowresearchlicense
license_link: >-
https://huggingface.co/Nexusflow/Athene-V2-Chat/resolve/main/Nexusflow_Research_License_.pdf
language:
- en
library_name: transformers
tags:
- RLHF
- Nexusflow
- Athene
- Chat Model
base_model:
- Qwen/Qwen2.5-72B-Instruct
---
# Athene-V2-Chat-72B: Rivaling GPT-4o across Benchmarks
- AWQ 4bit version of [Nexusflow/Athene-V2-Chat](https://huggingface.co/Nexusflow/Athene-V2-Chat)
- [Quantization code](https://docs.vllm.ai/en/latest/quantization/auto_awq.html)
## Eval AWQ version
Evaluation results on [ZebraLogic](https://github.com/WildEval/ZeroEval/blob/main/result_dirs/zebra-grid.summary.md)
```
│ Model │ Mode │ N_Mode │ N_Size │ Puzzle Acc │ Easy Puzzle Acc │ Hard Puzzle Acc │ Cell Acc │ No answer │ Total Puzzles │ Reason Lens │
│ o1-preview-2024-09-12 │ greedy │ single │ 1 │ 71.4 │ 98.57 │ 60.83 │ 75.14 │ 0.3 │ 1000 │ 1565.88 │
│ claude-3-5-sonnet-20241022 │ greedy │ single │ 1 │ 36.2 │ 91.07 │ 14.86 │ 54.27 │ 0 │ 1000 │ 861.18 │
│ Llama-3.1-405B-Inst-fp8@together │ greedy │ single │ 1 │ 32.6 │ 87.14 │ 11.39 │ 45.8 │ 12.5 │ 1000 │ 314.66 │
│ Athene-V2-Chat-AWQ │ greedy │ single │ 1 │ 27.8 │ 77.14 │ 8.61 │ 45.83 │ 6.4 │ 1000 │ 1785.7 │
│ Qwen2.5-72B-Instruct │ greedy │ single │ 1 │ 26.6 │ 76.43 │ 7.22 │ 40.92 │ 11.9 │ 1000 │ 1795.9 │
│ Qwen2.5-32B-Instruct │ greedy │ single │ 1 │ 26.1 │ 77.5 │ 6.11 │ 43.39 │ 6.3 │ 1000 │ 1333.07 │
│ Athene-70B │ greedy │ single │ 1 │ 16.7 │ 52.5 │ 2.78 │ 32.98 │ 21.1 │ 1000 │ 391.19 │
```
|