olinguyen
/

Llama-3.2-1B-Instruct-orpo

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

olinguyen commited on Oct 25, 2024

Commit

d1a66ff

·

verified ·

1 Parent(s): 866a11d

Update README.md

Files changed (1) hide show

README.md +11 -8

README.md CHANGED Viewed

@@ -14,12 +14,15 @@ tags: []
 Base model (meta-llama/Llama-3.2-1B-Instruct) results:
-|Tasks |Version|Filter|n-shot|        Metric         |   |Value |   |Stderr|
-|------|------:|------|-----:|-----------------------|---|-----:|---|------|
-|ifeval|      4|none  |     0|inst_level_loose_acc   |↑  |0.6475|±  |   N/A|
-|      |       |none  |     0|inst_level_strict_acc  |↑  |0.4856|±  |   N/A|
-|      |       |none  |     0|prompt_level_loose_acc |↑  |0.5342|±  |0.0215|
-|      |       |none  |     0|prompt_level_strict_acc|↑  |0.3549|±  |0.0206|
@@ -27,12 +30,12 @@ Fine-tuned model (meta-llama/Llama-3.2-1B-Instruct using [mlabonne/orpo-dpo-mix-
 |Tasks |Version|Filter|n-shot|        Metric         |   |Value |   |Stderr|
 |------|------:|------|-----:|-----------------------|---|-----:|---|------|
 |ifeval|      4|none  |     0|inst_level_loose_acc   |↑  |0.5132|±  |   N/A|
 |      |       |none  |     0|inst_level_strict_acc  |↑  |0.4748|±  |   N/A|
 |      |       |none  |     0|prompt_level_loose_acc |↑  |0.3752|±  |0.0208|
 |      |       |none  |     0|prompt_level_strict_acc|↑  |0.3327|±  |0.0203|
-|hellaswag|      1|none  |     0|acc     |↑  |0.4357|±  |0.0049|
-|         |       |none  |     0|acc_norm|↑  |0.5872|±  |0.0049|
 ### Model Description

 Base model (meta-llama/Llama-3.2-1B-Instruct) results:
+|  Tasks  |Version|Filter|n-shot|        Metric         |   |Value |   |Stderr|
+|---------|------:|------|-----:|-----------------------|---|-----:|---|------|
+|hellaswag|      1|none  |     0|acc                    |↑  |0.4514|±  |0.0045|
+|         |       |none  |     0|acc_norm               |↑  |0.6071|±  |0.0049|
+|ifeval   |      4|none  |     0|inst_level_loose_acc   |↑  |0.5479|±  |   N/A|
+|         |       |none  |     0|inst_level_strict_acc  |↑  |0.4940|±  |   N/A|
+|         |       |none  |     0|prompt_level_loose_acc |↑  |0.4103|±  |0.0212|
+|         |       |none  |     0|prompt_level_strict_acc|↑  |0.3548|±  |0.0206|
 |Tasks |Version|Filter|n-shot|        Metric         |   |Value |   |Stderr|
 |------|------:|------|-----:|-----------------------|---|-----:|---|------|
+|hellaswag|      1|none  |     0|acc     |↑  |0.4357|±  |0.0049|
+|         |       |none  |     0|acc_norm|↑  |0.5872|±  |0.0049|
 |ifeval|      4|none  |     0|inst_level_loose_acc   |↑  |0.5132|±  |   N/A|
 |      |       |none  |     0|inst_level_strict_acc  |↑  |0.4748|±  |   N/A|
 |      |       |none  |     0|prompt_level_loose_acc |↑  |0.3752|±  |0.0208|
 |      |       |none  |     0|prompt_level_strict_acc|↑  |0.3327|±  |0.0203|
 ### Model Description