VAGOsolutions
/

Llama-3-SauerkrautLM-70b-Instruct

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

DavidGF commited on Apr 29, 2024

Commit

5a48168

·

verified ·

1 Parent(s): 24fd931

Update README.md

Files changed (1) hide show

README.md +9 -8

README.md CHANGED Viewed

@@ -301,15 +301,16 @@ Llama-3-SauerkrautLM-70b-Instruct  8.6125
 **German RAG LLM Evaluation**
 ```
-|                         Task                         |Version|Metric|Value |   |Stderr|
-|------------------------------------------------------|------:|------|-----:|---|-----:|
-|all                                                   |       |acc   |0.9593|±  |0.0054|
-|community:german_rag_eval:_average:0                  |       |acc   |0.9593|±  |0.0054|
-|community:german_rag_eval:choose_context_by_question:0|      0|acc   |0.9050|±  |0.0093|
-|community:german_rag_eval:choose_question_by_context:0|      0|acc   |0.9980|±  |0.0014|
-|community:german_rag_eval:context_question_match:0    |      0|acc   |0.9790|±  |0.0045|
-|community:german_rag_eval:question_answer_match:0     |      0|acc   |0.9550|±  |0.0066|
 ```
 ## Disclaimer

 **German RAG LLM Evaluation**
+corrected result after FIX: https://github.com/huggingface/lighteval/pull/171
 ```
+|                         Task                         |Version|Metric|Value|   |Stderr|
+|------------------------------------------------------|------:|------|----:|---|-----:|
+|all                                                   |       |acc   |0.980|±  |0.0034|
+|community:german_rag_eval:_average:0                  |       |acc   |0.980|±  |0.0034|
+|community:german_rag_eval:choose_context_by_question:0|      0|acc   |0.998|±  |0.0014|
+|community:german_rag_eval:choose_question_by_context:0|      0|acc   |1.000|±  |0.0000|
+|community:german_rag_eval:context_question_match:0    |      0|acc   |0.973|±  |0.0051|
+|community:german_rag_eval:question_answer_match:0     |      0|acc   |0.949|±  |0.0070|
 ```
 ## Disclaimer