Update README.md

README.md

@@ -1,70 +1,109 @@
----
 base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
-datasets:
-- generator
 library_name: peft
 license: llama3.1
-tags:
-- trl
-- sft
-- generated_from_trainer
 model-index:
-- name: llama-8b-south-africa
-  results: []
----
+  - name: llama-8b-south-africa
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
-# llama-8b-south-africa
-
-This model is a fine-tuned version of [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) on the generator dataset.
-It achieves the following results on the evaluation set:
-- Loss: 1.0571
-
-## Model description
-
-More information needed
-
-## Intended uses & limitations
-
-More information needed
-
-## Training and evaluation data
-
-More information needed
-
-## Training procedure
-
-### Training hyperparameters
-
-The following hyperparameters were used during training:
-- learning_rate: 0.0002
-- train_batch_size: 4
-- eval_batch_size: 8
-- seed: 42
-- distributed_type: multi-GPU
-- gradient_accumulation_steps: 2
-- total_train_batch_size: 8
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: cosine
-- lr_scheduler_warmup_ratio: 0.1
-- num_epochs: 1
-
-### Training results
-
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 1.0959        | 0.9999 | 5596 | 1.0571          |
-
-
-### Framework versions
-
-- PEFT 0.12.0
-- Transformers 4.44.2
-- Pytorch 2.4.1+cu121
-- Datasets 3.0.0
-- Tokenizers 0.19.1
+model_description:
+  name: llama-8b-south-africa
+  description: |
+    This model is a fine-tuned version of [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) on the generator dataset.
+    [Alapa Cleaned](https://huggingface.co/datasets/yahma/alpaca-cleaned) translated into Xhose, Zulu, Tswana, Northern Sotho and Afrikaans using machine translation.
+  
+  details: |
+    The model could only be evaluated in Xhosa and Zulu due to Iroko language availability. Its aim is to show cross-lingual transfer can be achieved at a low cost. Translation cost roughly $370 per language and training cost roughly $15 using an Akash Compute Network GPU.
   
-### Terms of Use
-This model is governed by a Apache 2.0 License.
+  intended_use: This model is intended to be used for research.
+
+evaluation_results:
+  - task:
+      type: text-generation
+      name: African Language Evaluation
+    dataset:
+      name: afrimgsm_direct_xho
+      type: text-classification
+      split: test
+    metrics:
+      - name: Accuracy
+        type: accuracy
+        value: 0.02
+      - name: Dataset
+        type: dataset
+        value: MGS-Xho Direct
+
+  - task:
+      type: text-generation
+      name: African Language Evaluation
+    dataset:
+      name: afrimmlu_direct_xho
+      type: text-classification
+      split: test
+    metrics:
+      - name: Accuracy
+        type: accuracy
+        value: 0.29
+      - name: Dataset
+        type: dataset
+        value: MMLU-Xho Direct
+
+  - task:
+      type: text-generation
+      name: African Language Evaluation
+    dataset:
+      name: afrixnli_en_direct_xho
+      type: text-classification
+      split: test
+    metrics:
+      - name: Accuracy
+        type: accuracy
+        value: 0.44
+      - name: Dataset
+        type: dataset
+        value: XNLI-Xho Direct
+
+  - task:
+      type: text-generation
+      name: African Language Evaluation
+    dataset:
+      name: afrimgsm_direct_zul
+      type: text-classification
+      split: test
+    metrics:
+      - name: Accuracy
+        type: accuracy
+        value: 0.045
+      - name: Dataset
+        type: dataset
+        value: MGS-Zul Direct
+
+  - task:
+      type: text-generation
+      name: African Language Evaluation
+    dataset:
+      name: afrimmlu_direct_zul
+      type: text-classification
+      split: test
+    metrics:
+      - name: Accuracy
+        type: accuracy
+        value: 0.29
+      - name: Dataset
+        type: dataset
+        value: MMLU-Zul Direct
+
+  - task:
+      type: text-generation
+      name: African Language Evaluation
+    dataset:
+      name: afrixnli_en_direct_zul
+      type: text-classification
+      split: test
+    metrics:
+      - name: Accuracy
+        type: accuracy
+        value: 0.43
+      - name: Dataset
+        type: dataset
+        value: XNLI-Zul Direct
+
+terms_of_use: This model is governed by a Apache 2.0 License.