Update README.md
Browse files
README.md
CHANGED
|
@@ -45,20 +45,20 @@ Experimental Tagalog loras: safe or accurate outputs not guaranteed (not for pro
|
|
| 45 |
* From Open LLaMA 3b
|
| 46 |
|
| 47 |
# llama-2-7b-tagalog-v0.3 loras (09/01/2023)
|
| 48 |
-
* Fine tuned on a dataset of ~1k items (Tagalog/Taglish dataset, based off Tagalog sentences augmented by LLaMA-2-13b to create a 3-turn dialogue dataset between Human and Assistant)
|
| 49 |
* 3/3a fine-tuned for 1/2 epochs
|
| 50 |
* From chat LLaMA-2-7b
|
| 51 |
* v0.3 seems to be balanced between Tagalog translation and leveraging pretrained data, more than v0.3a (which may speak more Tagalog but be less accurate or helpful); will be further curating dataset
|
| 52 |
-
* Lora of [chat-tagalog v0.3 (recommended)](https://huggingface.co/922-Narra/llama-2-7b-chat-tagalog-v0.3) and [chat-tagalog v0.3
|
| 53 |
|
| 54 |
# llama-2-7b-tagalog-v0.3WC2 (09/01/2023)
|
| 55 |
* Fine tuned on experimental dataset of ~6k items (Tagalog/Taglish dataset, based off Tagalog sentences and Wiki entries augmented by LLaMA-2-13b to create a dialogue-QnA dataset between Human and Assistant)
|
| 56 |
* 1 epoch
|
| 57 |
* From chat LLaMA-2-7b
|
| 58 |
-
*
|
| 59 |
|
| 60 |
# llama-2-13b-tagalog-v0.3 loras (09/01/2023)
|
| 61 |
-
* Fine tuned on dataset of ~1k items (Tagalog/Taglish dataset, based off Tagalog sentences augmented by LLaMA-2-13b to create a 3-turn dialogue dataset between Human and Assistant)
|
| 62 |
* 3/3a fine-tuned for 1 epoch, rank = 16/8
|
| 63 |
* From LLaMA-2-13b
|
| 64 |
* Less helpful results than 7b (suspecting base and dataset, trying LLaMA-2-13b chat and curated dataset for next attempts)
|
|
|
|
| 45 |
* From Open LLaMA 3b
|
| 46 |
|
| 47 |
# llama-2-7b-tagalog-v0.3 loras (09/01/2023)
|
| 48 |
+
* Fine tuned on a dataset of ~1k items (Tagalog/Taglish dataset, based off Tagalog sentences augmented by LLaMA-2-13b base to create a 3-turn dialogue dataset between Human and Assistant)
|
| 49 |
* 3/3a fine-tuned for 1/2 epochs
|
| 50 |
* From chat LLaMA-2-7b
|
| 51 |
* v0.3 seems to be balanced between Tagalog translation and leveraging pretrained data, more than v0.3a (which may speak more Tagalog but be less accurate or helpful); will be further curating dataset
|
| 52 |
+
* Lora of [chat-tagalog v0.3 (recommended)](https://huggingface.co/922-Narra/llama-2-7b-chat-tagalog-v0.3) and [chat-tagalog v0.3](https://huggingface.co/922-Narra/llama-2-7b-chat-tagalog-v0.3a)
|
| 53 |
|
| 54 |
# llama-2-7b-tagalog-v0.3WC2 (09/01/2023)
|
| 55 |
* Fine tuned on experimental dataset of ~6k items (Tagalog/Taglish dataset, based off Tagalog sentences and Wiki entries augmented by LLaMA-2-13b to create a dialogue-QnA dataset between Human and Assistant)
|
| 56 |
* 1 epoch
|
| 57 |
* From chat LLaMA-2-7b
|
| 58 |
+
* Tends to fall into repetition loop
|
| 59 |
|
| 60 |
# llama-2-13b-tagalog-v0.3 loras (09/01/2023)
|
| 61 |
+
* Fine tuned on dataset of ~1k items (Tagalog/Taglish dataset, based off Tagalog sentences augmented by LLaMA-2-13b base to create a 3-turn dialogue dataset between Human and Assistant)
|
| 62 |
* 3/3a fine-tuned for 1 epoch, rank = 16/8
|
| 63 |
* From LLaMA-2-13b
|
| 64 |
* Less helpful results than 7b (suspecting base and dataset, trying LLaMA-2-13b chat and curated dataset for next attempts)
|