Update README.md
Browse files
README.md
CHANGED
@@ -42,4 +42,23 @@ Experimental Tagalog loras: safe or accurate outputs not guaranteed (not for pro
|
|
42 |
# hopia-3b-v0.1 (08/26/2023)
|
43 |
* Fine tuned on a small dataset of 14 items, manually edited
|
44 |
* 20 epochs
|
45 |
-
* From Open LLaMA 3b
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
42 |
# hopia-3b-v0.1 (08/26/2023)
|
43 |
* Fine tuned on a small dataset of 14 items, manually edited
|
44 |
* 20 epochs
|
45 |
+
* From Open LLaMA 3b
|
46 |
+
|
47 |
+
# llama-2-7b-tagalog-v0.3 loras (09/01/2023)
|
48 |
+
* Fine tuned on a dataset of ~1k items (Tagalog/Taglish dataset, based off Tagalog sentences augmented by LLaMA-2-13b to create a 3-turn dialogue dataset between Human and Assistant)
|
49 |
+
* 3/3a fine-tuned for 1/2 epochs
|
50 |
+
* From chat LLaMA-2-7b
|
51 |
+
* v0.3 seems to be balanced between Tagalog translation and leveraging pretrained data, more than v0.3a (which may speak more Tagalog but be less accurate or helpful); will be further curating dataset
|
52 |
+
* Lora of [chat-tagalog v0.3 (recommended)](https://huggingface.co/922-Narra/llama-2-7b-chat-tagalog-v0.3) and [chat-tagalog v0.3)](https://huggingface.co/922-Narra/llama-2-7b-chat-tagalog-v0.3a)
|
53 |
+
|
54 |
+
# llama-2-7b-tagalog-v0.3WC2 (09/01/2023)
|
55 |
+
* Fine tuned on experimental dataset of ~6k items (Tagalog/Taglish dataset, based off Tagalog sentences and Wiki entries augmented by LLaMA-2-13b to create a dialogue-QnA dataset between Human and Assistant)
|
56 |
+
* 1 epoch
|
57 |
+
* From chat LLaMA-2-7b
|
58 |
+
* Overfitted
|
59 |
+
|
60 |
+
# llama-2-13b-tagalog-v0.3 loras (09/01/2023)
|
61 |
+
* Fine tuned on dataset of ~1k items (Tagalog/Taglish dataset, based off Tagalog sentences augmented by LLaMA-2-13b to create a 3-turn dialogue dataset between Human and Assistant)
|
62 |
+
* 3/3a fine-tuned for 1 epoch, rank = 16/8
|
63 |
+
* From LLaMA-2-13b
|
64 |
+
* Less helpful results than 7b (suspecting base and dataset, trying LLaMA-2-13b chat and curated dataset for next attempts)
|