922CA commited on
Commit
8cb59a0
1 Parent(s): 42eef13

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -1
README.md CHANGED
@@ -42,4 +42,23 @@ Experimental Tagalog loras: safe or accurate outputs not guaranteed (not for pro
42
  # hopia-3b-v0.1 (08/26/2023)
43
  * Fine tuned on a small dataset of 14 items, manually edited
44
  * 20 epochs
45
- * From Open LLaMA 3b
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
42
  # hopia-3b-v0.1 (08/26/2023)
43
  * Fine tuned on a small dataset of 14 items, manually edited
44
  * 20 epochs
45
+ * From Open LLaMA 3b
46
+
47
+ # llama-2-7b-tagalog-v0.3 loras (09/01/2023)
48
+ * Fine tuned on a dataset of ~1k items (Tagalog/Taglish dataset, based off Tagalog sentences augmented by LLaMA-2-13b to create a 3-turn dialogue dataset between Human and Assistant)
49
+ * 3/3a fine-tuned for 1/2 epochs
50
+ * From chat LLaMA-2-7b
51
+ * v0.3 seems to be balanced between Tagalog translation and leveraging pretrained data, more than v0.3a (which may speak more Tagalog but be less accurate or helpful); will be further curating dataset
52
+ * Lora of [chat-tagalog v0.3 (recommended)](https://huggingface.co/922-Narra/llama-2-7b-chat-tagalog-v0.3) and [chat-tagalog v0.3)](https://huggingface.co/922-Narra/llama-2-7b-chat-tagalog-v0.3a)
53
+
54
+ # llama-2-7b-tagalog-v0.3WC2 (09/01/2023)
55
+ * Fine tuned on experimental dataset of ~6k items (Tagalog/Taglish dataset, based off Tagalog sentences and Wiki entries augmented by LLaMA-2-13b to create a dialogue-QnA dataset between Human and Assistant)
56
+ * 1 epoch
57
+ * From chat LLaMA-2-7b
58
+ * Overfitted
59
+
60
+ # llama-2-13b-tagalog-v0.3 loras (09/01/2023)
61
+ * Fine tuned on dataset of ~1k items (Tagalog/Taglish dataset, based off Tagalog sentences augmented by LLaMA-2-13b to create a 3-turn dialogue dataset between Human and Assistant)
62
+ * 3/3a fine-tuned for 1 epoch, rank = 16/8
63
+ * From LLaMA-2-13b
64
+ * Less helpful results than 7b (suspecting base and dataset, trying LLaMA-2-13b chat and curated dataset for next attempts)