KathirKs commited on
Commit
1d10463
·
1 Parent(s): 9d77608

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +36 -0
README.md CHANGED
@@ -1,3 +1,39 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ datasets:
4
+ - vicgalle/alpaca-gpt4
5
+ language:
6
+ - en
7
  ---
8
+ This is a finetuned version microsoft phi - 2. This is finetuned using the alpaca dataset.
9
+
10
+
11
+ ## Training arguments
12
+
13
+ num_train_epochs=1
14
+
15
+ per_device_train_batch_size=1
16
+
17
+ gradient_accumulation_steps=8
18
+
19
+ optim="paged_adamw_32bit"
20
+
21
+ logging_steps=25
22
+
23
+ learning_rate=2e-4
24
+
25
+ weight_decay=0.001
26
+
27
+ max_grad_norm=0.3
28
+
29
+ warmup_ratio=0.03
30
+
31
+ lr_scheduler_type="cosine"
32
+
33
+ ## Prompt format:
34
+
35
+ Use the below format
36
+
37
+ ```
38
+ Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction:{prompt} ### {output}
39
+ ```