aarohanverma commited on
Commit
62fb090
·
verified ·
1 Parent(s): 0002b39

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -155,10 +155,10 @@ and tokenized with a maximum length of 512 for inputs and 256 for responses usin
155
  - **LR Scheduler:** Cosine scheduler with a warmup ratio of 10%
156
  - **Quantization:** 4-bit NF4 (with double quantization) using `torch.bfloat16`
157
  - **LoRA Parameters:**
158
- - **Rank (r):** 32
159
- - **Alpha:** 64
160
- - **Dropout:** 0.1
161
- - **Target Modules:** `["q", "v"]`
162
  - **Checkpointing:**
163
  Model saved at the end of every epoch
164
  Early stopping with a patience of 2 epochs based on evaluation loss
 
155
  - **LR Scheduler:** Cosine scheduler with a warmup ratio of 10%
156
  - **Quantization:** 4-bit NF4 (with double quantization) using `torch.bfloat16`
157
  - **LoRA Parameters:**
158
+ **Rank (r):** 32
159
+ **Alpha:** 64
160
+ **Dropout:** 0.1
161
+ **Target Modules:** `["q", "v"]`
162
  - **Checkpointing:**
163
  Model saved at the end of every epoch
164
  Early stopping with a patience of 2 epochs based on evaluation loss