Hugging Face's logo

rAIfle
/
SorcererLM-8x22b-bf16

Safetensors
mixtral
Model card Files Community
3
rAIfle commited on
Commit
ec7ba65
·
verified ·
1 Parent(s): 165a631

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -5,7 +5,7 @@ base_model: alpindale/WizardLM-2-8x22B
5
 
6
  # SorcererLM-8x22b-bf16
7
 
8
- Oh boy, here we go. Low-rank LoRA on top of [WizardLM-2-8x22B](https://huggingface.co/alpindale/WizardLM-2-8x22B) trained on 2 epochs of the (cleaned & deduped) c2-logs. As far as I can tell, this is an upgrade from `WizardLM-2-8x22B` for RP purposes.
9
 
10
  ## Why LoRA?
11
 
 
5
 
6
  # SorcererLM-8x22b-bf16
7
 
8
+ Oh boy, here we go. Low-rank (`r=16, alpha=32`) LoRA on top of [WizardLM-2-8x22B](https://huggingface.co/alpindale/WizardLM-2-8x22B), trained on 2 epochs of (cleaned & deduped) c2-logs. As far as I can tell, this is an upgrade from `WizardLM-2-8x22B` for RP purposes.
9
 
10
  ## Why LoRA?
11