File size: 411 Bytes
624db31 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
Book (사회과학, 기술과학, 철학, 법학, 예술 등) - 5000개
qlora
max_seq_length=1024
num_train_epochs=3
per_device_train_batch_size=8
gradient_accumulation_steps=32,
evaluation_strategy="steps"
eval_steps=2000,
logging_steps=25,
optim="paged_adamw_8bit",
learning_rate=2e-4,
lr_scheduler_type="cosine",
warmup_steps=10,
warmup_ratio=0.05,
report_to="tensorboard",
weight_decay=0.01,
max_steps=-1,
|