Commit History
add back packing efficiency estimate so epochs and multi-gpu works properly (#1697)
ed8ef65
unverified
add support for rpo_alpha (#1681)
c996881
unverified
need to add back drop_last for sampler (#1676)
05b0bd0
unverified
use mixins for orpo and kto configs so they work with axolotl customizations (#1674)
f7332ac
unverified
enable loraplus setting for dpo trainer (#1646)
a27d5e1
unverified
allow report_to for multiple providers (#1647)
6299eb5
unverified
Add KTO support (#1640)
22ae21a
unverified
fixes to save on fractional save_steps (#1643)
ba45531
unverified
add save_only_model option (#1634)
702a669
unverified
FIX: max_length and max_prompt_length was not being sent to ORPOTrainer (#1584)
1e1921b
unverified
improve save callbacks (#1592)
29cf15a
unverified
FIX: TRL trainer preprocessing step was running in one process (#1583)
b9bb169
unverified
Ali Mosavian
Ali Mosavian
commited on
PoSE context length ext (#1567)
5294653
unverified
make sure everything stays in the same dtype when using dpo + FSDP (#1559)
68601ec
unverified
ORPO Trainer replacement (#1551)
7d1d22f
unverified
DBRX Model Support (#1462)
132eb74
unverified
WIP: Support table logging for mlflow, too (#1506)
057fa44
unverified
drop empty token from beginning if tokenizer has no bos_token (in the case of qwen) (#1490)
934fc85
unverified
Fix ORPO multi gpu (#1433)
34ba634
unverified
strip out hacky qlora-fsdp workarounds now that qlora-fsdp fixes are upstreamed (#1428)
2a1589f
unverified
HF / FEAT: Optimize HF tags (#1425) [skip ci]
7d55607
unverified
support galore once upstreamed into transformers (#1409)
dd449c5
unverified
fix(config): passing gradient_checkpoint_kwargs (#1412)
b1e3e1b
unverified
ORPO (#1419)
2ea70eb
unverified
FDSP + QLoRA (#1378)
9b6ee83
unverified
lora+ support (#1352)
decb66e
unverified
add lion-pytorch optimizer (#1299) [skip ci]
1648279
unverified
make mlflow optional (#1317)
5894f0e
unverified
Allow load_best_model_at_end to be configured for early stopping on custom evaluation datasets (#1291)
3c00f40
unverified
David Meikle
commited on
Add seq2seq eval benchmark callback (#1274)
5a5d474
unverified
Scheduler implementation of Continual Pre-Training of Large Language Models: How to (re)warm your model? (#1273)
8430db2
unverified
allow the optimizer prune ratio for ReLoRA to be configurable (#1287)
4b997c3
unverified
simplify haldning for newer multipack patches so they can be added in a single place (#1270)
5698943
unverified
Add more save strategies for DPO training. (#1255)
13eea21
unverified
Philip May
commited on
relora: magnitude pruning of the optimizer (#1245)
8c2e05a
unverified
support for true batches with multipack (#1230)
00568c1
unverified
Fix and document test_datasets (#1228)
5787e1a
unverified
FEAT: add tagging support to axolotl for DPOTrainer (#1209)
18f8119
unverified
precompute dpo logprobs setting and fixes (#1199) [skip ci]
33e1170
unverified
more dpo fixes for dataset loading and docs (#1185) [skip ci]
5bce45f
unverified
DPO fixes v2 (#1174)
59a31fe
unverified
Phi2 multipack (#1173)
814aee6
unverified
Add mlflow callback for pushing config to mlflow artifacts (#1125)
b8e5603
unverified
JohanWork
commited on