Fac256xc / tools

Commit History

fix(colab): use correct param name for CLIP
badd15c

boris commited on

feat(colab): use full precision for clip & vqgan
8845d77

boris commited on

feat(colab): require less memory
5f16fb0

boris commited on

fix(train): overwrite dropout only when specified
89bc9d4

boris commited on

feat: allow eval only
65bb95f

boris commited on

feat(colab): update mini checkpoint
a4d2af8

boris commited on

feat(train): allow editing dropout during training
80d791a

boris commited on

feat(colab): require less resources
b6f5026

boris commited on

feat(colab): change defaults
3de3769

boris commited on

feat(colab): handle dalle-mega
024f8f5

boris commited on

feat: support LR offset (#174)
c3e93df
unverified

boris commited on

feat: update shampoo
79a3849

boris commited on

feat(train): allow nesterov momentum
d08bf8d

boris commited on

feat: allow weight decay
acc1a4a

boris commited on

style: reformat
dca3ada

boris commited on

feat(train): use new HF _do_init api
6b84155

boris commited on

fix: model compatible with do_init
f3a8cbb

boris commited on

fix: apply learning rate offset only when requested
c6263f3

boris commited on

feat(train): arg to offset lr for resumed runs
89b4c45

boris commited on

feat(demo): use vaild model
23c1ef6

boris commited on

feat(mega): switch to gelu
cdefdd0

boris commited on

fix: use correctly cache during inference + allow unscan (#170)
42968cf
unverified

boris commited on

fix: allow non-scanned models (#168)
8ae9176
unverified

boris commited on

feat: vmap optimizer (#166)
b993d27
unverified

boris commited on

feat(demo): use fixed commit
2f1e5d9

boris commited on

feat: scan layers + gradient checkpointing (#161)
07a6f9a
unverified

boris commited on

feat: better multi-node support (#158)
728a3c3
unverified

boris commited on

feat: update shampoo
9ecdd3f

boris commited on

feat(demo): update model
b9a1a7d

boris commited on

feat(model): allow bias (#152)
361a994
unverified

boris commited on

feat(train): google-cloud-storage is optional
02b2308

boris commited on

feat(train): rename logged config
955dc20

boris commited on

feat: add mini_glu config
a7e5050

boris commited on

feat: update mini config
d9a16f2

boris commited on

feat: implement transformer variants (#144)
542378c
unverified

boris commited on

feat(train): log norm and histograms (#143)
b7b619a
unverified

boris commited on

feat(data): super conditioning (#141)
7939874
unverified

boris commited on

feat: support pod (#139)
803ccbf
unverified

boris commited on

fix: no gradient checkpointing for new model
2e02683

boris commited on

feat: no gradient checkpointing for params init
b798ed3

boris commited on

feat: update configs
79557f9

boris commited on

fix(train): consider schedule offset
bc4734f

boris commited on

feat(train): local jax cache
9f5e879

boris commited on

feat: add bucket reference to artifact
d368fb6

boris commited on

style: lint
d5d442a

boris commited on

feat: handle gradient checkpointing
5173ec7

boris commited on

feat: load from bucket
1c4e839

boris commited on

feat(train): save to bucket
50498e6

boris commited on

feat: reduce artifact space + offset step
34cf91c

boris commited on

feat(demo): update reference
e558000

boris commited on