transformersbook
/

codeparrot-small

Text Generation

text-generation-inference

Model card Files Files and versions Metrics Training metrics Community

lvwerra HF Staff commited on Aug 23, 2021

Commit

cd6f029

·

1 Parent(s): e3d9b66

fix typo

Files changed (1) hide show

codeparrot_training.py +3 -3

codeparrot_training.py CHANGED Viewed

@@ -123,11 +123,11 @@ config = {"train_batch_size": 12,
           "lr_scheduler_type": "cosine",
           "num_warmup_steps": 2000,
           "gradient_accumulation_steps": 1,
-          "max_train_steps": 8192,
-          "max_eval_steps": 512,
           "seq_length": 1024,
           "seed": 1,
-          "save_checkpoint_steps":512,}
 args = Namespace(**config)
 set_seed(args.seed)

           "lr_scheduler_type": "cosine",
           "num_warmup_steps": 2000,
           "gradient_accumulation_steps": 1,
+          "max_train_steps": 150_000,
+          "max_eval_steps": -1,
           "seq_length": 1024,
           "seed": 1,
+          "save_checkpoint_steps": 15_000}
 args = Namespace(**config)
 set_seed(args.seed)