Upload folder using huggingface_hub

README.md

@@ -0,0 +1,202 @@
+---
+base_model: ibm-granite/granite-3.0-8b-base
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "ibm-granite/granite-3.0-8b-base",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2d0b8a29c3135c69a6829a5fa04deab765b7d8b9e46b878fa0aa6e7adf59cd3
+size 34100216

checkpoint-200/README.md

@@ -0,0 +1,202 @@
+---
+base_model: ibm-granite/granite-3.0-8b-base
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

checkpoint-200/adapter_config.json

@@ -0,0 +1,29 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "ibm-granite/granite-3.0-8b-base",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2d0b8a29c3135c69a6829a5fa04deab765b7d8b9e46b878fa0aa6e7adf59cd3
+size 34100216

checkpoint-200/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86646c26406b51452fb929acb7dade1a121d4f042a341ac0318ba994d912757d
+size 68292346

checkpoint-200/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6db1565fdf085bc8560b9b4bdd5f55abb40db42feac17284b294419663599c75
+size 14244

checkpoint-200/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:112b68374ca1c950140f0870a98c4f3bc5a36bbea3cfec174b29f0bafea1fbee
+size 1064

checkpoint-200/trainer_state.json

@@ -0,0 +1,1433 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.15891934843067143,
+  "eval_steps": 500,
+  "global_step": 200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0007945967421533572,
+      "grad_norm": Infinity,
+      "learning_rate": 0.0,
+      "loss": 2.3299,
+      "step": 1
+    },
+    {
+      "epoch": 0.0015891934843067143,
+      "grad_norm": 1.911912202835083,
+      "learning_rate": 2.0000000000000003e-06,
+      "loss": 2.3258,
+      "step": 2
+    },
+    {
+      "epoch": 0.0023837902264600714,
+      "grad_norm": 2.0545005798339844,
+      "learning_rate": 4.000000000000001e-06,
+      "loss": 2.4887,
+      "step": 3
+    },
+    {
+      "epoch": 0.0031783869686134287,
+      "grad_norm": 2.26759934425354,
+      "learning_rate": 6e-06,
+      "loss": 2.5734,
+      "step": 4
+    },
+    {
+      "epoch": 0.003972983710766786,
+      "grad_norm": 2.5033371448516846,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 2.5724,
+      "step": 5
+    },
+    {
+      "epoch": 0.004767580452920143,
+      "grad_norm": 2.255168914794922,
+      "learning_rate": 1e-05,
+      "loss": 2.3823,
+      "step": 6
+    },
+    {
+      "epoch": 0.0055621771950735005,
+      "grad_norm": 2.9506964683532715,
+      "learning_rate": 1.2e-05,
+      "loss": 2.4605,
+      "step": 7
+    },
+    {
+      "epoch": 0.006356773937226857,
+      "grad_norm": 2.6070868968963623,
+      "learning_rate": 1.4000000000000001e-05,
+      "loss": 2.5116,
+      "step": 8
+    },
+    {
+      "epoch": 0.007151370679380214,
+      "grad_norm": 2.3181769847869873,
+      "learning_rate": 1.6000000000000003e-05,
+      "loss": 2.3101,
+      "step": 9
+    },
+    {
+      "epoch": 0.007945967421533572,
+      "grad_norm": 2.6403615474700928,
+      "learning_rate": 1.8e-05,
+      "loss": 2.6095,
+      "step": 10
+    },
+    {
+      "epoch": 0.00874056416368693,
+      "grad_norm": 22.647397994995117,
+      "learning_rate": 2e-05,
+      "loss": 2.1908,
+      "step": 11
+    },
+    {
+      "epoch": 0.009535160905840286,
+      "grad_norm": 2.9705121517181396,
+      "learning_rate": 2.2000000000000003e-05,
+      "loss": 2.3585,
+      "step": 12
+    },
+    {
+      "epoch": 0.010329757647993643,
+      "grad_norm": 2.596752643585205,
+      "learning_rate": 2.4e-05,
+      "loss": 2.3992,
+      "step": 13
+    },
+    {
+      "epoch": 0.011124354390147001,
+      "grad_norm": 3.1459226608276367,
+      "learning_rate": 2.6000000000000002e-05,
+      "loss": 2.141,
+      "step": 14
+    },
+    {
+      "epoch": 0.011918951132300357,
+      "grad_norm": 3.1640381813049316,
+      "learning_rate": 2.8000000000000003e-05,
+      "loss": 2.2152,
+      "step": 15
+    },
+    {
+      "epoch": 0.012713547874453715,
+      "grad_norm": 6.219038486480713,
+      "learning_rate": 3e-05,
+      "loss": 2.5193,
+      "step": 16
+    },
+    {
+      "epoch": 0.013508144616607072,
+      "grad_norm": 2.284005880355835,
+      "learning_rate": 3.2000000000000005e-05,
+      "loss": 2.3177,
+      "step": 17
+    },
+    {
+      "epoch": 0.014302741358760428,
+      "grad_norm": 3.4323999881744385,
+      "learning_rate": 3.4000000000000007e-05,
+      "loss": 2.5677,
+      "step": 18
+    },
+    {
+      "epoch": 0.015097338100913786,
+      "grad_norm": 3.651581048965454,
+      "learning_rate": 3.6e-05,
+      "loss": 2.5273,
+      "step": 19
+    },
+    {
+      "epoch": 0.015891934843067144,
+      "grad_norm": 2.9801113605499268,
+      "learning_rate": 3.8e-05,
+      "loss": 2.4065,
+      "step": 20
+    },
+    {
+      "epoch": 0.0166865315852205,
+      "grad_norm": 2.270622491836548,
+      "learning_rate": 4e-05,
+      "loss": 2.153,
+      "step": 21
+    },
+    {
+      "epoch": 0.01748112832737386,
+      "grad_norm": 2.5372538566589355,
+      "learning_rate": 4.2e-05,
+      "loss": 2.3193,
+      "step": 22
+    },
+    {
+      "epoch": 0.018275725069527213,
+      "grad_norm": 3.8377063274383545,
+      "learning_rate": 4.4000000000000006e-05,
+      "loss": 2.2763,
+      "step": 23
+    },
+    {
+      "epoch": 0.01907032181168057,
+      "grad_norm": 2.231738805770874,
+      "learning_rate": 4.600000000000001e-05,
+      "loss": 2.0902,
+      "step": 24
+    },
+    {
+      "epoch": 0.01986491855383393,
+      "grad_norm": 2.3231687545776367,
+      "learning_rate": 4.8e-05,
+      "loss": 2.0689,
+      "step": 25
+    },
+    {
+      "epoch": 0.020659515295987287,
+      "grad_norm": 2.2548906803131104,
+      "learning_rate": 5e-05,
+      "loss": 2.0295,
+      "step": 26
+    },
+    {
+      "epoch": 0.021454112038140644,
+      "grad_norm": 3.162144899368286,
+      "learning_rate": 5.2000000000000004e-05,
+      "loss": 2.0136,
+      "step": 27
+    },
+    {
+      "epoch": 0.022248708780294002,
+      "grad_norm": 2.1308748722076416,
+      "learning_rate": 5.4000000000000005e-05,
+      "loss": 1.9394,
+      "step": 28
+    },
+    {
+      "epoch": 0.023043305522447356,
+      "grad_norm": Infinity,
+      "learning_rate": 5.4000000000000005e-05,
+      "loss": 2.4027,
+      "step": 29
+    },
+    {
+      "epoch": 0.023837902264600714,
+      "grad_norm": 2.4424943923950195,
+      "learning_rate": 5.6000000000000006e-05,
+      "loss": 1.9803,
+      "step": 30
+    },
+    {
+      "epoch": 0.02463249900675407,
+      "grad_norm": 3.4497978687286377,
+      "learning_rate": 5.8e-05,
+      "loss": 1.997,
+      "step": 31
+    },
+    {
+      "epoch": 0.02542709574890743,
+      "grad_norm": 4.39043664932251,
+      "learning_rate": 6e-05,
+      "loss": 1.6071,
+      "step": 32
+    },
+    {
+      "epoch": 0.026221692491060787,
+      "grad_norm": 2.201096534729004,
+      "learning_rate": 6.2e-05,
+      "loss": 1.7088,
+      "step": 33
+    },
+    {
+      "epoch": 0.027016289233214145,
+      "grad_norm": 2.6511332988739014,
+      "learning_rate": 6.400000000000001e-05,
+      "loss": 1.7083,
+      "step": 34
+    },
+    {
+      "epoch": 0.027810885975367503,
+      "grad_norm": 2.4984240531921387,
+      "learning_rate": 6.6e-05,
+      "loss": 1.8376,
+      "step": 35
+    },
+    {
+      "epoch": 0.028605482717520857,
+      "grad_norm": 2.8109183311462402,
+      "learning_rate": 6.800000000000001e-05,
+      "loss": 1.6516,
+      "step": 36
+    },
+    {
+      "epoch": 0.029400079459674214,
+      "grad_norm": 2.5620100498199463,
+      "learning_rate": 7e-05,
+      "loss": 1.647,
+      "step": 37
+    },
+    {
+      "epoch": 0.030194676201827572,
+      "grad_norm": 4.938408851623535,
+      "learning_rate": 7.2e-05,
+      "loss": 1.9757,
+      "step": 38
+    },
+    {
+      "epoch": 0.03098927294398093,
+      "grad_norm": 3.667281150817871,
+      "learning_rate": 7.4e-05,
+      "loss": 1.6447,
+      "step": 39
+    },
+    {
+      "epoch": 0.03178386968613429,
+      "grad_norm": 2.729938507080078,
+      "learning_rate": 7.6e-05,
+      "loss": 1.2727,
+      "step": 40
+    },
+    {
+      "epoch": 0.03257846642828764,
+      "grad_norm": 38.9461555480957,
+      "learning_rate": 7.800000000000001e-05,
+      "loss": 1.4276,
+      "step": 41
+    },
+    {
+      "epoch": 0.033373063170441,
+      "grad_norm": 2.9990055561065674,
+      "learning_rate": 8e-05,
+      "loss": 1.3817,
+      "step": 42
+    },
+    {
+      "epoch": 0.03416765991259436,
+      "grad_norm": 4.270920276641846,
+      "learning_rate": 8.2e-05,
+      "loss": 1.3592,
+      "step": 43
+    },
+    {
+      "epoch": 0.03496225665474772,
+      "grad_norm": 3.589218854904175,
+      "learning_rate": 8.4e-05,
+      "loss": 1.4024,
+      "step": 44
+    },
+    {
+      "epoch": 0.03575685339690107,
+      "grad_norm": 3.490218162536621,
+      "learning_rate": 8.6e-05,
+      "loss": 1.5259,
+      "step": 45
+    },
+    {
+      "epoch": 0.03655145013905443,
+      "grad_norm": 3.454580307006836,
+      "learning_rate": 8.800000000000001e-05,
+      "loss": 1.3637,
+      "step": 46
+    },
+    {
+      "epoch": 0.03734604688120779,
+      "grad_norm": 12.431150436401367,
+      "learning_rate": 9e-05,
+      "loss": 1.3291,
+      "step": 47
+    },
+    {
+      "epoch": 0.03814064362336114,
+      "grad_norm": 3.1810104846954346,
+      "learning_rate": 9.200000000000001e-05,
+      "loss": 1.1738,
+      "step": 48
+    },
+    {
+      "epoch": 0.038935240365514504,
+      "grad_norm": 2.8327722549438477,
+      "learning_rate": 9.4e-05,
+      "loss": 1.2161,
+      "step": 49
+    },
+    {
+      "epoch": 0.03972983710766786,
+      "grad_norm": 2.5604488849639893,
+      "learning_rate": 9.6e-05,
+      "loss": 1.0742,
+      "step": 50
+    },
+    {
+      "epoch": 0.04052443384982122,
+      "grad_norm": NaN,
+      "learning_rate": 9.6e-05,
+      "loss": 1.1061,
+      "step": 51
+    },
+    {
+      "epoch": 0.04131903059197457,
+      "grad_norm": 2.889859199523926,
+      "learning_rate": 9.8e-05,
+      "loss": 1.2023,
+      "step": 52
+    },
+    {
+      "epoch": 0.04211362733412793,
+      "grad_norm": 3.6567516326904297,
+      "learning_rate": 0.0001,
+      "loss": 1.2181,
+      "step": 53
+    },
+    {
+      "epoch": 0.04290822407628129,
+      "grad_norm": 3.8911192417144775,
+      "learning_rate": 0.00010200000000000001,
+      "loss": 1.2285,
+      "step": 54
+    },
+    {
+      "epoch": 0.04370282081843464,
+      "grad_norm": 4.965387344360352,
+      "learning_rate": 0.00010400000000000001,
+      "loss": 1.3265,
+      "step": 55
+    },
+    {
+      "epoch": 0.044497417560588004,
+      "grad_norm": 2.449645757675171,
+      "learning_rate": 0.00010600000000000002,
+      "loss": 1.0391,
+      "step": 56
+    },
+    {
+      "epoch": 0.04529201430274136,
+      "grad_norm": NaN,
+      "learning_rate": 0.00010600000000000002,
+      "loss": 1.429,
+      "step": 57
+    },
+    {
+      "epoch": 0.04608661104489471,
+      "grad_norm": 3.373978853225708,
+      "learning_rate": 0.00010800000000000001,
+      "loss": 1.1097,
+      "step": 58
+    },
+    {
+      "epoch": 0.046881207787048074,
+      "grad_norm": 3.261880874633789,
+      "learning_rate": 0.00011000000000000002,
+      "loss": 1.137,
+      "step": 59
+    },
+    {
+      "epoch": 0.04767580452920143,
+      "grad_norm": 2.4367783069610596,
+      "learning_rate": 0.00011200000000000001,
+      "loss": 1.0847,
+      "step": 60
+    },
+    {
+      "epoch": 0.04847040127135479,
+      "grad_norm": 2.794142007827759,
+      "learning_rate": 0.00011399999999999999,
+      "loss": 1.1413,
+      "step": 61
+    },
+    {
+      "epoch": 0.04926499801350814,
+      "grad_norm": 3.1448540687561035,
+      "learning_rate": 0.000116,
+      "loss": 0.9289,
+      "step": 62
+    },
+    {
+      "epoch": 0.050059594755661505,
+      "grad_norm": 2.2872281074523926,
+      "learning_rate": 0.000118,
+      "loss": 1.0671,
+      "step": 63
+    },
+    {
+      "epoch": 0.05085419149781486,
+      "grad_norm": 3.0548996925354004,
+      "learning_rate": 0.00012,
+      "loss": 1.0712,
+      "step": 64
+    },
+    {
+      "epoch": 0.05164878823996821,
+      "grad_norm": 2.4567928314208984,
+      "learning_rate": 0.000122,
+      "loss": 0.901,
+      "step": 65
+    },
+    {
+      "epoch": 0.052443384982121574,
+      "grad_norm": 2.2397348880767822,
+      "learning_rate": 0.000124,
+      "loss": 1.069,
+      "step": 66
+    },
+    {
+      "epoch": 0.05323798172427493,
+      "grad_norm": 3.3711724281311035,
+      "learning_rate": 0.000126,
+      "loss": 1.2578,
+      "step": 67
+    },
+    {
+      "epoch": 0.05403257846642829,
+      "grad_norm": 2.211885690689087,
+      "learning_rate": 0.00012800000000000002,
+      "loss": 0.826,
+      "step": 68
+    },
+    {
+      "epoch": 0.054827175208581644,
+      "grad_norm": 2.015852451324463,
+      "learning_rate": 0.00013000000000000002,
+      "loss": 0.743,
+      "step": 69
+    },
+    {
+      "epoch": 0.055621771950735005,
+      "grad_norm": 2.225627899169922,
+      "learning_rate": 0.000132,
+      "loss": 0.8531,
+      "step": 70
+    },
+    {
+      "epoch": 0.05641636869288836,
+      "grad_norm": 2.7798657417297363,
+      "learning_rate": 0.000134,
+      "loss": 0.8408,
+      "step": 71
+    },
+    {
+      "epoch": 0.057210965435041714,
+      "grad_norm": 18.012056350708008,
+      "learning_rate": 0.00013600000000000003,
+      "loss": 1.058,
+      "step": 72
+    },
+    {
+      "epoch": 0.058005562177195075,
+      "grad_norm": 2.5155575275421143,
+      "learning_rate": 0.000138,
+      "loss": 0.8449,
+      "step": 73
+    },
+    {
+      "epoch": 0.05880015891934843,
+      "grad_norm": 2.9396917819976807,
+      "learning_rate": 0.00014,
+      "loss": 0.8726,
+      "step": 74
+    },
+    {
+      "epoch": 0.05959475566150179,
+      "grad_norm": 2.8626365661621094,
+      "learning_rate": 0.000142,
+      "loss": 0.8525,
+      "step": 75
+    },
+    {
+      "epoch": 0.060389352403655144,
+      "grad_norm": 2.85284686088562,
+      "learning_rate": 0.000144,
+      "loss": 0.8809,
+      "step": 76
+    },
+    {
+      "epoch": 0.061183949145808506,
+      "grad_norm": 2.2844254970550537,
+      "learning_rate": 0.000146,
+      "loss": 0.7768,
+      "step": 77
+    },
+    {
+      "epoch": 0.06197854588796186,
+      "grad_norm": 2.318669080734253,
+      "learning_rate": 0.000148,
+      "loss": 0.9564,
+      "step": 78
+    },
+    {
+      "epoch": 0.06277314263011521,
+      "grad_norm": 2.5069074630737305,
+      "learning_rate": 0.00015000000000000001,
+      "loss": 0.875,
+      "step": 79
+    },
+    {
+      "epoch": 0.06356773937226858,
+      "grad_norm": 2.549753189086914,
+      "learning_rate": 0.000152,
+      "loss": 0.8328,
+      "step": 80
+    },
+    {
+      "epoch": 0.06436233611442194,
+      "grad_norm": 2.4056267738342285,
+      "learning_rate": 0.000154,
+      "loss": 0.8967,
+      "step": 81
+    },
+    {
+      "epoch": 0.06515693285657528,
+      "grad_norm": 4.7111406326293945,
+      "learning_rate": 0.00015600000000000002,
+      "loss": 1.1932,
+      "step": 82
+    },
+    {
+      "epoch": 0.06595152959872864,
+      "grad_norm": 6.380517482757568,
+      "learning_rate": 0.00015800000000000002,
+      "loss": 0.7799,
+      "step": 83
+    },
+    {
+      "epoch": 0.066746126340882,
+      "grad_norm": 2.5293846130371094,
+      "learning_rate": 0.00016,
+      "loss": 0.788,
+      "step": 84
+    },
+    {
+      "epoch": 0.06754072308303535,
+      "grad_norm": 2.2254135608673096,
+      "learning_rate": 0.000162,
+      "loss": 0.72,
+      "step": 85
+    },
+    {
+      "epoch": 0.06833531982518871,
+      "grad_norm": 36.8549919128418,
+      "learning_rate": 0.000164,
+      "loss": 0.9031,
+      "step": 86
+    },
+    {
+      "epoch": 0.06912991656734208,
+      "grad_norm": 3.616895914077759,
+      "learning_rate": 0.000166,
+      "loss": 0.8132,
+      "step": 87
+    },
+    {
+      "epoch": 0.06992451330949544,
+      "grad_norm": 2.306208848953247,
+      "learning_rate": 0.000168,
+      "loss": 1.0545,
+      "step": 88
+    },
+    {
+      "epoch": 0.07071911005164878,
+      "grad_norm": 3.26766300201416,
+      "learning_rate": 0.00017,
+      "loss": 0.877,
+      "step": 89
+    },
+    {
+      "epoch": 0.07151370679380215,
+      "grad_norm": 2.4347503185272217,
+      "learning_rate": 0.000172,
+      "loss": 0.8697,
+      "step": 90
+    },
+    {
+      "epoch": 0.0723083035359555,
+      "grad_norm": 4.748445987701416,
+      "learning_rate": 0.000174,
+      "loss": 1.1449,
+      "step": 91
+    },
+    {
+      "epoch": 0.07310290027810885,
+      "grad_norm": 15.90105152130127,
+      "learning_rate": 0.00017600000000000002,
+      "loss": 0.9489,
+      "step": 92
+    },
+    {
+      "epoch": 0.07389749702026222,
+      "grad_norm": 16.799253463745117,
+      "learning_rate": 0.00017800000000000002,
+      "loss": 1.0519,
+      "step": 93
+    },
+    {
+      "epoch": 0.07469209376241558,
+      "grad_norm": 8.348183631896973,
+      "learning_rate": 0.00018,
+      "loss": 1.1629,
+      "step": 94
+    },
+    {
+      "epoch": 0.07548669050456894,
+      "grad_norm": 3.1119446754455566,
+      "learning_rate": 0.000182,
+      "loss": 0.9083,
+      "step": 95
+    },
+    {
+      "epoch": 0.07628128724672228,
+      "grad_norm": 2.9583566188812256,
+      "learning_rate": 0.00018400000000000003,
+      "loss": 0.9221,
+      "step": 96
+    },
+    {
+      "epoch": 0.07707588398887565,
+      "grad_norm": 6.272480010986328,
+      "learning_rate": 0.00018600000000000002,
+      "loss": 0.7889,
+      "step": 97
+    },
+    {
+      "epoch": 0.07787048073102901,
+      "grad_norm": 186.46322631835938,
+      "learning_rate": 0.000188,
+      "loss": 0.9363,
+      "step": 98
+    },
+    {
+      "epoch": 0.07866507747318235,
+      "grad_norm": 2.471153974533081,
+      "learning_rate": 0.00019,
+      "loss": 0.8514,
+      "step": 99
+    },
+    {
+      "epoch": 0.07945967421533572,
+      "grad_norm": 2.3059628009796143,
+      "learning_rate": 0.000192,
+      "loss": 0.8168,
+      "step": 100
+    },
+    {
+      "epoch": 0.08025427095748908,
+      "grad_norm": 5.590695858001709,
+      "learning_rate": 0.000194,
+      "loss": 0.9188,
+      "step": 101
+    },
+    {
+      "epoch": 0.08104886769964244,
+      "grad_norm": 3.5238022804260254,
+      "learning_rate": 0.000196,
+      "loss": 1.1099,
+      "step": 102
+    },
+    {
+      "epoch": 0.08184346444179579,
+      "grad_norm": 2.4687702655792236,
+      "learning_rate": 0.00019800000000000002,
+      "loss": 0.9437,
+      "step": 103
+    },
+    {
+      "epoch": 0.08263806118394915,
+      "grad_norm": 2.516974687576294,
+      "learning_rate": 0.0002,
+      "loss": 0.7963,
+      "step": 104
+    },
+    {
+      "epoch": 0.08343265792610251,
+      "grad_norm": 2.3318846225738525,
+      "learning_rate": 0.00019800000000000002,
+      "loss": 0.9484,
+      "step": 105
+    },
+    {
+      "epoch": 0.08422725466825585,
+      "grad_norm": 7.662753105163574,
+      "learning_rate": 0.000196,
+      "loss": 0.9288,
+      "step": 106
+    },
+    {
+      "epoch": 0.08502185141040922,
+      "grad_norm": 2.047351360321045,
+      "learning_rate": 0.000194,
+      "loss": 0.7647,
+      "step": 107
+    },
+    {
+      "epoch": 0.08581644815256258,
+      "grad_norm": 2.478816509246826,
+      "learning_rate": 0.000192,
+      "loss": 0.8114,
+      "step": 108
+    },
+    {
+      "epoch": 0.08661104489471594,
+      "grad_norm": 2.1056058406829834,
+      "learning_rate": 0.00019,
+      "loss": 0.8324,
+      "step": 109
+    },
+    {
+      "epoch": 0.08740564163686929,
+      "grad_norm": 1.7560831308364868,
+      "learning_rate": 0.000188,
+      "loss": 0.6646,
+      "step": 110
+    },
+    {
+      "epoch": 0.08820023837902265,
+      "grad_norm": 2.5464038848876953,
+      "learning_rate": 0.00018600000000000002,
+      "loss": 0.9712,
+      "step": 111
+    },
+    {
+      "epoch": 0.08899483512117601,
+      "grad_norm": 2.308013916015625,
+      "learning_rate": 0.00018400000000000003,
+      "loss": 0.8075,
+      "step": 112
+    },
+    {
+      "epoch": 0.08978943186332936,
+      "grad_norm": 2.0754380226135254,
+      "learning_rate": 0.000182,
+      "loss": 0.8554,
+      "step": 113
+    },
+    {
+      "epoch": 0.09058402860548272,
+      "grad_norm": 2.6636457443237305,
+      "learning_rate": 0.00018,
+      "loss": 0.88,
+      "step": 114
+    },
+    {
+      "epoch": 0.09137862534763608,
+      "grad_norm": 2.1432785987854004,
+      "learning_rate": 0.00017800000000000002,
+      "loss": 0.8459,
+      "step": 115
+    },
+    {
+      "epoch": 0.09217322208978942,
+      "grad_norm": 2.042858600616455,
+      "learning_rate": 0.00017600000000000002,
+      "loss": 0.616,
+      "step": 116
+    },
+    {
+      "epoch": 0.09296781883194279,
+      "grad_norm": 2.1585450172424316,
+      "learning_rate": 0.000174,
+      "loss": 0.8856,
+      "step": 117
+    },
+    {
+      "epoch": 0.09376241557409615,
+      "grad_norm": 103.10526275634766,
+      "learning_rate": 0.000172,
+      "loss": 0.8779,
+      "step": 118
+    },
+    {
+      "epoch": 0.09455701231624951,
+      "grad_norm": 124.418212890625,
+      "learning_rate": 0.00017,
+      "loss": 0.9904,
+      "step": 119
+    },
+    {
+      "epoch": 0.09535160905840286,
+      "grad_norm": 2.8180806636810303,
+      "learning_rate": 0.000168,
+      "loss": 0.9871,
+      "step": 120
+    },
+    {
+      "epoch": 0.09614620580055622,
+      "grad_norm": 2.0952980518341064,
+      "learning_rate": 0.000166,
+      "loss": 0.7901,
+      "step": 121
+    },
+    {
+      "epoch": 0.09694080254270958,
+      "grad_norm": 2.030365228652954,
+      "learning_rate": 0.000164,
+      "loss": 0.8494,
+      "step": 122
+    },
+    {
+      "epoch": 0.09773539928486293,
+      "grad_norm": 1.9386011362075806,
+      "learning_rate": 0.000162,
+      "loss": 0.8348,
+      "step": 123
+    },
+    {
+      "epoch": 0.09852999602701629,
+      "grad_norm": 2.730750322341919,
+      "learning_rate": 0.00016,
+      "loss": 0.8657,
+      "step": 124
+    },
+    {
+      "epoch": 0.09932459276916965,
+      "grad_norm": 2.068394899368286,
+      "learning_rate": 0.00015800000000000002,
+      "loss": 0.9055,
+      "step": 125
+    },
+    {
+      "epoch": 0.10011918951132301,
+      "grad_norm": 3.9055023193359375,
+      "learning_rate": 0.00015600000000000002,
+      "loss": 1.2512,
+      "step": 126
+    },
+    {
+      "epoch": 0.10091378625347636,
+      "grad_norm": 2.002406597137451,
+      "learning_rate": 0.000154,
+      "loss": 0.9051,
+      "step": 127
+    },
+    {
+      "epoch": 0.10170838299562972,
+      "grad_norm": 1.9883900880813599,
+      "learning_rate": 0.000152,
+      "loss": 0.681,
+      "step": 128
+    },
+    {
+      "epoch": 0.10250297973778308,
+      "grad_norm": 2.800804376602173,
+      "learning_rate": 0.00015000000000000001,
+      "loss": 0.8668,
+      "step": 129
+    },
+    {
+      "epoch": 0.10329757647993643,
+      "grad_norm": 1.7889888286590576,
+      "learning_rate": 0.000148,
+      "loss": 0.6518,
+      "step": 130
+    },
+    {
+      "epoch": 0.10409217322208979,
+      "grad_norm": 2.3057503700256348,
+      "learning_rate": 0.000146,
+      "loss": 0.904,
+      "step": 131
+    },
+    {
+      "epoch": 0.10488676996424315,
+      "grad_norm": 1.8843817710876465,
+      "learning_rate": 0.000144,
+      "loss": 0.6771,
+      "step": 132
+    },
+    {
+      "epoch": 0.10568136670639651,
+      "grad_norm": 1.7966408729553223,
+      "learning_rate": 0.000142,
+      "loss": 0.8351,
+      "step": 133
+    },
+    {
+      "epoch": 0.10647596344854986,
+      "grad_norm": 1.9850845336914062,
+      "learning_rate": 0.00014,
+      "loss": 0.7157,
+      "step": 134
+    },
+    {
+      "epoch": 0.10727056019070322,
+      "grad_norm": 1.807133674621582,
+      "learning_rate": 0.000138,
+      "loss": 0.8309,
+      "step": 135
+    },
+    {
+      "epoch": 0.10806515693285658,
+      "grad_norm": 1.8250856399536133,
+      "learning_rate": 0.00013600000000000003,
+      "loss": 0.7187,
+      "step": 136
+    },
+    {
+      "epoch": 0.10885975367500993,
+      "grad_norm": 2.611520290374756,
+      "learning_rate": 0.000134,
+      "loss": 0.7643,
+      "step": 137
+    },
+    {
+      "epoch": 0.10965435041716329,
+      "grad_norm": 1.7620738744735718,
+      "learning_rate": 0.000132,
+      "loss": 0.6759,
+      "step": 138
+    },
+    {
+      "epoch": 0.11044894715931665,
+      "grad_norm": 1.880923867225647,
+      "learning_rate": 0.00013000000000000002,
+      "loss": 0.8368,
+      "step": 139
+    },
+    {
+      "epoch": 0.11124354390147001,
+      "grad_norm": 2.0019538402557373,
+      "learning_rate": 0.00012800000000000002,
+      "loss": 0.8729,
+      "step": 140
+    },
+    {
+      "epoch": 0.11203814064362336,
+      "grad_norm": 1.7756866216659546,
+      "learning_rate": 0.000126,
+      "loss": 0.667,
+      "step": 141
+    },
+    {
+      "epoch": 0.11283273738577672,
+      "grad_norm": 1.967138648033142,
+      "learning_rate": 0.000124,
+      "loss": 1.0259,
+      "step": 142
+    },
+    {
+      "epoch": 0.11362733412793008,
+      "grad_norm": 1.5994105339050293,
+      "learning_rate": 0.000122,
+      "loss": 0.6227,
+      "step": 143
+    },
+    {
+      "epoch": 0.11442193087008343,
+      "grad_norm": 1.6467610597610474,
+      "learning_rate": 0.00012,
+      "loss": 0.5927,
+      "step": 144
+    },
+    {
+      "epoch": 0.11521652761223679,
+      "grad_norm": 1.620017170906067,
+      "learning_rate": 0.000118,
+      "loss": 0.5825,
+      "step": 145
+    },
+    {
+      "epoch": 0.11601112435439015,
+      "grad_norm": 1.9276095628738403,
+      "learning_rate": 0.000116,
+      "loss": 0.6245,
+      "step": 146
+    },
+    {
+      "epoch": 0.11680572109654351,
+      "grad_norm": 1.8510079383850098,
+      "learning_rate": 0.00011399999999999999,
+      "loss": 0.8246,
+      "step": 147
+    },
+    {
+      "epoch": 0.11760031783869686,
+      "grad_norm": 1.917414903640747,
+      "learning_rate": 0.00011200000000000001,
+      "loss": 0.7777,
+      "step": 148
+    },
+    {
+      "epoch": 0.11839491458085022,
+      "grad_norm": 2.138396978378296,
+      "learning_rate": 0.00011000000000000002,
+      "loss": 0.6388,
+      "step": 149
+    },
+    {
+      "epoch": 0.11918951132300358,
+      "grad_norm": 2.658886194229126,
+      "learning_rate": 0.00010800000000000001,
+      "loss": 0.9487,
+      "step": 150
+    },
+    {
+      "epoch": 0.11998410806515693,
+      "grad_norm": 2.1239211559295654,
+      "learning_rate": 0.00010600000000000002,
+      "loss": 0.6178,
+      "step": 151
+    },
+    {
+      "epoch": 0.12077870480731029,
+      "grad_norm": 2.7476749420166016,
+      "learning_rate": 0.00010400000000000001,
+      "loss": 0.798,
+      "step": 152
+    },
+    {
+      "epoch": 0.12157330154946365,
+      "grad_norm": 1.8282620906829834,
+      "learning_rate": 0.00010200000000000001,
+      "loss": 0.6655,
+      "step": 153
+    },
+    {
+      "epoch": 0.12236789829161701,
+      "grad_norm": 2.5939972400665283,
+      "learning_rate": 0.0001,
+      "loss": 0.7459,
+      "step": 154
+    },
+    {
+      "epoch": 0.12316249503377036,
+      "grad_norm": 1.6962834596633911,
+      "learning_rate": 9.8e-05,
+      "loss": 0.655,
+      "step": 155
+    },
+    {
+      "epoch": 0.12395709177592372,
+      "grad_norm": 1.7708353996276855,
+      "learning_rate": 9.6e-05,
+      "loss": 0.5709,
+      "step": 156
+    },
+    {
+      "epoch": 0.12475168851807708,
+      "grad_norm": 2.2659738063812256,
+      "learning_rate": 9.4e-05,
+      "loss": 0.7079,
+      "step": 157
+    },
+    {
+      "epoch": 0.12554628526023043,
+      "grad_norm": 2.2045202255249023,
+      "learning_rate": 9.200000000000001e-05,
+      "loss": 0.9584,
+      "step": 158
+    },
+    {
+      "epoch": 0.1263408820023838,
+      "grad_norm": 2.035980701446533,
+      "learning_rate": 9e-05,
+      "loss": 0.7845,
+      "step": 159
+    },
+    {
+      "epoch": 0.12713547874453715,
+      "grad_norm": 2.0655910968780518,
+      "learning_rate": 8.800000000000001e-05,
+      "loss": 0.7504,
+      "step": 160
+    },
+    {
+      "epoch": 0.1279300754866905,
+      "grad_norm": 2.151850461959839,
+      "learning_rate": 8.6e-05,
+      "loss": 0.7934,
+      "step": 161
+    },
+    {
+      "epoch": 0.12872467222884387,
+      "grad_norm": 3.585190534591675,
+      "learning_rate": 8.4e-05,
+      "loss": 0.6943,
+      "step": 162
+    },
+    {
+      "epoch": 0.12951926897099722,
+      "grad_norm": 2.1465415954589844,
+      "learning_rate": 8.2e-05,
+      "loss": 0.7167,
+      "step": 163
+    },
+    {
+      "epoch": 0.13031386571315057,
+      "grad_norm": 2.2896647453308105,
+      "learning_rate": 8e-05,
+      "loss": 0.7453,
+      "step": 164
+    },
+    {
+      "epoch": 0.13110846245530394,
+      "grad_norm": 1.584289312362671,
+      "learning_rate": 7.800000000000001e-05,
+      "loss": 0.474,
+      "step": 165
+    },
+    {
+      "epoch": 0.1319030591974573,
+      "grad_norm": 2.6200954914093018,
+      "learning_rate": 7.6e-05,
+      "loss": 1.0956,
+      "step": 166
+    },
+    {
+      "epoch": 0.13269765593961064,
+      "grad_norm": 1.9935890436172485,
+      "learning_rate": 7.4e-05,
+      "loss": 0.6213,
+      "step": 167
+    },
+    {
+      "epoch": 0.133492252681764,
+      "grad_norm": 2.0985817909240723,
+      "learning_rate": 7.2e-05,
+      "loss": 0.847,
+      "step": 168
+    },
+    {
+      "epoch": 0.13428684942391736,
+      "grad_norm": 2.132389545440674,
+      "learning_rate": 7e-05,
+      "loss": 0.8484,
+      "step": 169
+    },
+    {
+      "epoch": 0.1350814461660707,
+      "grad_norm": 1.9410291910171509,
+      "learning_rate": 6.800000000000001e-05,
+      "loss": 0.6914,
+      "step": 170
+    },
+    {
+      "epoch": 0.13587604290822408,
+      "grad_norm": 1.9908583164215088,
+      "learning_rate": 6.6e-05,
+      "loss": 0.7885,
+      "step": 171
+    },
+    {
+      "epoch": 0.13667063965037743,
+      "grad_norm": 2.510380744934082,
+      "learning_rate": 6.400000000000001e-05,
+      "loss": 0.836,
+      "step": 172
+    },
+    {
+      "epoch": 0.1374652363925308,
+      "grad_norm": 1.9242165088653564,
+      "learning_rate": 6.2e-05,
+      "loss": 0.7703,
+      "step": 173
+    },
+    {
+      "epoch": 0.13825983313468415,
+      "grad_norm": 1.9182121753692627,
+      "learning_rate": 6e-05,
+      "loss": 0.5647,
+      "step": 174
+    },
+    {
+      "epoch": 0.1390544298768375,
+      "grad_norm": 1.8557018041610718,
+      "learning_rate": 5.8e-05,
+      "loss": 0.659,
+      "step": 175
+    },
+    {
+      "epoch": 0.13984902661899087,
+      "grad_norm": 2.031205892562866,
+      "learning_rate": 5.6000000000000006e-05,
+      "loss": 0.4815,
+      "step": 176
+    },
+    {
+      "epoch": 0.14064362336114422,
+      "grad_norm": 1.851654291152954,
+      "learning_rate": 5.4000000000000005e-05,
+      "loss": 0.6417,
+      "step": 177
+    },
+    {
+      "epoch": 0.14143822010329757,
+      "grad_norm": 2.5643835067749023,
+      "learning_rate": 5.2000000000000004e-05,
+      "loss": 0.7067,
+      "step": 178
+    },
+    {
+      "epoch": 0.14223281684545094,
+      "grad_norm": 2.3075978755950928,
+      "learning_rate": 5e-05,
+      "loss": 0.7721,
+      "step": 179
+    },
+    {
+      "epoch": 0.1430274135876043,
+      "grad_norm": 1.8168667554855347,
+      "learning_rate": 4.8e-05,
+      "loss": 0.8236,
+      "step": 180
+    },
+    {
+      "epoch": 0.14382201032975764,
+      "grad_norm": 2.1798055171966553,
+      "learning_rate": 4.600000000000001e-05,
+      "loss": 0.845,
+      "step": 181
+    },
+    {
+      "epoch": 0.144616607071911,
+      "grad_norm": 1.837917685508728,
+      "learning_rate": 4.4000000000000006e-05,
+      "loss": 0.6554,
+      "step": 182
+    },
+    {
+      "epoch": 0.14541120381406436,
+      "grad_norm": 1.6694142818450928,
+      "learning_rate": 4.2e-05,
+      "loss": 0.5733,
+      "step": 183
+    },
+    {
+      "epoch": 0.1462058005562177,
+      "grad_norm": 1.8454749584197998,
+      "learning_rate": 4e-05,
+      "loss": 0.7121,
+      "step": 184
+    },
+    {
+      "epoch": 0.14700039729837108,
+      "grad_norm": 1.826249122619629,
+      "learning_rate": 3.8e-05,
+      "loss": 0.7163,
+      "step": 185
+    },
+    {
+      "epoch": 0.14779499404052443,
+      "grad_norm": 1.8959565162658691,
+      "learning_rate": 3.6e-05,
+      "loss": 0.8287,
+      "step": 186
+    },
+    {
+      "epoch": 0.14858959078267778,
+      "grad_norm": 2.27337908744812,
+      "learning_rate": 3.4000000000000007e-05,
+      "loss": 0.8789,
+      "step": 187
+    },
+    {
+      "epoch": 0.14938418752483115,
+      "grad_norm": 1.5930019617080688,
+      "learning_rate": 3.2000000000000005e-05,
+      "loss": 0.5562,
+      "step": 188
+    },
+    {
+      "epoch": 0.1501787842669845,
+      "grad_norm": 1.6384660005569458,
+      "learning_rate": 3e-05,
+      "loss": 0.7357,
+      "step": 189
+    },
+    {
+      "epoch": 0.15097338100913787,
+      "grad_norm": 1.683895468711853,
+      "learning_rate": 2.8000000000000003e-05,
+      "loss": 0.6649,
+      "step": 190
+    },
+    {
+      "epoch": 0.15176797775129122,
+      "grad_norm": 1.675856113433838,
+      "learning_rate": 2.6000000000000002e-05,
+      "loss": 0.7476,
+      "step": 191
+    },
+    {
+      "epoch": 0.15256257449344457,
+      "grad_norm": 1.638994812965393,
+      "learning_rate": 2.4e-05,
+      "loss": 0.6634,
+      "step": 192
+    },
+    {
+      "epoch": 0.15335717123559794,
+      "grad_norm": 1.7587087154388428,
+      "learning_rate": 2.2000000000000003e-05,
+      "loss": 0.5597,
+      "step": 193
+    },
+    {
+      "epoch": 0.1541517679777513,
+      "grad_norm": 1.8964810371398926,
+      "learning_rate": 2e-05,
+      "loss": 0.791,
+      "step": 194
+    },
+    {
+      "epoch": 0.15494636471990464,
+      "grad_norm": 1.616877794265747,
+      "learning_rate": 1.8e-05,
+      "loss": 0.6717,
+      "step": 195
+    },
+    {
+      "epoch": 0.15574096146205801,
+      "grad_norm": 2.102555513381958,
+      "learning_rate": 1.6000000000000003e-05,
+      "loss": 0.5495,
+      "step": 196
+    },
+    {
+      "epoch": 0.15653555820421136,
+      "grad_norm": 2.0815021991729736,
+      "learning_rate": 1.4000000000000001e-05,
+      "loss": 0.7395,
+      "step": 197
+    },
+    {
+      "epoch": 0.1573301549463647,
+      "grad_norm": 1.726131796836853,
+      "learning_rate": 1.2e-05,
+      "loss": 0.6106,
+      "step": 198
+    },
+    {
+      "epoch": 0.15812475168851808,
+      "grad_norm": 2.1197216510772705,
+      "learning_rate": 1e-05,
+      "loss": 0.9239,
+      "step": 199
+    },
+    {
+      "epoch": 0.15891934843067143,
+      "grad_norm": 1.799057126045227,
+      "learning_rate": 8.000000000000001e-06,
+      "loss": 0.5982,
+      "step": 200
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 200,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2473157095243776.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-200/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa3e8d3b1b6a8e4852c2cd8a9e914dd29ad73547105e60642fcb9d0511974d3b
+size 5176

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa3e8d3b1b6a8e4852c2cd8a9e914dd29ad73547105e60642fcb9d0511974d3b
+size 5176