metadata
tags:
- generated_from_keras_callback
model-index:
- name: madatnlp/ke-t5-scratch
results: []
madatnlp/ke-t5-scratch
This model is a fine-tuned version of madatnlp/ke-t5-math-py on an unknown dataset. It achieves the following results on the evaluation set:
- Train Loss: 0.2662
- Validation Loss: 0.7210
- Epoch: 27
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'Adam', 'learning_rate': 0.001, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
- training_precision: float32
Training results
| Train Loss | Validation Loss | Epoch |
|---|---|---|
| 3.9330 | 1.9960 | 0 |
| 1.9578 | 1.4484 | 1 |
| 1.5358 | 1.3202 | 2 |
| 1.3807 | 1.1251 | 3 |
| 1.2885 | 1.0331 | 4 |
| 1.1943 | 1.0004 | 5 |
| 1.1366 | 0.9263 | 6 |
| 1.0507 | 0.8866 | 7 |
| 1.0160 | 0.8788 | 8 |
| 0.9553 | 0.8301 | 9 |
| 0.9149 | 0.8480 | 10 |
| 0.8545 | 0.8021 | 11 |
| 0.8271 | 0.7890 | 12 |
| 0.7783 | 0.7549 | 13 |
| 0.7166 | 0.6960 | 14 |
| 0.6853 | 0.6828 | 15 |
| 0.6142 | 0.7129 | 16 |
| 0.5774 | 0.6368 | 17 |
| 0.5612 | 0.6432 | 18 |
| 0.4980 | 0.6483 | 19 |
| 0.4723 | 0.6485 | 20 |
| 0.4245 | 0.6569 | 21 |
| 0.4040 | 0.6494 | 22 |
| 0.3733 | 0.6970 | 23 |
| 0.3461 | 0.7069 | 24 |
| 0.3166 | 0.6597 | 25 |
| 0.2912 | 0.6372 | 26 |
| 0.2662 | 0.7210 | 27 |
Framework versions
- Transformers 4.18.0
- TensorFlow 2.8.0
- Datasets 2.1.0
- Tokenizers 0.12.1