madatnlp/ke-t5-scratch
This model is a fine-tuned version of madatnlp/ke-t5-math-py on an unknown dataset. It achieves the following results on the evaluation set:
- Train Loss: 0.4760
- Validation Loss: 0.7360
- Epoch: 36
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'Adam', 'learning_rate': 1e-04, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
- training_precision: float32
Training results
Train Loss | Validation Loss | Epoch |
---|---|---|
4.2751 | 2.1074 | 0 |
2.2716 | 1.7945 | 1 |
1.8889 | 1.5726 | 2 |
1.6760 | 1.3722 | 3 |
1.5021 | 1.3280 | 4 |
1.4369 | 1.2523 | 5 |
1.3352 | 1.0619 | 6 |
1.2749 | 1.1156 | 7 |
1.2170 | 1.0452 | 8 |
1.1713 | 1.0596 | 9 |
1.1410 | 1.0080 | 10 |
1.0884 | 1.0213 | 11 |
1.0508 | 0.9223 | 12 |
0.9933 | 0.9353 | 13 |
0.9871 | 0.8749 | 14 |
0.9251 | 0.9173 | 15 |
0.9282 | 0.8620 | 16 |
0.8849 | 0.8093 | 17 |
0.8613 | 0.7823 | 18 |
0.8322 | 0.8016 | 19 |
0.8070 | 0.8844 | 20 |
0.7737 | 0.7635 | 21 |
0.7465 | 0.8440 | 22 |
0.7178 | 0.7958 | 23 |
0.7036 | 0.7739 | 24 |
0.6813 | 0.7347 | 25 |
0.6597 | 0.7545 | 26 |
0.6427 | 0.7394 | 27 |
0.6154 | 0.7212 | 28 |
0.5892 | 0.7653 | 29 |
0.5696 | 0.7073 | 30 |
0.5644 | 0.6977 | 31 |
0.5307 | 0.6977 | 32 |
0.5159 | 0.7736 | 33 |
0.5131 | 0.8138 | 34 |
0.4812 | 0.7623 | 35 |
0.4760 | 0.7360 | 36 |
Framework versions
- Transformers 4.18.0
- TensorFlow 2.8.0
- Datasets 2.1.0
- Tokenizers 0.12.1
- Downloads last month
- 6
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.