madatnlp/kor-math-roberta-finetune

This model is a fine-tuned version of klue/roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.3205
• Validation Loss: 1.1407
• Epoch: 26

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 1e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: mixed_bfloat16

Training results

Train Loss	Validation Loss	Epoch
3.4242	2.0873	0
1.9159	1.6264	1
1.5933	1.4521	2
1.3806	1.3584	3
1.2487	1.2904	4
1.1464	1.2388	5
1.0552	1.2076	6
0.9889	1.1818	7
0.9118	1.1607	8
0.8459	1.1349	9
0.7838	1.1193	10
0.7389	1.1193	11
0.6864	1.1080	12
0.6495	1.1001	13
0.6103	1.1001	14
0.5795	1.0990	15
0.5436	1.0954	16
0.5136	1.0997	17
0.4906	1.0954	18
0.4565	1.1021	19
0.4347	1.1075	20
0.4131	1.1075	21
0.3924	1.1220	22
0.3741	1.1298	23
0.3549	1.1352	24
0.3395	1.1286	25
0.3205	1.1407	26

Framework versions

• Transformers 4.18.0
• TensorFlow 2.8.0
• Datasets 2.1.0
• Tokenizers 0.12.1