crossencoder-airline-refine-030

This model is a fine-tuned version of cross-encoder/stsb-roberta-large on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0451

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.312	1.0	159	0.3001
0.197	2.0	318	0.3300
0.2195	3.0	477	0.1976
0.5386	4.0	636	0.1339
0.1417	5.0	795	0.1625
0.2544	6.0	954	0.5143
0.0716	7.0	1113	0.1888
0.1214	8.0	1272	0.1115
0.1658	9.0	1431	0.1007
0.1489	10.0	1590	0.1735
0.1163	11.0	1749	0.0781
0.1411	12.0	1908	0.1386
0.1636	13.0	2067	0.1462
0.1122	14.0	2226	0.2000
0.1287	15.0	2385	0.1702
0.1954	16.0	2544	0.1191
0.0632	17.0	2703	0.1155
0.0377	18.0	2862	0.0979
0.0714	19.0	3021	0.0810
0.0727	20.0	3180	0.0710
0.0449	21.0	3339	0.0832
0.0964	22.0	3498	0.0721
0.0276	23.0	3657	0.0516
0.0269	24.0	3816	0.0601
0.0483	25.0	3975	0.0583
0.0442	26.0	4134	0.0627
0.0181	27.0	4293	0.0704
0.0177	28.0	4452	0.0631
0.0315	29.0	4611	0.0614
0.0184	30.0	4770	0.0623

Framework versions

• Transformers 4.38.1
• Pytorch 2.0.1
• Datasets 2.17.1
• Tokenizers 0.15.2