yemen2016/MeMo-BERT-WSD_before_last

MeMo_BERT-WSD-03

This model is a fine-tuned version of MiMe-MeMo/MeMo-BERT-03 on https://huggingface.co/MiMe-MeMo/MeMo-Dataset-WSD dataset. It achieves the following results on the evaluation set:

• Loss: 2.7403
• F1-score: 0.5317

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	F1-score
No log	1.0	61	1.5313	0.2765
No log	2.0	122	1.1618	0.3798
No log	3.0	183	1.1259	0.4814
No log	4.0	244	1.3069	0.4656
No log	5.0	305	1.9109	0.4598
No log	6.0	366	2.0905	0.4766
No log	7.0	427	2.3842	0.4609
No log	8.0	488	2.7403	0.5317
0.5392	9.0	549	2.6113	0.4633
0.5392	10.0	610	3.0131	0.5016
0.5392	11.0	671	2.8423	0.5196
0.5392	12.0	732	2.9776	0.4876
0.5392	13.0	793	2.9717	0.4881
0.5392	14.0	854	2.9801	0.4887
0.5392	15.0	915	3.0105	0.4669
0.5392	16.0	976	3.0355	0.4887
0.0089	17.0	1037	3.0591	0.4887
0.0089	18.0	1098	3.0704	0.4887
0.0089	19.0	1159	3.0760	0.5012
0.0089	20.0	1220	3.0780	0.5012

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2