electra_hongkongese_small_pos_hkcancor

This model is a fine-tuned version of toastynews/electra-hongkongese-small-discriminator on the hkcancor dataset. It achieves the following results on the evaluation set:

• Loss: 0.2846

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
1.475	0.3004	201	1.3204
0.8403	0.6009	402	0.7444
0.6056	0.9013	603	0.5415
0.486	1.2018	804	0.4398
0.4053	1.5022	1005	0.3744
0.3702	1.8027	1206	0.3419
0.3413	2.1031	1407	0.3125
0.3239	2.4036	1608	0.2950
0.3098	2.7040	1809	0.2846

Framework versions

• Transformers 4.43.3
• Pytorch 2.4.0
• Datasets 2.20.0
• Tokenizers 0.19.1