bert-base-multilingual-cased-finetuned-yiddish-experiment-1

This model is a fine-tuned version of bert-base-multilingual-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.4022

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 200
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss
6.9585	0.4717	100	2.0347
1.7233	0.9434	200	1.5785
1.4538	1.4151	300	1.5119
1.3844	1.8868	400	1.4678
1.3024	2.3585	500	1.4263
1.2709	2.8302	600	1.4057
1.2155	3.3019	700	1.4144
1.2136	3.7736	800	1.4022
1.151	4.2453	900	1.4880
1.1371	4.7170	1000	1.4477
1.1091	5.1887	1100	1.4028
1.0638	5.6604	1200	1.4788
1.0468	6.1321	1300	1.4812
1.0122	6.6038	1400	1.4641
1.0158	7.0755	1500	1.5584
0.9775	7.5472	1600	1.5608
0.9455	8.0189	1700	1.6017
0.929	8.4906	1800	1.5681
0.9406	8.9623	1900	1.5814
0.9066	9.4340	2000	1.6071
0.9317	9.9057	2100	1.5979

Framework versions

• Transformers 4.47.0
• Pytorch 2.5.1+cu121
• Datasets 3.1.0
• Tokenizers 0.21.0