LucasMagnana/Pictalk_large

pictalk

This model is a fine-tuned version of bert-large-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.3395

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
4.4213	1.0	25	3.2802
3.1204	2.0	50	2.8289
2.7337	3.0	75	2.5070
2.4701	4.0	100	2.1833
2.2536	5.0	125	2.0859
2.1284	6.0	150	2.0973
1.9703	7.0	175	1.8079
1.9372	8.0	200	1.8733
1.9115	9.0	225	1.7319
1.7705	10.0	250	1.8154
1.7454	11.0	275	1.6135
1.7338	12.0	300	1.6072
1.6741	13.0	325	1.4479
1.6552	14.0	350	1.6893
1.5546	15.0	375	1.5714
1.5905	16.0	400	1.6661
1.5136	17.0	425	1.6100
1.5403	18.0	450	1.5664
1.4947	19.0	475	1.4803
1.4654	20.0	500	1.6041
1.4449	21.0	525	1.4071
1.4817	22.0	550	1.5543
1.377	23.0	575	1.3897
1.4102	24.0	600	1.4572
1.3246	25.0	625	1.5699
1.3323	26.0	650	1.4316
1.2745	27.0	675	1.5004
1.2589	28.0	700	1.5209
1.3488	29.0	725	1.4734
1.301	30.0	750	1.5197
1.2824	31.0	775	1.5087
1.2771	32.0	800	1.4041
1.2794	33.0	825	1.5773
1.2343	34.0	850	1.3722
1.3235	35.0	875	1.5125
1.2567	36.0	900	1.3877
1.2682	37.0	925	1.5471
1.2028	38.0	950	1.3677
1.2059	39.0	975	1.4233
1.2103	40.0	1000	1.5361
1.1987	41.0	1025	1.5492
1.2853	42.0	1050	1.4274
1.2088	43.0	1075	1.5027
1.2573	44.0	1100	1.5138
1.2511	45.0	1125	1.4198
1.1932	46.0	1150	1.3065
1.1864	47.0	1175	1.4521
1.2362	48.0	1200	1.4576
1.215	49.0	1225	1.4246
1.2118	50.0	1250	1.3395

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0