hubert-base-ls960-finetuned-ic-slurp-wt_init-frz-v1

This model is a fine-tuned version of facebook/hubert-base-ls960 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.8557
• Accuracy: 0.4665

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 24
• eval_batch_size: 24
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 96
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
3.6161	1.0	527	3.6149	0.1639
3.4319	2.0	1055	3.4045	0.1837
3.2109	3.0	1582	3.1534	0.2204
3.0227	4.0	2110	3.0869	0.2425
2.7612	5.0	2637	2.8947	0.2796
2.6536	6.0	3165	2.7741	0.3162
2.2984	7.0	3692	2.5992	0.3517
2.2411	8.0	4220	2.5695	0.3678
2.0698	9.0	4747	2.5301	0.3828
1.781	10.0	5275	2.4942	0.4076
1.7756	11.0	5802	2.4456	0.4145
1.429	12.0	6330	2.4907	0.4214
1.4662	13.0	6857	2.5513	0.4287
1.2868	14.0	7385	2.6220	0.4254
1.0628	15.0	7912	2.6932	0.4294
1.0636	16.0	8440	2.7047	0.4348
0.861	17.0	8967	2.7132	0.4405
0.8748	18.0	9495	2.8117	0.4414
0.7779	19.0	10022	2.8338	0.4454
0.7247	20.0	10550	2.9349	0.4407
0.6041	21.0	11077	2.9980	0.4396
0.6234	22.0	11605	3.0899	0.4418
0.4641	23.0	12132	3.1206	0.4470
0.5321	24.0	12660	3.2098	0.4427
0.4293	25.0	13187	3.2953	0.4414
0.5322	26.0	13715	3.2976	0.4458
0.3345	27.0	14242	3.3888	0.4441
0.4868	28.0	14770	3.3955	0.4472
0.29	29.0	15297	3.4445	0.4451
0.2429	30.0	15825	3.4317	0.4537
0.3375	31.0	16352	3.4972	0.4534
0.26	32.0	16880	3.6675	0.4434
0.2337	33.0	17407	3.5817	0.4491
0.2984	34.0	17935	3.5766	0.4485
0.2249	35.0	18462	3.5912	0.4538
0.1962	36.0	18990	3.6414	0.4556
0.2243	37.0	19517	3.7025	0.4563
0.2169	38.0	20045	3.7524	0.4557
0.1509	39.0	20572	3.6993	0.4583
0.2106	40.0	21100	3.8040	0.4550
0.224	41.0	21627	3.7628	0.4628
0.1154	42.0	22155	3.7545	0.4652
0.1453	43.0	22682	3.7632	0.4651
0.1221	44.0	23210	3.8144	0.4596
0.1419	45.0	23737	3.8580	0.4627
0.1178	46.0	24265	3.8238	0.4656
0.1517	47.0	24792	3.8614	0.4635
0.1207	48.0	25320	3.8786	0.4644
0.1223	49.0	25847	3.8557	0.4665
0.067	49.95	26350	3.8611	0.4651

Framework versions

• Transformers 4.36.2
• Pytorch 2.2.1+cu118
• Datasets 2.17.0
• Tokenizers 0.15.0