mms-1b-all-swagen-combined-20hrs-62

This model is a fine-tuned version of facebook/mms-1b-all on the SWAGEN - SWA dataset. It achieves the following results on the evaluation set:

• Loss: 0.2244
• Wer: 0.1892

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 4
• seed: 62
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
5.1379	0.1193	100	0.3352	0.2050
0.3005	0.2385	200	0.2586	0.1950
0.2623	0.3578	300	0.2405	0.1942
0.2437	0.4770	400	0.2349	0.1934
0.2479	0.5963	500	0.2308	0.1917
0.2239	0.7156	600	0.2309	0.1893
0.2346	0.8348	700	0.2293	0.1934
0.2316	0.9541	800	0.2296	0.1930
0.2242	1.0727	900	0.2284	0.1941
0.2444	1.1920	1000	0.2269	0.1919
0.2177	1.3113	1100	0.2259	0.1892
0.2216	1.4305	1200	0.2243	0.1891
0.2144	1.5498	1300	0.2248	0.1932
0.2166	1.6691	1400	0.2229	0.1869
0.2127	1.7883	1500	0.2221	0.1892
0.2099	1.9076	1600	0.2229	0.1879
0.2184	2.0262	1700	0.2232	0.1878
0.225	2.1455	1800	0.2264	0.1887

Framework versions

• Transformers 4.53.0.dev0
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.0