ind-to-bbc-nmt-v5

This model is a fine-tuned version of facebook/nllb-200-distilled-600M on the nusatranslation_mt dataset. It achieves the following results on the evaluation set:

• Loss: 1.1894
• Sacrebleu: 31.266
• Gen Len: 44.965

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Sacrebleu	Gen Len
3.6651	1.0	1650	1.4838	26.4515	46.9715
1.3236	2.0	3300	1.2132	30.7977	45.688
1.0377	3.0	4950	1.1590	31.5249	45.2095
0.871	4.0	6600	1.1329	31.7139	44.965
0.7493	5.0	8250	1.1319	31.3062	45.139
0.6536	6.0	9900	1.1331	30.8031	45.242
0.5772	7.0	11550	1.1492	31.1586	45.1815
0.5195	8.0	13200	1.1684	31.0977	45.019
0.4763	9.0	14850	1.1798	31.2488	44.8915
0.4478	10.0	16500	1.1894	31.266	44.965

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.14.6
• Tokenizers 0.19.1