ind-to-bbc-nmt-v7

This model is a fine-tuned version of facebook/nllb-200-distilled-600M on the nusatranslation_mt dataset. It achieves the following results on the evaluation set:

• Loss: 1.1534
• Sacrebleu: 31.4148
• Gen Len: 45.246

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Sacrebleu	Gen Len
5.1799	1.0	413	2.3351	25.3863	45.489
1.6805	2.0	826	1.3384	30.3818	45.661
1.2114	3.0	1239	1.2202	30.9982	45.562
1.0517	4.0	1652	1.1827	31.2905	45.3925
0.9461	5.0	2065	1.1678	31.6094	45.2625
0.8728	6.0	2478	1.1471	31.2517	45.4265
0.8153	7.0	2891	1.1497	31.332	45.1645
0.7719	8.0	3304	1.1467	31.372	45.3915
0.743	9.0	3717	1.1491	31.4979	45.0825
0.7204	10.0	4130	1.1534	31.4148	45.246

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.14.6
• Tokenizers 0.19.1