---
library_name: transformers
base_model: rohanrajpal/bert-base-codemixed-uncased-sentiment
tags:
- generated_from_trainer
model-index:
- name: mlm-code-roman-finetuned-final
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# mlm-code-roman-finetuned-final

This model is a fine-tuned version of [rohanrajpal/bert-base-codemixed-uncased-sentiment](https://huggingface.co/rohanrajpal/bert-base-codemixed-uncased-sentiment) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 3.2295
- Model Preparation Time: 0.0031

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 50
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Model Preparation Time |
|:-------------:|:-----:|:-----:|:---------------:|:----------------------:|
| 4.8669        | 1.0   | 1834  | 4.6996          | 0.0031                 |
| 4.2832        | 2.0   | 3668  | 4.1303          | 0.0031                 |
| 3.9681        | 3.0   | 5502  | 3.9828          | 0.0031                 |
| 3.8238        | 4.0   | 7336  | nan             | 0.0031                 |
| 3.8079        | 5.0   | 9170  | 3.6848          | 0.0031                 |
| 3.5443        | 6.0   | 11004 | nan             | 0.0031                 |
| 3.4345        | 7.0   | 12838 | 3.6615          | 0.0031                 |
| 3.3339        | 8.0   | 14672 | nan             | 0.0031                 |
| 3.2448        | 9.0   | 16506 | 3.5461          | 0.0031                 |
| 3.1693        | 10.0  | 18340 | 3.4980          | 0.0031                 |
| 3.2229        | 11.0  | 20174 | nan             | 0.0031                 |
| 3.0313        | 12.0  | 22008 | 3.4317          | 0.0031                 |
| 2.9187        | 13.0  | 23842 | 3.4374          | 0.0031                 |
| 2.7454        | 14.0  | 25676 | 3.3087          | 0.0031                 |
| 2.8174        | 15.0  | 27510 | nan             | 0.0031                 |
| 2.6992        | 16.0  | 29344 | 3.2808          | 0.0031                 |
| 2.7873        | 17.0  | 31178 | nan             | 0.0031                 |
| 2.7171        | 18.0  | 33012 | 3.3550          | 0.0031                 |
| 2.6304        | 19.0  | 34846 | 3.1810          | 0.0031                 |
| 2.5512        | 20.0  | 36680 | 3.4333          | 0.0031                 |
| 2.4428        | 21.0  | 38514 | 3.2822          | 0.0031                 |
| 2.4032        | 22.0  | 40348 | 3.2255          | 0.0031                 |
| 2.3325        | 23.0  | 42182 | nan             | 0.0031                 |
| 2.3242        | 24.0  | 44016 | nan             | 0.0031                 |
| 2.2707        | 25.0  | 45850 | 3.2783          | 0.0031                 |
| 2.2549        | 26.0  | 47684 | 3.1391          | 0.0031                 |
| 2.1746        | 27.0  | 49518 | 3.2615          | 0.0031                 |
| 2.1261        | 28.0  | 51352 | 3.3844          | 0.0031                 |
| 2.131         | 29.0  | 53186 | nan             | 0.0031                 |
| 2.0258        | 30.0  | 55020 | nan             | 0.0031                 |
| 2.0013        | 31.0  | 56854 | 3.2128          | 0.0031                 |
| 1.9835        | 32.0  | 58688 | 3.2159          | 0.0031                 |
| 1.9533        | 33.0  | 60522 | 3.1775          | 0.0031                 |
| 1.8876        | 34.0  | 62356 | nan             | 0.0031                 |
| 1.8624        | 35.0  | 64190 | nan             | 0.0031                 |
| 1.8181        | 36.0  | 66024 | nan             | 0.0031                 |
| 1.8068        | 37.0  | 67858 | 3.2109          | 0.0031                 |
| 1.7557        | 38.0  | 69692 | 3.0247          | 0.0031                 |
| 1.6394        | 39.0  | 71526 | nan             | 0.0031                 |
| 1.7325        | 40.0  | 73360 | 3.1127          | 0.0031                 |
| 1.7365        | 41.0  | 75194 | nan             | 0.0031                 |
| 1.6807        | 42.0  | 77028 | 3.0874          | 0.0031                 |
| 1.5981        | 43.0  | 78862 | 3.0828          | 0.0031                 |
| 1.6304        | 44.0  | 80696 | 3.1521          | 0.0031                 |
| 1.568         | 45.0  | 82530 | 3.2340          | 0.0031                 |
| 1.5382        | 46.0  | 84364 | nan             | 0.0031                 |
| 1.387         | 47.0  | 86198 | 3.2172          | 0.0031                 |
| 1.5521        | 48.0  | 88032 | 3.1709          | 0.0031                 |
| 1.4808        | 49.0  | 89866 | nan             | 0.0031                 |
| 1.4629        | 50.0  | 91700 | nan             | 0.0031                 |


### Framework versions

- Transformers 4.46.1
- Pytorch 2.2.1+cu121
- Datasets 3.1.0
- Tokenizers 0.20.1