---
license: apache-2.0
base_model: t5-base
tags:
- generated_from_trainer
datasets:
- glue
metrics:
- accuracy
model-index:
- name: t5-base_cola_dense
  results:
  - task:
      name: Text Classification
      type: text-classification
    dataset:
      name: glue
      type: glue
      config: cola
      split: validation
      args: cola
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.8360498561840843
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# t5-base_cola_dense

This model is a fine-tuned version of [t5-base](https://huggingface.co/t5-base) on the glue dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5247
- Accuracy: 0.8360

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 64
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 200
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 0.6253        | 0.07  | 10   | 0.6218          | 0.6913   |
| 0.6283        | 0.15  | 20   | 0.6207          | 0.6913   |
| 0.6297        | 0.22  | 30   | 0.6193          | 0.6913   |
| 0.5936        | 0.3   | 40   | 0.6200          | 0.6913   |
| 0.609         | 0.37  | 50   | 0.6216          | 0.6913   |
| 0.5818        | 0.45  | 60   | 0.6182          | 0.6913   |
| 0.5984        | 0.52  | 70   | 0.6139          | 0.6913   |
| 0.5961        | 0.6   | 80   | 0.6089          | 0.6913   |
| 0.5594        | 0.67  | 90   | 0.6168          | 0.6913   |
| 0.597         | 0.75  | 100  | 0.6022          | 0.6913   |
| 0.6092        | 0.82  | 110  | 0.5537          | 0.6903   |
| 0.5454        | 0.9   | 120  | 0.5120          | 0.7296   |
| 0.5368        | 0.97  | 130  | 0.5120          | 0.7584   |
| 0.5185        | 1.04  | 140  | 0.4615          | 0.7987   |
| 0.4664        | 1.12  | 150  | 0.4893          | 0.7977   |
| 0.4938        | 1.19  | 160  | 0.4793          | 0.8044   |
| 0.3994        | 1.27  | 170  | 0.4912          | 0.8025   |
| 0.4989        | 1.34  | 180  | 0.5515          | 0.8092   |
| 0.4709        | 1.42  | 190  | 0.4909          | 0.8054   |
| 0.4099        | 1.49  | 200  | 0.5397          | 0.8121   |
| 0.4671        | 1.57  | 210  | 0.4736          | 0.8102   |
| 0.3893        | 1.64  | 220  | 0.4803          | 0.8178   |
| 0.4027        | 1.72  | 230  | 0.5195          | 0.8159   |
| 0.4208        | 1.79  | 240  | 0.4521          | 0.8188   |
| 0.4506        | 1.87  | 250  | 0.4943          | 0.8188   |
| 0.3647        | 1.94  | 260  | 0.4650          | 0.8255   |
| 0.4223        | 2.01  | 270  | 0.4865          | 0.8284   |
| 0.3584        | 2.09  | 280  | 0.4639          | 0.8284   |
| 0.3555        | 2.16  | 290  | 0.5321          | 0.8236   |
| 0.3433        | 2.24  | 300  | 0.5174          | 0.8303   |
| 0.3904        | 2.31  | 310  | 0.4811          | 0.8274   |
| 0.3418        | 2.39  | 320  | 0.5135          | 0.8265   |
| 0.3397        | 2.46  | 330  | 0.4854          | 0.8322   |
| 0.3336        | 2.54  | 340  | 0.5008          | 0.8332   |
| 0.3471        | 2.61  | 350  | 0.5065          | 0.8293   |
| 0.382         | 2.69  | 360  | 0.4708          | 0.8274   |
| 0.3533        | 2.76  | 370  | 0.4862          | 0.8265   |
| 0.3199        | 2.84  | 380  | 0.4904          | 0.8293   |
| 0.3757        | 2.91  | 390  | 0.4970          | 0.8332   |
| 0.3726        | 2.99  | 400  | 0.4965          | 0.8322   |
| 0.2957        | 3.06  | 410  | 0.4628          | 0.8303   |
| 0.3232        | 3.13  | 420  | 0.5174          | 0.8322   |
| 0.2836        | 3.21  | 430  | 0.5038          | 0.8351   |
| 0.2919        | 3.28  | 440  | 0.4987          | 0.8341   |
| 0.3578        | 3.36  | 450  | 0.5187          | 0.8313   |
| 0.398         | 3.43  | 460  | 0.5285          | 0.8380   |
| 0.3024        | 3.51  | 470  | 0.4971          | 0.8351   |
| 0.3153        | 3.58  | 480  | 0.5084          | 0.8351   |
| 0.307         | 3.66  | 490  | 0.5371          | 0.8332   |
| 0.2753        | 3.73  | 500  | 0.5247          | 0.8360   |
| 0.3515        | 3.81  | 510  | 0.4782          | 0.8360   |
| 0.2881        | 3.88  | 520  | 0.4784          | 0.8389   |
| 0.3203        | 3.96  | 530  | 0.5115          | 0.8351   |
| 0.2791        | 4.03  | 540  | 0.5294          | 0.8360   |
| 0.301         | 4.1   | 550  | 0.5218          | 0.8322   |
| 0.2652        | 4.18  | 560  | 0.4956          | 0.8360   |
| 0.2954        | 4.25  | 570  | 0.4878          | 0.8332   |
| 0.2345        | 4.33  | 580  | 0.5190          | 0.8313   |
| 0.3762        | 4.4   | 590  | 0.5315          | 0.8351   |
| 0.3614        | 4.48  | 600  | 0.5200          | 0.8341   |
| 0.3178        | 4.55  | 610  | 0.5237          | 0.8341   |
| 0.306         | 4.63  | 620  | 0.5232          | 0.8341   |
| 0.2828        | 4.7   | 630  | 0.5278          | 0.8360   |
| 0.3442        | 4.78  | 640  | 0.5270          | 0.8360   |
| 0.3268        | 4.85  | 650  | 0.5252          | 0.8351   |
| 0.2959        | 4.93  | 660  | 0.5284          | 0.8370   |
| 0.2861        | 5.0   | 670  | 0.5277          | 0.8351   |


### Framework versions

- Transformers 4.34.0
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.14.1