Hugging Face's logo

C2001ljy
/
rag_sequence_nq_on_cqa

Safetensors
rag
Model card Files Community
rag_sequence_nq_on_cqa / README.md
C2001ljy's picture
C2001ljy
Upload RagSequenceForGeneration
2aae262 verified
preview code
|
raw
history blame
1.74 kB
metadata
license: apache-2.0
tags:
  - generated_from_trainer
base_model: facebook/rag-sequence-nq
datasets:
  - cqa_v1
model-index:
  - name: rag_sequence_nq_on_cqa
    results: []

rag_sequence_nq_on_cqa

This model is a fine-tuned version of facebook/rag-sequence-nq on the cqa_v1 dataset. It achieves the following results on the evaluation set:

  • Loss: 31.3296

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-05
  • train_batch_size: 12
  • eval_batch_size: 12
  • seed: 47
  • gradient_accumulation_steps: 128
  • total_train_batch_size: 1536
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: polynomial
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 6
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
No log 0.9968 17 41.6343
No log 1.9936 34 36.0718
No log 2.9968 51 33.7388
No log 3.9936 68 32.4122
No log 4.9904 85 31.6654
No log 5.9872 102 31.3296

Framework versions

  • Transformers 4.40.0
  • Pytorch 2.0.1+cu118
  • Datasets 2.14.5
  • Tokenizers 0.19.1