transcriber-t5-v8

This model is a fine-tuned version of odunola/transcriber-t5-v7 on the odunola/transcriberv3 dataset. It achieves the following results on the evaluation set:

• Loss: 0.1924

Model description

This model, a fine-tuned T5, is designed to pinpoint and extract specific Bible scriptures' chapter and verse from a string of text. It's a major component of a broader project but is also versatile enough for your other applications.

The model's learning is driven by a unique dataset, painstakingly compiled from transcripts of sermons on YouTube and various online platforms. This handpicked, curated data equips the model with a specialized understanding of religious discourse and biblical references.

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 20
• eval_batch_size: 20
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
0.2215	0.23	500	0.2216
0.1814	0.47	1000	0.2197
0.2011	0.7	1500	0.2059
0.2595	0.94	2000	0.2009
0.2412	1.17	2500	0.2019
0.1785	1.41	3000	0.1970
0.1962	1.64	3500	0.1983
0.2009	1.88	4000	0.1963
0.2013	2.11	4500	0.1923
0.2715	2.35	5000	0.1929
0.2488	2.58	5500	0.1936
0.1185	2.81	6000	0.1924

Framework versions

• Transformers 4.28.1
• Pytorch 2.0.0+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3