|
--- |
|
license: apache-2.0 |
|
base_model: openai/whisper-large-v2 |
|
tags: |
|
- audio |
|
- automatic-speech-recognition |
|
- translate |
|
- generated_from_trainer |
|
language: |
|
- zh |
|
metrics: |
|
- cer |
|
- wer |
|
model-index: |
|
- name: whisper-large-v2-translate-zh-v0.1-lt-ct2 |
|
results: [] |
|
--- |
|
|
|
# whisper-large-v2-translate-zh-v0.1-lt-ct2 |
|
|
|
This model is a fine-tuned version of [openai/whisper-large-v2](https://huggingface.co/openai/whisper-large-v2). |
|
|
|
## Model description |
|
|
|
3500小时 (日语音频,中文字幕) 数据微调, 翻译模式直出中文 |
|
|
|
## Usage |
|
|
|
task='translate', language='ja' |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 1e-05 |
|
- train_batch_size: 32 |
|
- eval_batch_size: 32 |
|
- seed: 42 |
|
- gradient_accumulation_steps: 2 |
|
- total_train_batch_size: 64 |
|
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- lr_scheduler_warmup_steps: 4000 |
|
- dropout: 0.1 |
|
- mask_time_prob: 0.05 |
|
- mask_feature_prob: 0.2 |
|
- condition_on_previous_text_rate: 0.5 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Cer | Wer | |
|
|:-------------:|:------:|:-----:|:---------------:|:------:|:------:| |
|
| 1.743 | 0.0740 | 1000 | 1.5631 | 0.8223 | 1.4517 | |
|
| 1.6014 | 0.1479 | 2000 | 1.4808 | 0.6775 | 1.0950 | |
|
| 1.5549 | 0.2219 | 3000 | 1.4381 | 0.6756 | 1.1158 | |
|
| 1.5283 | 0.2958 | 4000 | 1.4174 | 0.6992 | 1.1137 | |
|
| 1.474 | 0.3698 | 5000 | 1.3849 | 0.6570 | 1.1369 | |
|
| 1.4193 | 0.4437 | 6000 | 1.3657 | 0.6544 | 1.1339 | |
|
| 1.4148 | 0.5177 | 7000 | 1.3477 | 0.6386 | 1.1647 | |
|
| 1.3754 | 0.5916 | 8000 | 1.3392 | 0.6228 | 1.0461 | |
|
| 1.3441 | 0.6656 | 9000 | 1.3362 | 0.6196 | 1.0609 | |
|
| 1.3545 | 0.7395 | 10000 | 1.3176 | 0.6354 | 1.2138 | |
|
| 1.3498 | 0.8135 | 11000 | 1.3236 | 0.6631 | 1.2232 | |
|
| 1.31 | 0.8874 | 12000 | 1.3020 | 0.6199 | 1.0018 | |
|
| 1.3213 | 0.9614 | 13000 | 1.2966 | 0.5922 | 1.0021 | |
|
| 1.2375 | 1.0353 | 14000 | 1.2900 | 0.6097 | 1.0639 | |
|
| 1.2334 | 1.1093 | 15000 | 1.2963 | 0.6150 | 1.0920 | |
|
| 1.2277 | 1.1832 | 16000 | 1.2888 | 0.6077 | 1.0929 | |
|
| 1.2087 | 1.2572 | 17000 | 1.2779 | 0.5954 | 1.0012 | |
|
| 1.2131 | 1.3311 | 18000 | 1.2722 | 0.5776 | 1.0075 | |
|
| 1.2012 | 1.4051 | 19000 | 1.2716 | 0.5726 | 1.0211 | |
|
| 1.1912 | 1.4790 | 20000 | 1.2707 | 0.6007 | 1.1538 | |
|
| 1.2127 | 1.5530 | 21000 | 1.2749 | 0.6086 | 1.0742 | |
|
| 1.1789 | 1.6269 | 22000 | 1.2797 | 0.5765 | 1.0072 | |
|
| 1.1527 | 1.7009 | 23000 | 1.2761 | 0.5855 | 1.0588 | |
|
| 1.1693 | 1.7748 | 24000 | 1.2701 | 0.5635 | 0.9928 | |
|
| 1.1709 | 1.8488 | 25000 | 1.2662 | 0.5980 | 1.0697 | |
|
| 1.1637 | 1.9227 | 26000 | 1.2749 | 0.5872 | 1.0392 | |
|
| 1.1562 | 1.9967 | 27000 | 1.2587 | 0.5651 | 1.0121 | |
|
| 1.0929 | 2.0706 | 28000 | 1.2668 | 0.5857 | 1.0139 | |
|
| 1.1232 | 2.1446 | 29000 | 1.2710 | 0.5742 | 0.9997 | |
|
| 1.1045 | 2.2185 | 30000 | 1.2656 | 0.5643 | 0.9897 | |
|
| 1.0841 | 2.2925 | 31000 | 1.2695 | 0.5835 | 1.0181 | |
|
| 1.0868 | 2.3664 | 32000 | 1.2707 | 0.5673 | 0.9964 | |
|
| 1.0938 | 2.4404 | 33000 | 1.2644 | 0.5712 | 0.9928 | |
|
| 1.0938 | 2.5143 | 34000 | 1.2662 | 0.5750 | 1.0109 | |
|
| 1.0848 | 2.5883 | 35000 | 1.2677 | 0.5841 | 1.0832 | |
|
| 1.0914 | 2.6622 | 36000 | 1.2638 | 0.5801 | 1.0299 | |
|
| 1.0688 | 2.7362 | 37000 | 1.2587 | 0.5694 | 1.0072 | |
|
| 1.0856 | 2.8101 | 38000 | 1.2581 | 0.5646 | 1.0057 | |
|
| 1.1037 | 2.8841 | 39000 | 1.2557 | 0.5771 | 1.0262 | |
|
| 1.0652 | 2.9580 | 40000 | 1.2566 | 0.5634 | 0.9979 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.41.1 |
|
- Pytorch 2.3.1+cu121 |
|
- Datasets 2.19.1 |
|
- Tokenizers 0.19.1 |
