Training in progress, step 500

.ipynb_checkpoints/README-checkpoint.md

@@ -0,0 +1,75 @@
+---
+language:
+- sv-SE
+license: apache-2.0
+tags:
+- automatic-speech-recognition
+- mozilla-foundation/common_voice_8_0
+- generated_from_trainer
+datasets:
+- common_voice
+model-index:
+- name: ''
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# 
+
+This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the MOZILLA-FOUNDATION/COMMON_VOICE_8_0 - SV-SE dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.3549
+- Wer: 0.3827
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 7.5e-05
+- train_batch_size: 32
+- eval_batch_size: 32
+- seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 128
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 2000
+- num_epochs: 50.0
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Wer    |
+|:-------------:|:-----:|:----:|:---------------:|:------:|
+| 3.4129        | 5.49  | 500  | 3.3224          | 1.0    |
+| 2.9323        | 10.98 | 1000 | 2.9128          | 1.0000 |
+| 1.6839        | 16.48 | 1500 | 0.7740          | 0.6854 |
+| 1.485         | 21.97 | 2000 | 0.5830          | 0.5976 |
+| 1.362         | 27.47 | 2500 | 0.4866          | 0.4905 |
+| 1.2752        | 32.96 | 3000 | 0.4240          | 0.4967 |
+| 1.1957        | 38.46 | 3500 | 0.3899          | 0.4258 |
+| 1.1646        | 43.95 | 4000 | 0.3597          | 0.4014 |
+| 1.1265        | 49.45 | 4500 | 0.3559          | 0.3829 |
+
+
+### Framework versions
+
+- Transformers 4.17.0.dev0
+- Pytorch 1.10.2+cu102
+- Datasets 1.18.3
+- Tokenizers 0.11.0

config.json

@@ -76,7 +76,7 @@
   "num_hidden_layers": 24,
   "num_negatives": 100,
   "output_hidden_size": 1024,
-  "pad_token_id": 34,
+  "pad_token_id": 173,
   "proj_codevector_dim": 768,
   "tdnn_dilation": [
     1,
@@ -102,6 +102,6 @@
   "torch_dtype": "float32",
   "transformers_version": "4.17.0.dev0",
   "use_weighted_layer_sum": false,
-  "vocab_size": 37,
+  "vocab_size": 176,
   "xvector_output_dim": 512
 }

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:40b73af8306c0cfa719e96a4cd2fd6a50c8a41a0f9ee401e03bcbcce4d023123
-size 1262075377
+oid sha256:947eaeff297765ce57a36e9aae6c5113d13a6460597abeb07d5f258d81fe4518
+size 1262645297

special_tokens_map.json

@@ -1 +1 @@
-{"bos_token": "<s>", "eos_token": "</s>", "unk_token": "[UNK]", "pad_token": "[PAD]", "additional_special_tokens": [{"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, {"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}]}
+{"bos_token": "<s>", "eos_token": "</s>", "unk_token": "[UNK]", "pad_token": "[PAD]", "additional_special_tokens": [{"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, {"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, {"content": "<s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}]}

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:57e329018d47d30a41ba854e1445d35f9ed6fbfab2d2d444a48835c0e504d21f
+oid sha256:c43ad3864e30809e713fc2867749f2de8782624775f4983f4324caae4a0e301d
 size 2991