End of training

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+predictions_common_voice_13_en_common_voice_13_en_test_wer22.42_hyp.trn.snt.utt filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,95 @@
+---
+tags:
+- generated_from_trainer
+datasets:
+- common_voice_13_0
+metrics:
+- wer
+model-index:
+- name: ED_small_cv_v2
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# ED_small_cv_v2
+
+This model is a fine-tuned version of [](https://huggingface.co/) on the common_voice_13_0 dataset.
+It achieves the following results on the evaluation set:
+- Cer: 0.0805
+- Deletions: 3333
+- Hits: 124879
+- Insertions: 3716
+- Loss: 1.1152
+- Mer: 0.1779
+- Substitutions: 19969
+- Wer: 0.1823
+- Wil: 0.2916
+- Wip: 0.7084
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.001
+- train_batch_size: 128
+- eval_batch_size: 64
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 4
+- total_train_batch_size: 512
+- total_eval_batch_size: 256
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 15000
+- num_epochs: 50.0
+
+### Training results
+
+| Training Loss | Epoch | Step  | Cer    | Deletions | Hits   | Insertions | Validation Loss | Mer    | Substitutions | Wer    | Wil    | Wip    |
+|:-------------:|:-----:|:-----:|:------:|:---------:|:------:|:----------:|:---------------:|:------:|:-------------:|:------:|:------:|:------:|
+| 1.7038        | 5.0   | 14885 | 0.1731 | 6463      | 103703 | 7394       | 1.5150          | 0.3334 | 38015         | 0.3501 | 0.5133 | 0.4867 |
+| 1.6051        | 6.0   | 17862 | 0.1512 | 6309      | 107805 | 6257       | 1.4257          | 0.3020 | 34067         | 0.3147 | 0.4705 | 0.5295 |
+| 1.5396        | 7.0   | 20839 | 0.1368 | 5262      | 110900 | 6029       | 1.3715          | 0.2809 | 32019         | 0.2923 | 0.4428 | 0.5572 |
+| 1.4436        | 8.0   | 23816 | 0.1299 | 5464      | 112866 | 5686       | 1.3285          | 0.2665 | 29851         | 0.2767 | 0.4207 | 0.5793 |
+| 1.4287        | 9.0   | 26793 | 0.1241 | 4793      | 114249 | 5680       | 1.3090          | 0.2575 | 29139         | 0.2673 | 0.4091 | 0.5909 |
+| 1.395         | 10.0  | 29770 | 0.1200 | 4698      | 115580 | 5505       | 1.2842          | 0.2479 | 27903         | 0.2572 | 0.3949 | 0.6051 |
+| 1.3668        | 11.0  | 32747 | 0.1111 | 4400      | 117281 | 4991       | 1.2510          | 0.2343 | 26500         | 0.2422 | 0.3761 | 0.6239 |
+| 1.3238        | 12.0  | 35724 | 0.1064 | 4560      | 117845 | 4500       | 1.2363          | 0.2282 | 25776         | 0.2351 | 0.3673 | 0.6327 |
+| 1.3133        | 13.0  | 38701 | 0.1028 | 4173      | 118971 | 4613       | 1.2215          | 0.2214 | 25037         | 0.2283 | 0.3573 | 0.6427 |
+| 1.2968        | 14.0  | 41678 | 0.0995 | 3937      | 119798 | 4466       | 1.2026          | 0.2152 | 24446         | 0.2217 | 0.3487 | 0.6513 |
+| 1.2783        | 15.0  | 44655 | 0.0974 | 4071      | 120231 | 4295       | 1.1939          | 0.2115 | 23879         | 0.2176 | 0.3427 | 0.6573 |
+| 1.2359        | 16.0  | 47632 | 0.0961 | 3946      | 120640 | 4313       | 1.1884          | 0.2089 | 23595         | 0.2150 | 0.3388 | 0.6612 |
+| 1.2543        | 17.0  | 50609 | 0.0939 | 3757      | 121623 | 4476       | 1.1743          | 0.2033 | 22801         | 0.2094 | 0.3296 | 0.6704 |
+| 1.2245        | 18.0  | 53586 | 0.0919 | 3981      | 121522 | 3944       | 1.1690          | 0.2012 | 22678         | 0.2065 | 0.3273 | 0.6727 |
+| 1.2           | 19.0  | 56563 | 0.0903 | 3819      | 122029 | 3995       | 1.1626          | 0.1981 | 22333         | 0.2034 | 0.3226 | 0.6774 |
+| 1.1964        | 20.0  | 59540 | 0.0916 | 3822      | 122170 | 4154       | 1.1598          | 0.1980 | 22189         | 0.2036 | 0.3218 | 0.6782 |
+| 1.1822        | 21.0  | 62517 | 0.0871 | 3630      | 122825 | 3981       | 1.1471          | 0.1928 | 21726         | 0.1980 | 0.3146 | 0.6854 |
+| 1.1758        | 22.0  | 65494 | 0.0862 | 3556      | 123114 | 3918       | 1.1413          | 0.1906 | 21511         | 0.1956 | 0.3114 | 0.6886 |
+| 1.1735        | 23.0  | 68471 | 0.0847 | 3431      | 123623 | 4013       | 1.1381          | 0.1877 | 21127         | 0.1928 | 0.3067 | 0.6933 |
+| 1.1556        | 24.0  | 71448 | 0.0839 | 3668      | 123854 | 3698       | 1.1282          | 0.1845 | 20659         | 0.1891 | 0.3015 | 0.6985 |
+| 1.1538        | 25.0  | 74425 | 0.0819 | 3475      | 124201 | 3716       | 1.1240          | 0.1823 | 20505         | 0.1869 | 0.2986 | 0.7014 |
+| 1.1078        | 26.0  | 77402 | 0.0819 | 3410      | 124426 | 3751       | 1.1259          | 0.1810 | 20345         | 0.1856 | 0.2965 | 0.7035 |
+| 1.1539        | 27.0  | 80379 | 0.0805 | 3333      | 124879 | 3716       | 1.1152          | 0.1779 | 19969         | 0.1823 | 0.2916 | 0.7084 |
+
+
+### Framework versions
+
+- Transformers 4.40.0.dev0
+- Pytorch 2.2.0+rocm5.6
+- Datasets 2.18.0
+- Tokenizers 0.15.2

config.json

@@ -0,0 +1,267 @@
+{
+  "architectures": [
+    "JointCTCAttentionEncoderDecoder"
+  ],
+  "bos_token_id": 0,
+  "ctc_weight": 0.3,
+  "decoder": {
+    "_name_or_path": "Lakoc/gpt2_tiny_decoder_6_layers",
+    "activation_function": "gelu_new",
+    "add_cross_attention": true,
+    "architectures": null,
+    "attn_pdrop": 0.1,
+    "bad_words_ids": null,
+    "begin_suppress_tokens": null,
+    "bos_token_id": 50256,
+    "chunk_size_feed_forward": 0,
+    "cross_attention_hidden_size": null,
+    "decoder_start_token_id": null,
+    "diversity_penalty": 0.0,
+    "do_sample": false,
+    "early_stopping": false,
+    "embd_pdrop": 0.1,
+    "encoder_no_repeat_ngram_size": 0,
+    "eos_token_id": 50256,
+    "exponential_decay_length_penalty": null,
+    "finetuning_task": null,
+    "forced_bos_token_id": null,
+    "forced_eos_token_id": null,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "initializer_range": 0.02,
+    "is_decoder": true,
+    "is_encoder_decoder": false,
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "layer_norm_epsilon": 1e-05,
+    "length_penalty": 1.0,
+    "max_length": 20,
+    "min_length": 0,
+    "model_type": "gpt2",
+    "n_embd": 256,
+    "n_head": 4,
+    "n_inner": 2048,
+    "n_layer": 6,
+    "n_positions": 1024,
+    "no_repeat_ngram_size": 0,
+    "num_beam_groups": 1,
+    "num_beams": 1,
+    "num_return_sequences": 1,
+    "output_attentions": false,
+    "output_hidden_size": 256,
+    "output_hidden_states": false,
+    "output_scores": false,
+    "pad_token_id": null,
+    "pos_emb_fixed": true,
+    "prefix": null,
+    "problem_type": null,
+    "pruned_heads": {},
+    "remove_invalid_values": false,
+    "reorder_and_upcast_attn": false,
+    "repetition_penalty": 1.0,
+    "resid_pdrop": 0.1,
+    "return_dict": true,
+    "return_dict_in_generate": false,
+    "scale_attn_by_inverse_layer_idx": false,
+    "scale_attn_weights": true,
+    "sep_token_id": null,
+    "summary_activation": null,
+    "summary_first_dropout": 0.1,
+    "summary_proj_to_labels": true,
+    "summary_type": "cls_index",
+    "summary_use_proj": true,
+    "suppress_tokens": null,
+    "task_specific_params": null,
+    "temperature": 1.0,
+    "tf_legacy_loss": false,
+    "tie_encoder_decoder": false,
+    "tie_word_embeddings": false,
+    "tokenizer_class": null,
+    "top_k": 50,
+    "top_p": 1.0,
+    "torch_dtype": null,
+    "torchscript": false,
+    "typical_p": 1.0,
+    "use_bfloat16": false,
+    "use_cache": true,
+    "vocab_size": 1000
+  },
+  "decoder_start_token_id": 0,
+  "encoder": {
+    "_name_or_path": "Lakoc/fisher_ebranchformer_enc_12_layers_fixed",
+    "activation_dropout": 0.1,
+    "adapter_attn_dim": null,
+    "adapter_kernel_size": 3,
+    "adapter_stride": 2,
+    "add_adapter": false,
+    "add_cross_attention": false,
+    "apply_spec_augment": false,
+    "architectures": null,
+    "attention_dropout": 0.1,
+    "bad_words_ids": null,
+    "begin_suppress_tokens": null,
+    "bos_token_id": 1,
+    "chunk_size_feed_forward": 0,
+    "classifier_proj_size": 256,
+    "codevector_dim": 256,
+    "conformer_conv_dropout": 0.1,
+    "contrastive_logits_temperature": 0.1,
+    "conv_bias": false,
+    "conv_depthwise_kernel_size": 31,
+    "conv_dim": [
+      256,
+      256
+    ],
+    "conv_kernel": [
+      3,
+      3
+    ],
+    "conv_padding": [
+      1,
+      1
+    ],
+    "conv_stride": [
+      2,
+      2
+    ],
+    "cross_attention_hidden_size": null,
+    "csgu_activation": "identity",
+    "csgu_conv_dropout": 0.1,
+    "csgu_kernel_size": 31,
+    "csgu_use_linear_after_conv": false,
+    "ctc_loss_reduction": "mean",
+    "ctc_zero_infinity": true,
+    "decoder_start_token_id": null,
+    "diversity_loss_weight": 0.1,
+    "diversity_penalty": 0.0,
+    "do_sample": false,
+    "do_stable_layer_norm": false,
+    "early_stopping": false,
+    "encoder_no_repeat_ngram_size": 0,
+    "eos_token_id": 2,
+    "exponential_decay_length_penalty": null,
+    "fe_position_embeddings": false,
+    "feat_extract_activation": "gelu",
+    "feat_extract_norm": "group",
+    "feat_proj_dropout": 0.0,
+    "feat_quantizer_dropout": 0.0,
+    "final_dropout": 0.1,
+    "finetuning_task": null,
+    "forced_bos_token_id": null,
+    "forced_eos_token_id": null,
+    "hidden_act": "gelu",
+    "hidden_dropout": 0.1,
+    "hidden_size": 256,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "initializer_range": 0.02,
+    "intermediate_size": 1024,
+    "is_causal": false,
+    "is_decoder": false,
+    "is_encoder_decoder": false,
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "layer_norm_eps": 1e-05,
+    "layerdrop": 0.0,
+    "length_penalty": 1.0,
+    "mask_feature_length": 10,
+    "mask_feature_min_masks": 0,
+    "mask_feature_prob": 0.0,
+    "mask_time_length": 10,
+    "mask_time_min_masks": 2,
+    "mask_time_prob": 0.05,
+    "max_length": 20,
+    "max_source_positions": 1024,
+    "merge_conv_kernel": 31,
+    "min_length": 0,
+    "model_type": "wav2vec2-ebranchformer",
+    "no_repeat_ngram_size": 0,
+    "num_adapter_layers": 3,
+    "num_attention_heads": 4,
+    "num_beam_groups": 1,
+    "num_beams": 1,
+    "num_codevector_groups": 2,
+    "num_codevectors_per_group": 320,
+    "num_conv_pos_embedding_groups": 16,
+    "num_conv_pos_embeddings": 128,
+    "num_fbanks": 80,
+    "num_feat_extract_layers": 2,
+    "num_hidden_layers": 12,
+    "num_mel_bins": 80,
+    "num_negatives": 100,
+    "num_return_sequences": 1,
+    "output_attentions": false,
+    "output_hidden_size": 256,
+    "output_hidden_states": false,
+    "output_scores": false,
+    "pad_token_id": 3,
+    "position_embeddings_type": "relative",
+    "prefix": null,
+    "problem_type": null,
+    "proj_codevector_dim": 256,
+    "pruned_heads": {},
+    "remove_invalid_values": false,
+    "repetition_penalty": 1.0,
+    "return_dict": true,
+    "return_dict_in_generate": false,
+    "rotary_embedding_base": 10000,
+    "sep_token_id": null,
+    "suppress_tokens": null,
+    "task_specific_params": null,
+    "tdnn_dilation": [
+      1,
+      2,
+      3,
+      1,
+      1
+    ],
+    "tdnn_dim": [
+      512,
+      512,
+      512,
+      512,
+      1500
+    ],
+    "tdnn_kernel": [
+      5,
+      3,
+      3,
+      1,
+      1
+    ],
+    "temperature": 1.0,
+    "tf_legacy_loss": false,
+    "tie_encoder_decoder": false,
+    "tie_word_embeddings": true,
+    "tokenizer_class": null,
+    "top_k": 50,
+    "top_p": 1.0,
+    "torch_dtype": null,
+    "torchscript": false,
+    "typical_p": 1.0,
+    "use_bfloat16": false,
+    "use_fbanks": true,
+    "use_macaron_ff": true,
+    "use_weighted_layer_sum": false,
+    "vocab_size": 1000,
+    "xvector_output_dim": 512
+  },
+  "eos_token_id": 1,
+  "is_encoder_decoder": true,
+  "lsm_factor": 0.1,
+  "mask_token_id": 4,
+  "model_type": "joint_aed_ctc_speech-encoder-decoder",
+  "pad_token_id": 3,
+  "shared_lm_head": false,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.40.0.dev0"
+}

generation_config.json

@@ -0,0 +1,15 @@
+{
+  "apply_eos_space_trick": false,
+  "bos_token_id": 0,
+  "ctc_margin": 0,
+  "ctc_weight": 0.0,
+  "decoder_start_token_id": 0,
+  "eos_space_trick_weight": 0.0,
+  "eos_token_id": 1,
+  "lm_model": null,
+  "lm_weight": 0.0,
+  "max_length": 512,
+  "pad_token_id": 3,
+  "space_token_id": -1,
+  "transformers_version": "4.40.0.dev0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec9aa716122a67ba716a0cc098837038e2f1c645a2257b3ed26bf4009ed8a086
+size 143219080

predictions_common_voice_13_en_common_voice_13_en_test_wer22.42_hyp.trn.snt.utt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f352bfd80bac176c188137cef685f041653a5d8938485d59d6d9e90d8daaa55
+size 11467228

predictions_common_voice_13_en_common_voice_13_en_test_wer22.42_hyp.trn.sys

@@ -0,0 +1,18 @@
+
+
+
+                     SYSTEM SUMMARY PERCENTAGES by SPEAKER                      
+
+,------------------------------------------------------------------------------------------------------------------------------------------------------------------------.
+|/scratch/project_465000836/ipoloka/huggingface_asr/experiments/decred/commonvoice/ED_small_cv_v2/predictions_common_voice_13_en_common_voice_13_en_test_wer22.42_hyp.trn|
+|------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+|      SPKR         |       # Snt               # Wrd        |      Corr               Sub               Del               Ins               Err             S.Err       |
+|-------------------+----------------------------------------+-----------------------------------------------------------------------------------------------------------|
+|      utt          |      16384               144172        |      80.7              16.1               3.1               3.2              22.4              67.6       |
+|========================================================================================================================================================================|
+|      Sum/Avg      |      16384               144172        |      80.7              16.1               3.1               3.2              22.4              67.6       |
+|========================================================================================================================================================================|
+|       Mean        |     16384.0             144172.0       |      80.7              16.1               3.1               3.2              22.4              67.6       |
+|       S.D.        |        0.0                 0.0         |       0.0               0.0               0.0               0.0               0.0               0.0       |
+|      Median       |     16384.0             144172.0       |      80.7              16.1               3.1               3.2              22.4              67.6       |
+`------------------------------------------------------------------------------------------------------------------------------------------------------------------------'

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c8ce3e41c814767e6cf45a76302247ed0c308f558534b18d8ec51d5bbcd57c9f
+size 5816