update model card README.md

README.md

@@ -0,0 +1,81 @@
+---
+language:
+- ar
+license: apache-2.0
+tags:
+- whisper-event
+- generated_from_trainer
+datasets:
+- mozilla-foundation/common_voice_11_0
+metrics:
+- wer
+model-index:
+- name: Whisper Small ar
+  results:
+  - task:
+      name: Automatic Speech Recognition
+      type: automatic-speech-recognition
+    dataset:
+      name: common_voice_11_0
+      type: mozilla-foundation/common_voice_11_0
+      config: ar
+      split: test
+      args: ar
+    metrics:
+    - name: Wer
+      type: wer
+      value: 73.65866666666666
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# Whisper Small ar
+
+This model is a fine-tuned version of [openai/whisper-small](https://huggingface.co/openai/whisper-small) on the common_voice_11_0 dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.3855
+- Wer: 73.6587
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 1e-05
+- train_batch_size: 16
+- eval_batch_size: 16
+- seed: 42
+- gradient_accumulation_steps: 2
+- total_train_batch_size: 32
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 500
+- training_steps: 2000
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Wer     |
+|:-------------:|:-----:|:----:|:---------------:|:-------:|
+| 0.1151        | 2.12  | 2000 | 0.3855          | 73.6587 |
+
+
+### Framework versions
+
+- Transformers 4.26.0.dev0
+- Pytorch 1.13.0+cu117
+- Datasets 2.8.1.dev0
+- Tokenizers 0.12.1

fine-tune-whisper-streaming.ipynb

@@ -2,7 +2,7 @@
  "cells": [
   {
    "cell_type": "markdown",
-   "id": "5eb0df35",
+   "id": "7ee06e76",
    "metadata": {},
    "source": [
     "# Fine-Tune Whisper With 🤗 Transformers and Streaming Mode"
@@ -10,7 +10,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "df8cc52a",
+   "id": "e840bc77",
    "metadata": {},
    "source": [
     "In this Colab, we present a step-by-step guide on fine-tuning Whisper with Hugging Face 🤗 Transformers on 400 hours of speech data! Using streaming mode, we'll show how you can train a speech recongition model on any dataset, irrespective of size. With streaming mode, storage requirements are no longer a consideration: you can train a model on whatever dataset you want, even if it's download size exceeds your devices disk space. How can this be possible? It simply seems too good to be true! Well, rest assured it's not 😉 Carry on reading to find out more."
@@ -18,7 +18,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "143f0620",
+   "id": "a2ab74b4",
    "metadata": {},
    "source": [
     "## Introduction"
@@ -26,7 +26,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "37745350",
+   "id": "ebedf365",
    "metadata": {},
    "source": [
     "Speech recognition datasets are large. A typical speech dataset consists of approximately 100 hours of audio-transcription data, requiring upwards of 130GB of storage space for download and preparation. For most ASR researchers, this is already at the upper limit of what is feasible for disk space. So what happens when we want to train on a larger dataset? The full [LibriSpeech](https://huggingface.co/datasets/librispeech_asr) dataset consists of 960 hours of audio data. Kensho's [SPGISpeech](https://huggingface.co/datasets/kensho/spgispeech) contains 5,000 hours of audio data. ML Commons [People's Speech](https://huggingface.co/datasets/MLCommons/peoples_speech) contains **30,000+** hours of audio data! Do we need to bite the bullet and buy additional storage? Or is there a way we can train on all of these datasets with no disk drive requirements?\n",
@@ -42,7 +42,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e4c2618e",
+   "id": "76a56c73",
    "metadata": {},
    "source": [
     "<figure>\n",
@@ -53,7 +53,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "2daf6b53",
+   "id": "ccdf81af",
    "metadata": {},
    "source": [
     "This notebook provides a guide to fine-tuning on the task of _speech recognition_, which involves learning a\n",
@@ -92,7 +92,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "409c4267",
+   "id": "76a78106",
    "metadata": {},
    "source": [
     "## Load Dataset with Streaming"
@@ -100,7 +100,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "a0ce3342",
+   "id": "24a396ef",
    "metadata": {},
    "source": [
     "This is where the magic happens! We'll first write a wrapper function around 🤗 Datasets `load_dataset` method. This function downloads the required splits using streaming mode by forcing `streaming=True` in the `load_dataset` method. Multiple splits can be combined (interleaved) by concatenating them with the \"+\" symbol when specifying the split name, e.g. `split=train+validation` will return a single split with the training and validation splits interleaved together. The function has the same arguments and key-word arguments as 🤗 Datasets `load_dataset` method, so we can use it in exactly the same way!"
@@ -109,7 +109,7 @@
   {
    "cell_type": "code",
    "execution_count": 1,
-   "id": "e49e547c",
+   "id": "5b120832",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -130,7 +130,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "2b7da79d",
+   "id": "a4c01c76",
    "metadata": {},
    "source": [
     "We'll train our system on the Spanish split of [Common Voice 11](https://huggingface.co/datasets/mozilla-foundation/common_voice_11_0). We can see how much training data we have by viewing the [language page](https://commonvoice.mozilla.org/en/datasets) on the Common Voice website. The Spanish split has over 400 hours of labelled training data - that's enourmous! More than we could ever fit on a Google Colab or a standard workstation. But with streaming mode, we'll only download data as and when we need it, making training on this dataset possible!\n",
@@ -143,7 +143,7 @@
   {
    "cell_type": "code",
    "execution_count": 2,
-   "id": "211fdf8c",
+   "id": "046af31a",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -157,7 +157,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c483c427",
+   "id": "10730e4d",
    "metadata": {},
    "source": [
     "## Prepare Processor and Pre-Process Data"
@@ -165,7 +165,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "0a811a5e",
+   "id": "7f1a10c5",
    "metadata": {},
    "source": [
     "The ASR pipeline can be de-composed into three stages: \n",
@@ -186,7 +186,7 @@
   {
    "cell_type": "code",
    "execution_count": 3,
-   "id": "57b1770f",
+   "id": "2c491742",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -197,7 +197,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "cec6a161",
+   "id": "cce2486f",
    "metadata": {},
    "source": [
     "### Pre-Process Data"
@@ -205,7 +205,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "2166ce4e",
+   "id": "e089f77e",
    "metadata": {},
    "source": [
     "Let's have a look at the dataset features. Pay particular attention to the `\"audio\"` column - this details the sampling rate of our audio inputs:"
@@ -214,7 +214,7 @@
   {
    "cell_type": "code",
    "execution_count": 4,
-   "id": "c8031a36",
+   "id": "558b546f",
    "metadata": {},
    "outputs": [
     {
@@ -244,7 +244,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "66ea302f",
+   "id": "fdea2685",
    "metadata": {},
    "source": [
     "Since our input audio is sampled at 48kHz, we need to _downsample_ it to\n",
@@ -260,7 +260,7 @@
   {
    "cell_type": "code",
    "execution_count": 5,
-   "id": "a7374f6b",
+   "id": "fb5ab66e",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -271,7 +271,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3d61f48f",
+   "id": "a0fe8d76",
    "metadata": {},
    "source": [
     "We'll define our pre-processing strategy. We advise that you **do not** lower-case the transcriptions or remove punctuation unless mixing different datasets. This will enable you to fine-tune Whisper models that can predict punctuation and casing. Later, you will see how we can evaluate the predictions without punctuation or casing, so that the models benefit from the WER improvement obtained by normalising the transcriptions while still predicting fully formatted transcriptions."
@@ -280,7 +280,7 @@
   {
    "cell_type": "code",
    "execution_count": 6,
-   "id": "ede17e4f",
+   "id": "e458e800",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -294,7 +294,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f09fbe96",
+   "id": "1c68a2f7",
    "metadata": {},
    "source": [
     "Now we can write a function to prepare our data ready for the model:\n",
@@ -307,7 +307,7 @@
   {
    "cell_type": "code",
    "execution_count": 7,
-   "id": "e76b6638",
+   "id": "fb7ff9e0",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -334,7 +334,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "8255126a",
+   "id": "f7f8f446",
    "metadata": {},
    "source": [
     "We can apply the data preparation function to all of our training examples using 🤗 Datasets' `.map` method. We'll remove all of the columns from the raw training data, leaving just the `input_features` and `labels` defined in the `prepare_dataset` function:"
@@ -343,7 +343,7 @@
   {
    "cell_type": "code",
    "execution_count": 8,
-   "id": "062d63f9",
+   "id": "20ad4d6d",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -352,7 +352,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "eb19118e",
+   "id": "522d4732",
    "metadata": {},
    "source": [
     "We can now define how we shuffle the data in the train split. The size of the subset we load is set by the variable `buffer_size`. You can increase or decrease this depending on your memory constraints. In this example, the `buffer_size` is set to 500, meaning 500 samples are loaded before shuffling across the subset. The larger we set this value, the closer to True offline shuffling. The `seed` is set for reproducibility:"
@@ -361,7 +361,7 @@
   {
    "cell_type": "code",
    "execution_count": 9,
-   "id": "25342d4d",
+   "id": "149e1f43",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -373,7 +373,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e76d637f",
+   "id": "3352d9b2",
    "metadata": {},
    "source": [
     "Finally, we filter any training data with audio samples longer than 30s. These samples would otherwise be truncated by the Whisper feature-extractor which could affect the stability of training. We define a function that returns `True` for samples that are less than 30s, and `False` for those that are longer:"
@@ -382,7 +382,7 @@
   {
    "cell_type": "code",
    "execution_count": 10,
-   "id": "0bdf50a7",
+   "id": "81bf2f51",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -394,7 +394,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1e98b230",
+   "id": "77456cf9",
    "metadata": {},
    "source": [
     "We apply our filter function to all samples of our training dataset through 🤗 Datasets' `.filter` method:"
@@ -403,7 +403,7 @@
   {
    "cell_type": "code",
    "execution_count": 11,
-   "id": "4496f42a",
+   "id": "3d13426b",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -415,7 +415,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "cf987c26",
+   "id": "482d7981",
    "metadata": {},
    "source": [
     "## Training and Evaluation"
@@ -423,7 +423,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "391bbd26",
+   "id": "d23e3a80",
    "metadata": {},
    "source": [
     "Now that we've prepared our data, we're ready to dive into the training pipeline. \n",
@@ -441,7 +441,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "4226eb22",
+   "id": "67d93ae7",
    "metadata": {},
    "source": [
     "### Define a Data Collator"
@@ -449,7 +449,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "2a8d1ca6",
+   "id": "8c180b86",
    "metadata": {},
    "source": [
     "The data collator for a sequence-to-sequence speech model is unique in the sense that it \n",
@@ -473,7 +473,7 @@
   {
    "cell_type": "code",
    "execution_count": 12,
-   "id": "c96bef0e",
+   "id": "aa4f7d60",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -512,7 +512,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3cb76e95",
+   "id": "d9393af2",
    "metadata": {},
    "source": [
     "Let's initialise the data collator we've just defined:"
@@ -521,7 +521,7 @@
   {
    "cell_type": "code",
    "execution_count": 13,
-   "id": "4979b797",
+   "id": "69ea823d",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -530,7 +530,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "eb008e83",
+   "id": "18ec9be0",
    "metadata": {},
    "source": [
     "### Evaluation Metrics"
@@ -538,7 +538,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "fd0c691d",
+   "id": "32ab245e",
    "metadata": {},
    "source": [
     "We'll use the word error rate (WER) metric, the 'de-facto' metric for assessing \n",
@@ -548,7 +548,7 @@
   {
    "cell_type": "code",
    "execution_count": 14,
-   "id": "3044e87b",
+   "id": "73dee279",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -559,7 +559,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "b851965a",
+   "id": "f3951d48",
    "metadata": {},
    "source": [
     "We then simply have to define a function that takes our model \n",
@@ -577,7 +577,7 @@
   {
    "cell_type": "code",
    "execution_count": 15,
-   "id": "5d367fe0",
+   "id": "6974fa39",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -609,7 +609,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3335d64a",
+   "id": "95d6936b",
    "metadata": {},
    "source": [
     "### Load a Pre-Trained Checkpoint"
@@ -617,7 +617,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "70cffe1a",
+   "id": "f5213c1d",
    "metadata": {},
    "source": [
     "Now let's load the pre-trained Whisper `small` checkpoint. Again, this \n",
@@ -627,7 +627,7 @@
   {
    "cell_type": "code",
    "execution_count": 16,
-   "id": "377f7eb4",
+   "id": "7d22d4d7",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -638,7 +638,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "b1125075",
+   "id": "8ada21a6",
    "metadata": {},
    "source": [
     "Override generation arguments - no tokens are forced as decoder outputs (see [`forced_decoder_ids`](https://huggingface.co/docs/transformers/main_classes/text_generation#transformers.generation_utils.GenerationMixin.generate.forced_decoder_ids)), no tokens are suppressed during generation (see [`suppress_tokens`](https://huggingface.co/docs/transformers/main_classes/text_generation#transformers.generation_utils.GenerationMixin.generate.suppress_tokens)). Set `use_cache` to False since we're using gradient checkpointing, and the two are incompatible:"
@@ -647,7 +647,7 @@
   {
    "cell_type": "code",
    "execution_count": 17,
-   "id": "fb0955c6",
+   "id": "5c1266ac",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -658,7 +658,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "21eb2e5a",
+   "id": "628c7f4e",
    "metadata": {},
    "source": [
     "### Define the Training Configuration"
@@ -666,7 +666,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "0e97f32d",
+   "id": "309768d9",
    "metadata": {},
    "source": [
     "In the final step, we define all the parameters related to training. Here, you can set the `max_steps` to train for longer. For more detail on the training arguments, refer to the Seq2SeqTrainingArguments [docs](https://huggingface.co/docs/transformers/main_classes/trainer#transformers.Seq2SeqTrainingArguments)."
@@ -675,7 +675,7 @@
   {
    "cell_type": "code",
    "execution_count": 18,
-   "id": "1a3a6612",
+   "id": "253e7071",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -707,7 +707,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "0bded5c7",
+   "id": "e9e4f70c",
    "metadata": {},
    "source": [
     "**Note**: if one does not want to upload the model checkpoints to the Hub, \n",
@@ -716,7 +716,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "6c7feb58",
+   "id": "a2d970ab",
    "metadata": {},
    "source": [
     "We then define a custom [Callback](https://huggingface.co/docs/transformers/main_classes/callback) that is called by the 🤗 Trainer on the end of each epoch. The Callback reinitialises and reshuffles the streaming dataset at the beginning of each new epoch - this gives different shuffling across our subsets for every epoch."
@@ -725,7 +725,7 @@
   {
    "cell_type": "code",
    "execution_count": 19,
-   "id": "d17f4ee1",
+   "id": "8c7821b6",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -744,7 +744,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "0b679650",
+   "id": "1a52a229",
    "metadata": {},
    "source": [
     "We can forward the training arguments to the 🤗 Trainer along with our model,\n",
@@ -754,7 +754,7 @@
   {
    "cell_type": "code",
    "execution_count": 20,
-   "id": "274bc709",
+   "id": "1d86db55",
    "metadata": {},
    "outputs": [
     {
@@ -786,7 +786,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "932e7d2e",
+   "id": "e8d4699b",
    "metadata": {},
    "source": [
     "We'll save the model and processor to the output directory before training:"
@@ -795,7 +795,7 @@
   {
    "cell_type": "code",
    "execution_count": 21,
-   "id": "9413aba6",
+   "id": "8ec1bd4e",
    "metadata": {},
    "outputs": [
     {
@@ -818,7 +818,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "552289a8",
+   "id": "cedff338",
    "metadata": {},
    "source": [
     "### Training"
@@ -826,7 +826,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "48ca9e46",
+   "id": "d5ac0619",
    "metadata": {},
    "source": [
     "Training will take approximately 5-10 hours depending on your GPU. The peak GPU memory for the given training configuration is approximately 36GB. \n",
@@ -840,8 +840,8 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "8401cbc1",
+   "execution_count": 22,
+   "id": "c66858bf",
    "metadata": {},
    "outputs": [
     {
@@ -868,8 +868,8 @@
        "\n",
        "    <div>\n",
        "      \n",
-       "      <progress value='2001' max='2000' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
-       "      [2000/2000 5:52:53, Epoch 2.12/9223372036854775807]\n",
+       "      <progress value='2000' max='2000' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
+       "      [2000/2000 9:44:45, Epoch 2/9223372036854775807]\n",
        "    </div>\n",
        "    <table border=\"1\" class=\"dataframe\">\n",
        "  <thead>\n",
@@ -877,9 +877,16 @@
        "      <th>Step</th>\n",
        "      <th>Training Loss</th>\n",
        "      <th>Validation Loss</th>\n",
+       "      <th>Wer</th>\n",
        "    </tr>\n",
        "  </thead>\n",
        "  <tbody>\n",
+       "    <tr>\n",
+       "      <td>2000</td>\n",
+       "      <td>0.115100</td>\n",
+       "      <td>0.385472</td>\n",
+       "      <td>73.658667</td>\n",
+       "    </tr>\n",
        "  </tbody>\n",
        "</table><p>"
       ],
@@ -10682,8 +10689,50 @@
       "  \"transformers_version\": \"4.26.0.dev0\",\n",
       "  \"use_cache\": false\n",
       "}\n",
-      "\n"
+      "\n",
+      "Generate config GenerationConfig {\n",
+      "  \"begin_suppress_tokens\": [\n",
+      "    220,\n",
+      "    50257\n",
+      "  ],\n",
+      "  \"bos_token_id\": 50257,\n",
+      "  \"decoder_start_token_id\": 50258,\n",
+      "  \"eos_token_id\": 50257,\n",
+      "  \"max_length\": 448,\n",
+      "  \"pad_token_id\": 50257,\n",
+      "  \"suppress_tokens\": [],\n",
+      "  \"transformers_version\": \"4.26.0.dev0\",\n",
+      "  \"use_cache\": false\n",
+      "}\n",
+      "\n",
+      "Saving model checkpoint to ./checkpoint-2000\n",
+      "Configuration saved in ./checkpoint-2000/config.json\n",
+      "Model weights saved in ./checkpoint-2000/pytorch_model.bin\n",
+      "Feature extractor saved in ./checkpoint-2000/preprocessor_config.json\n",
+      "tokenizer config file saved in ./checkpoint-2000/tokenizer_config.json\n",
+      "Special tokens file saved in ./checkpoint-2000/special_tokens_map.json\n",
+      "added tokens file saved in ./checkpoint-2000/added_tokens.json\n",
+      "Feature extractor saved in ./preprocessor_config.json\n",
+      "tokenizer config file saved in ./tokenizer_config.json\n",
+      "Special tokens file saved in ./special_tokens_map.json\n",
+      "added tokens file saved in ./added_tokens.json\n",
+      "\n",
+      "\n",
+      "Training completed. Do not forget to share your model on huggingface.co/models =)\n",
+      "\n",
+      "\n",
+      "Loading best model from ./checkpoint-2000 (score: 73.65866666666666).\n"
      ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "TrainOutput(global_step=2000, training_loss=0.3622547821998596, metrics={'train_runtime': 35158.6606, 'train_samples_per_second': 1.82, 'train_steps_per_second': 0.057, 'total_flos': 1.847581449928704e+19, 'train_loss': 0.3622547821998596, 'epoch': 2.12})"
+      ]
+     },
+     "execution_count": 22,
+     "metadata": {},
+     "output_type": "execute_result"
     }
    ],
    "source": [
@@ -10692,7 +10741,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "128c5b2a",
+   "id": "41f44439",
    "metadata": {
     "pycharm": {
      "name": "#%% md\n"
@@ -10704,7 +10753,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "33c489c4",
+   "id": "2c05ac2d",
    "metadata": {},
    "source": [
     "We can label our checkpoint with the `whisper-event` tag on push by setting the appropriate key-word arguments (kwargs):"
@@ -10712,8 +10761,8 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "f54e6c30",
+   "execution_count": 23,
+   "id": "005fbee5",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -10730,7 +10779,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "bef7e4be",
+   "id": "a92005f6",
    "metadata": {},
    "source": [
     "The training results can now be uploaded to the Hub. To do so, execute the `push_to_hub` command:"
@@ -10739,9 +10788,53 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "f1ada39d",
+   "id": "9aa840fa",
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Saving model checkpoint to ./\n",
+      "Configuration saved in ./config.json\n",
+      "Model weights saved in ./pytorch_model.bin\n",
+      "Feature extractor saved in ./preprocessor_config.json\n",
+      "tokenizer config file saved in ./tokenizer_config.json\n",
+      "Special tokens file saved in ./special_tokens_map.json\n",
+      "added tokens file saved in ./added_tokens.json\n",
+      "Several commits (2) will be pushed upstream.\n",
+      "The progress bars may be unreliable.\n"
+     ]
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "8e232938a5444cb2810ce5010da7951a",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Upload file pytorch_model.bin:   0%|          | 32.0k/922M [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "47d78028cf7445cab5aca33b58f58b24",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Upload file runs/Dec21_00-50-57_c4dc565b4234/events.out.tfevents.1671583868.c4dc565b4234.3974847.0: 100%|#####…"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
    "source": [
     "trainer.push_to_hub(**kwargs)"
    ]