![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "# Project IO Achievement - Speech-to-text Model Comparizon"
+ ],
+ "metadata": {
+ "id": "PIbogPXyM0wr"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "## Problem Definition\n",
+ "\n",
+ "We'll start by developing this in Colab, and we want a simple speech transcription. It is known that there are some speed issues in terms of transcription, so this Colab should implement and compare pros/cons of:\n",
+ "\n",
+ "- OpenAI Whisper API \n",
+ "- Huggingface API (by downloading model) \n",
+ "- Huggingface API (by using API endpoint, so you never download the model, you just use the inference API + Huggingface Hub).\n",
+ "\n",
+ "Which seems to do the best, and are there differences? What are the caveats of using one approach vs the other?\n"
+ ],
+ "metadata": {
+ "id": "x_Vp8SiKM4p1"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "## Libraries\n",
+ "\n",
+ "This section will install and import some important libraries such as Langchain, openai, Gradio, and so on"
+ ],
+ "metadata": {
+ "id": "o_60X8H3NEne"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# install libraries here\n",
+ "# -q flag for \"quiet\" install\n",
+ "%%capture\n",
+ "!pip install -q langchain\n",
+ "!pip install -q openai\n",
+ "!pip install -q gradio\n",
+ "!pip install -q transformers\n",
+ "!pip install -q datasets\n",
+ "!pip install -q huggingsound\n",
+ "!pip install -q torchaudio\n",
+ "!pip install -q git+https://github.com/openai/whisper.git"
+ ],
+ "metadata": {
+ "id": "pxcqXgg2aAN7"
+ },
+ "execution_count": 1,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 44,
+ "metadata": {
+ "id": "pEjM1tLsMZBq"
+ },
+ "outputs": [],
+ "source": [
+ "# import libraries here\n",
+ "from langchain.llms import OpenAI\n",
+ "from langchain.prompts import PromptTemplate\n",
+ "from langchain.document_loaders import TextLoader\n",
+ "from langchain.indexes import VectorstoreIndexCreator\n",
+ "from langchain import ConversationChain, LLMChain, PromptTemplate\n",
+ "from langchain.chat_models import ChatOpenAI\n",
+ "from langchain.memory import ConversationBufferWindowMemory\n",
+ "from langchain.prompts import ChatPromptTemplate\n",
+ "from langchain.text_splitter import CharacterTextSplitter\n",
+ "from langchain.embeddings import OpenAIEmbeddings\n",
+ "import openai\n",
+ "import os\n",
+ "from getpass import getpass\n",
+ "from IPython.display import display, Javascript, HTML\n",
+ "from google.colab.output import eval_js\n",
+ "from base64 import b64decode\n",
+ "import ipywidgets as widgets\n",
+ "from IPython.display import clear_output\n",
+ "import time\n",
+ "import requests\n",
+ "from transformers import WhisperProcessor, WhisperForConditionalGeneration\n",
+ "from datasets import load_dataset\n",
+ "# from torchaudio.transforms import Resample\n",
+ "import whisper\n",
+ "from huggingsound import SpeechRecognitionModel\n",
+ "import numpy as np\n",
+ "import torch\n",
+ "import librosa\n",
+ "from datasets import load_dataset\n",
+ "from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor\n",
+ "from jiwer import wer\n",
+ "import pandas as pd\n",
+ "from IPython.display import display, HTML"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "## API Keys\n",
+ "\n",
+ "Use these cells to load the API keys required for this notebook. The below code cell uses the `getpass` library."
+ ],
+ "metadata": {
+ "id": "03KLZGI_a5W5"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "openai_api_key = getpass()\n",
+ "os.environ[\"OPENAI_API_KEY\"] = openai_api_key\n",
+ "openai.api_key = openai_api_key"
+ ],
+ "metadata": {
+ "id": "5smcWj4DbFgy",
+ "outputId": "ccfdf55a-72ba-40c7-bf25-b5e9e7bb5c41",
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ }
+ },
+ "execution_count": 3,
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "··········\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "## Data Loading"
+ ],
+ "metadata": {
+ "id": "M6IzVTjz5cex"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def download(audio_url, save_path):\n",
+ " r = requests.get(audio_url, stream = True)\n",
+ "\n",
+ " with open(save_path, \"wb\") as file:\n",
+ " for block in r.iter_content(chunk_size = 1024):\n",
+ " if block:\n",
+ " file.write(block)"
+ ],
+ "metadata": {
+ "id": "ObVO5iLX5kPh"
+ },
+ "execution_count": 4,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "audio_url1 = 'https://drive.google.com/uc?export=download&id=1fIPAktKZEScqcNqv4GTXvB7eukh41Arf'\n",
+ "# audio_url2 = 'https://drive.google.com/uc?export=download&id=1uPk3EbQokbY7NpoolE3vSoitHeySctpJ'\n",
+ "audio_url2 = 'https://drive.google.com/uc?export=download&id=1X0IcDLyQvyGBH8JBxRXpIIk1iaUae1ZE'\n",
+ "# audio_url3 = 'https://drive.google.com/uc?export=download&id=1gzluFXFPZ7t8tq32MTTKOozHNyb7s8Ll'\n",
+ "audio_url3 = 'https://drive.google.com/uc?export=download&id=1xn1q1uLr6HhDLmG7kX6ZllbX2AUFyAI4'\n",
+ "\n",
+ "# ds = load_dataset(\"hf-internal-testing/librispeech_asr_dummy\", \"clean\", split=\"validation\")\n",
+ "# sample = ds[0][\"audio\"]"
+ ],
+ "metadata": {
+ "id": "Pq6VAxbx5crg"
+ },
+ "execution_count": 5,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "download(audio_url1, 'audio1.mp3')\n",
+ "download(audio_url2, 'audio2.mp3')\n",
+ "download(audio_url3, 'audio3.mp3')"
+ ],
+ "metadata": {
+ "id": "MExVuGlL6dUn"
+ },
+ "execution_count": 6,
+ "outputs": []
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "## Models\n",
+ "\n",
+ "OpenAI Whisper API: https://platform.openai.com/docs/guides/speech-to-text\n",
+ "\n",
+ "Hugging Face API: https://huggingface.co/models?pipeline_tag=automatic-speech-recognition&sort=trending\n",
+ "- openai/whisper-large-v2\n",
+ "- pyannote/speaker-diarization ?\n",
+ "- jonatasgrosman/wav2vec2-large-xlsr-53-english\n",
+ "- openai/whisper-large\n",
+ "- openai/whisper-base\n",
+ "- openai/whisper-small\n",
+ "- openai/whisper-medium\n",
+ "- guillaumekln/faster-whisper-large-v2\n",
+ "- facebook/mms-1b-all"
+ ],
+ "metadata": {
+ "id": "2JspJunJbSqa"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ ""
+ ],
+ "metadata": {
+ "id": "C02r7hJgR90N"
+ }
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "### OpenAI Whisper API\n",
+ "\n",
+ "https://platform.openai.com/docs/guides/speech-to-text"
+ ],
+ "metadata": {
+ "id": "TlWF8u4dYnMJ"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "total_time = 0\n",
+ "runs = 10\n",
+ "\n",
+ "for i in range(runs):\n",
+ " file = open('audio1.mp3', \"rb\")\n",
+ "\n",
+ " start = time.time()\n",
+ " transcription1 = openai.Audio.transcribe(\"whisper-1\", file)\n",
+ " end = time.time()\n",
+ "\n",
+ " total_time += (end - start)\n",
+ "\n",
+ "average_time1 = total_time / runs\n",
+ "print(\"Average OpenAI Whisper API transcription time: \", average_time1)\n",
+ "\n",
+ "for i in range(runs):\n",
+ " file = open('audio2.mp3', \"rb\")\n",
+ "\n",
+ " start = time.time()\n",
+ " transcription2 = openai.Audio.transcribe(\"whisper-1\", file)\n",
+ " end = time.time()\n",
+ "\n",
+ " total_time += (end - start)\n",
+ "\n",
+ "average_time2 = total_time / runs\n",
+ "print(\"Average OpenAI Whisper API transcription time: \", average_time2)\n",
+ "\n",
+ "for i in range(runs):\n",
+ " file = open('audio3.mp3', \"rb\")\n",
+ "\n",
+ " start = time.time()\n",
+ " transcription3 = openai.Audio.transcribe(\"whisper-1\", file)\n",
+ " end = time.time()\n",
+ "\n",
+ " total_time += (end - start)\n",
+ "\n",
+ "average_time3 = total_time / runs\n",
+ "print(\"Average OpenAI Whisper API transcription time: \", average_time3)\n"
+ ],
+ "metadata": {
+ "id": "I5H1xfkQZRLV",
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ },
+ "outputId": "2cc03d5d-d253-4ea1-eb20-594adf3db9e9"
+ },
+ "execution_count": 12,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Average OpenAI Whisper API transcription time: 2.087932562828064\n",
+ "Average OpenAI Whisper API transcription time: 3.9225234985351562\n",
+ "Average OpenAI Whisper API transcription time: 5.70369656085968\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "transcription1['text']"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 87
+ },
+ "id": "7mCnVJuzZxZA",
+ "outputId": "3b87e57f-5a69-4da1-b5d9-02979f8adcbb"
+ },
+ "execution_count": 13,
+ "outputs": [
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/plain": [
+ "\"Sawyer and Susie stared at the big door in front of them. For twins, they didn't look anything alike. Sawyer was blonde and short. Susie was tall with dark hair and freckles. But those were just the smallest differences between them. In fact, they were pretty sure they were born on different planets, on opposite ends of the universe. Even so, they were stuck together as usual, and now they stood on their grandpa's front porch, deciding their next move.\""
+ ],
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "string"
+ }
+ },
+ "metadata": {},
+ "execution_count": 13
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "transcription2['text']"
+ ],
+ "metadata": {
+ "id": "sYUAZDPH_d35",
+ "outputId": "9c21b190-b34f-4bba-e5a0-e70529229a27",
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 70
+ }
+ },
+ "execution_count": 14,
+ "outputs": [
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/plain": [
+ "'Once upon a time, there was a giraffe named Ginger. Ginger lived in Kenya, a country in Africa. Like all giraffes, Ginger had a long neck and long legs. Because she was so tall, she was able to eat food from the very tops of the trees in the savannah. The savannah in Africa is an area with lots of grass and some trees.'"
+ ],
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "string"
+ }
+ },
+ "metadata": {},
+ "execution_count": 14
+ }
+ ]
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "transcription3['text']"
+ ],
+ "metadata": {
+ "id": "5Wt8k3_X_d87",
+ "outputId": "ac72bead-77f1-4c1b-c75c-31c949e00b3e",
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 70
+ }
+ },
+ "execution_count": 15,
+ "outputs": [
+ {
+ "output_type": "execute_result",
+ "data": {
+ "text/plain": [
+ "\"Well, how do you go about making a small rowboat? We just make the small scale model and draft it from that. Make a keel out. You make a scale model first? Most everybody does. Make a scale model or else they draft them out. Draw them out on paper. Either one you want to, it doesn't matter. How big are these scale models?\""
+ ],
+ "application/vnd.google.colaboratory.intrinsic+json": {
+ "type": "string"
+ }
+ },
+ "metadata": {},
+ "execution_count": 15
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "### openai/whisper-large-v2\n",
+ "\n",
+ "https://huggingface.co/openai/whisper-large-v2\n",
+ "\n",
+ "https://github.com/openai/whisper"
+ ],
+ "metadata": {
+ "id": "VtWGpXmEoRwK"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "def transcribe_audio(save_path, model_name, num_iterations):\n",
+ "\n",
+ " model = whisper.load_model(model_name)\n",
+ " total_times = []\n",
+ "\n",
+ " for i in range(num_iterations): # number of iterations\n",
+ " start = time.time()\n",
+ "\n",
+ " # load audio and pad/trim it to fit 30 seconds\n",
+ " audio = whisper.load_audio(save_path)\n",
+ " audio = whisper.pad_or_trim(audio)\n",
+ "\n",
+ " # make log-Mel spectrogram and move to the same device as the model\n",
+ " mel = whisper.log_mel_spectrogram(audio).to(model.device)\n",
+ "\n",
+ " # detect the spoken language\n",
+ " _, probs = model.detect_language(mel)\n",
+ "\n",
+ " # decode the audio\n",
+ " options = whisper.DecodingOptions()\n",
+ " result = whisper.decode(model, mel, options)\n",
+ "\n",
+ " end = time.time()\n",
+ " total_times.append(end - start)\n",
+ "\n",
+ " # Calculate the average\n",
+ " average_time = np.mean(total_times)\n",
+ " print(\"Average\", model_name, \"transcription time: \", average_time)\n",
+ "\n",
+ " # print the recognized text\n",
+ " print(result.text)\n",
+ "\n",
+ " return average_time, result.text\n"
+ ],
+ "metadata": {
+ "id": "VEEBEAncRLOd"
+ },
+ "execution_count": 16,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "whisper_large_v2_1 = transcribe_audio('audio1.mp3', \"large-v2\", 10)\n",
+ "whisper_large_v2_2 = transcribe_audio('audio2.mp3', \"large-v2\", 10)\n",
+ "whisper_large_v2_3 = transcribe_audio('audio3.mp3', \"large-v2\", 10)"
+ ],
+ "metadata": {
+ "id": "bWT-XFv4RO3G",
+ "outputId": "462aa825-9b22-4f0d-8944-15c25319e400",
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ }
+ },
+ "execution_count": 21,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Average large-v2 transcription time: 7.058776140213013\n",
+ "Sawyer and Susie stared at the big door in front of them. For twins, they didn't look anything alike. Sawyer was blonde and short, Susie was tall with dark hair and freckles. But those were just the smallest differences between them. In fact, they were pretty sure they were born on different planets, on opposite ends of the universe. Even so, they were stuck together as usual, and now they stood on their grandpa's front porch, deciding their next move.\n",
+ "Average large-v2 transcription time: 5.123278546333313\n",
+ "Once upon a time, there was a giraffe named Ginger. Ginger lived in Kenya, a country in Africa. Like all giraffes, Ginger had a long neck and long legs. Because she was so tall, she was able to eat food from the very tops of the trees in the savannah. The savannah in Africa is an area with lots of grass and some trees.\n",
+ "Average large-v2 transcription time: 5.159456348419189\n",
+ "How do you go about making a small rowboat? We just make the small scale model and draft it from that. Make a keel out. You make a scale model first? Most everybody does. Or else they draft them out, draw them out on paper. Either one you want to, it doesn't matter. How big are these scale models?\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "### whisper-large"
+ ],
+ "metadata": {
+ "id": "PHArXLRT1lVL"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "whisper_large_1 = transcribe_audio('audio1.mp3', \"large\", 10)\n",
+ "whisper_large_2 = transcribe_audio('audio2.mp3', \"large\", 10)\n",
+ "whisper_large_3 = transcribe_audio('audio3.mp3', \"large\", 10)"
+ ],
+ "metadata": {
+ "id": "42tKvN3M8zGY",
+ "outputId": "87fb4841-d0d7-49a0-f935-b17835173c58",
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ }
+ },
+ "execution_count": 22,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Average large transcription time: 7.4871098279953\n",
+ "Sawyer and Susie stared at the big door in front of them. For twins, they didn't look anything alike. Sawyer was blonde and short, Susie was tall with dark hair and freckles. But those were just the smallest differences between them. In fact, they were pretty sure they were born on different planets, on opposite ends of the universe. Even so, they were stuck together as usual, and now they stood on their grandpa's front porch, deciding their next move.\n",
+ "Average large transcription time: 5.103005862236023\n",
+ "Once upon a time, there was a giraffe named Ginger. Ginger lived in Kenya, a country in Africa. Like all giraffes, Ginger had a long neck and long legs. Because she was so tall, she was able to eat food from the very tops of the trees in the savannah. The savannah in Africa is an area with lots of grass and some trees.\n",
+ "Average large transcription time: 5.136638736724853\n",
+ "How do you go about making a small rowboat? We just make the small scale model and draft it from that. Make a keel out. You make a scale model first? Most everybody does. Or else they draft them out, draw them out on paper. Either one you want to, it doesn't matter. How big are these scale models?\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "### whisper-medium"
+ ],
+ "metadata": {
+ "id": "xEWJ9_il8OT8"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "whisper_medium_1 = transcribe_audio('audio1.mp3', \"medium\", 10)\n",
+ "whisper_medium_2 = transcribe_audio('audio2.mp3', \"medium\", 10)\n",
+ "whisper_medium_3 = transcribe_audio('audio3.mp3', \"medium\", 10)"
+ ],
+ "metadata": {
+ "id": "coMe9vGT8348",
+ "outputId": "a6b145cd-a26f-4dbd-db49-ba267df4f3c8",
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ }
+ },
+ "execution_count": 23,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "100%|██████████████████████████████████████| 1.42G/1.42G [00:10<00:00, 141MiB/s]\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Average medium transcription time: 4.867928552627563\n",
+ "Sawyer and Susie stared at the big door in front of them. For twins, they didn't look anything alike. Sawyer was blonde and short, Susie was tall with dark hair and freckles. But those were just the smallest differences between them. In fact, they were pretty sure they were born on different planets, on opposite ends of the universe. Even so, they were stuck together as usual, and now they stood on their grandpa's front porch, deciding their next move.\n",
+ "Average medium transcription time: 3.6824806213378904\n",
+ "Once upon a time, there was a giraffe named Ginger. Ginger lived in Kenya, a country in Africa. Like all giraffes, Ginger had a long neck and long legs. Because she was so tall, she was able to eat food from the very tops of the trees in the savanna. The savanna in Africa is an area with lots of grass and some trees.\n",
+ "Average medium transcription time: 4.072535729408264\n",
+ "How do you go about making a small rowboat? We just make the small scale model and draft it from there. Make a keel out. You make a scale model first? Most everybody does. Make a scale model. Or I say draft them out. Draw the model on paper. Either one you want to, it doesn't matter. How big are these scale models?\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "### whisper-small"
+ ],
+ "metadata": {
+ "id": "TAIdua7D8Py9"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "whisper_small_1 = transcribe_audio('audio1.mp3', \"small\", 10)\n",
+ "whisper_small_2 = transcribe_audio('audio2.mp3', \"small\", 10)\n",
+ "whisper_small_3 = transcribe_audio('audio3.mp3', \"small\", 10)"
+ ],
+ "metadata": {
+ "id": "4YGrdrQH-Dt5",
+ "outputId": "16f087ca-529c-45e0-a86f-5813e755c025",
+ "colab": {
+ "base_uri": "https://localhost:8080/"
+ }
+ },
+ "execution_count": 24,
+ "outputs": [
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "100%|████████████████████████████████████████| 461M/461M [00:04<00:00, 118MiB/s]\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "Average small transcription time: 2.7674500703811646\n",
+ "Sawyer and Susie stared at the big door in front of them. For twins, they didn't look anything alike. Sawyer was blonde and short, Susie was tall with dark hair and freckles, but those were just the smallest differences between them. In fact, they were pretty sure they were born on different planets, on opposite ends of the universe. Even so, they were stuck together as usual, and now they stood on their grandpa's front porch, deciding their next move.\n",
+ "Average small transcription time: 1.9433160781860352\n",
+ "Once upon a time, there was a giraffe named Ginger. Ginger lived in Kenya, a country in Africa. Like all giraffes, Ginger had a long neck and long legs. Because she was so tall, she was able to eat food from the very tops of the trees in the savannah. The savannah in Africa is an area with lots of grass and sun trees.\n",
+ "Average small transcription time: 2.037943434715271\n",
+ "How do you go about making a small robot? We just make the small scale model and draft it from there. Make a key loop. You make a scale model first? Most everybody does. Make a scale model or just draft them out. Draw them out on paper. Either when you want to do it or not. How big are these scale models?\n"
+ ]
+ }
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "source": [
+ "### jonatasgrosman/wav2vec2-large-xlsr-53-english"
+ ],
+ "metadata": {
+ "id": "m76G2guxCL6c"
+ }
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "LANG_ID = \"en\"\n",
+ "MODEL_ID = \"jonatasgrosman/wav2vec2-large-xlsr-53-english\"\n",
+ "SAMPLES = 10"
+ ],
+ "metadata": {
+ "id": "nR7KoHjWCMBy"
+ },
+ "execution_count": 25,
+ "outputs": []
+ },
+ {
+ "cell_type": "code",
+ "source": [
+ "# Load pre-trained model and tokenizer\n",
+ "processor = Wav2Vec2Processor.from_pretrained(\"jonatasgrosman/wav2vec2-large-xlsr-53-english\")\n",
+ "model = Wav2Vec2ForCTC.from_pretrained(\"jonatasgrosman/wav2vec2-large-xlsr-53-english\")\n"
+ ],
+ "metadata": {
+ "colab": {
+ "base_uri": "https://localhost:8080/",
+ "height": 177,
+ "referenced_widgets": [
+ "03c1da0b20c446f9a9c34c9ca0662642",
+ "6d76bbac63d64e5e98ea216007619f2f",
+ "c5e7c57a9f1649fd9bb29704636e019d",
+ "8267325b66ea44c98cd8a5e6aba35cd8",
+ "26f9e5969279480b8ad1a72de4c92257",
+ "a3f471898d1047e181edb17472110958",
+ "411507129ad844e3ae40a0765ab6214d",
+ "c6cd7d8d8a47428fb2c90ac1cb2b3e4a",
+ "bf808d909e2e4e53b066a8f20a55b909",
+ "b54cb241248741fcb26138d5ec3f9ab3",
+ "f6871be60613435cb9d5a0bec2c8fb1f",
+ "66e9c0d23e144b5095ed9ed2dd6c9290",
+ "4c0605c3e93145b5b6f25e124fc8bb89",
+ "e2700293d1ba4ed2b035225d322ca5b6",
+ "0a90d2810c3e47e2840b89b60159090c",
+ "77f69899b71b496383261c98a99a6232",
+ "40ee12edfdc541d588778c35f824bf5e",
+ "730c2826b20d48a4aaa6518654372e7c",
+ "5461dc739f6c4c2e881bc38a8c06e043",
+ "c81f69d943d44958b58f21f9788898d2",
+ "89f410e70f1745c9a386728e833f398c",
+ "1494e5d28af848bb8f326f3f2e3c6dff",
+ "c8014c60bebf4b2ca62efdc261133ffc",
+ "1280a7b79b3b413eb0a1348f6111dc8a",
+ "eb4a699ff0204806a548f8f2e23de187",
+ "d05d0f87686d4464b0e8eaac839a3f59",
+ "0410f3bc5fe64250b7960a0cae9a0eba",
+ "a41fdd1ad5194b92aacda4d5fe440bdc",
+ "519adf57201341baabd2fb0b2c6a1e4d",
+ "814aadafdcd04bc188be07fff4ebb0d4",
+ "5787b06b414d415492fec5c9ecc28835",
+ "55a7300ee1e44e82a124eef58eddc20f",
+ "acaf7a0e88ac4c589363df6dbc46e681",
+ "f15253b3c53048d69d3214ab5ef60c02",
+ "4ce4979b4f584dde9e246d9094401892",
+ "5f8838c416a14b5c89354615ef1601f2",
+ "883a5ba721214fb5a2dce378f2bd373b",
+ "f62ca08633404deca0553ecda9068353",
+ "fb9616c0abb043869d95d9616a8066ff",
+ "0708cc0aeffa4abe8cca1b62e4e2be8e",
+ "b865735a3ee94504b4284a7295e3fa6c",
+ "8348862d9f8946f3a9bcf4c8079c1a6c",
+ "3889ea736e2b424192ce603f8aab7074",
+ "b8ff088d8b964cb29f6dab93192c5642",
+ "df45119bc9bd4d8b810c5790ece14c7b",
+ "651cba0cc8764316bcede4250ab41394",
+ "d8eaa70a2b8d41c5bcd1141efc2b3212",
+ "531d7419655e43b69f76d6634164b812",
+ "53cded6d647a47a7844e7f1c0e930fb6",
+ "6cc3e3efaba148e680c7d7547e511946",
+ "eda3b220bb704a28bf61d69e2623a246",
+ "34da2d465a1540f583f05a6deabf2efe",
+ "064c4037ee254d979a41d3f167e673da",
+ "51dbf2b79116448a9ab5504a67e3490d",
+ "ffcc76aec104431a9808dcc7fd944af6"
+ ]
+ },
+ "id": "pr8L7QpdEHNz",
+ "outputId": "757dd098-0c64-4f9b-d50b-f6f8541dc301"
+ },
+ "execution_count": 26,
+ "outputs": [
+ {
+ "output_type": "display_data",
+ "data": {
+ "text/plain": [
+ "Downloading (…)rocessor_config.json: 0%| | 0.00/262 [00:00"
+ ],
+ "text/html": [
+ "| \n", + " | Model | \n", + "Size | \n", + "
|---|---|---|
| 1 | \n", + "OpenAI Whisper API | \n", + "API | \n", + "
| 2 | \n", + "OpenAI Large-V2 | \n", + "2.67GB | \n", + "
| 3 | \n", + "OpenAI Large | \n", + "2.67GB | \n", + "
| 4 | \n", + "OpenAI Medium | \n", + "1.42GB | \n", + "
| 5 | \n", + "OpenAI Small | \n", + "461MB | \n", + "
| 6 | \n", + "wav2vec2-large-xlsr-53-english | \n", + "1.26GB | \n", + "
| \n", + " | Model | \n", + "Audio1(storybook 27s) | \n", + "Audio2(recording myself 27s) | \n", + "Audio3(interview 27s) | \n", + "
|---|---|---|---|---|
| 1 | \n", + "OpenAI Whisper API | \n", + "2.087933 | \n", + "3.922523 | \n", + "5.703697 | \n", + "
| 2 | \n", + "OpenAI Large-V2 | \n", + "7.058776 | \n", + "5.123279 | \n", + "5.159456 | \n", + "
| 3 | \n", + "OpenAI Large | \n", + "7.487110 | \n", + "5.103006 | \n", + "5.136639 | \n", + "
| 4 | \n", + "OpenAI Medium | \n", + "4.867929 | \n", + "3.682481 | \n", + "4.072536 | \n", + "
| 5 | \n", + "OpenAI Small | \n", + "2.767450 | \n", + "1.943316 | \n", + "2.037943 | \n", + "
| 6 | \n", + "wav2vec2-large-xlsr-53-english | \n", + "10.434880 | \n", + "9.464190 | \n", + "9.232625 | \n", + "
| \n", + " | Model | \n", + "Audio1(storybook 27s) | \n", + "Audio2(recording myself 27s) | \n", + "Audio3(interview 27s) | \n", + "
|---|---|---|---|---|
| 1 | \n", + "OpenAI Whisper API | \n", + "0.0000 | \n", + "0.36 | \n", + "0.808333 | \n", + "
| 2 | \n", + "OpenAI Large-V2 | \n", + "0.0125 | \n", + "0.36 | \n", + "0.808333 | \n", + "
| 3 | \n", + "OpenAI Large | \n", + "0.0125 | \n", + "0.36 | \n", + "0.808333 | \n", + "
| 4 | \n", + "OpenAI Medium | \n", + "0.0125 | \n", + "0.38 | \n", + "0.829167 | \n", + "
| 5 | \n", + "OpenAI Small | \n", + "0.0375 | \n", + "0.37 | \n", + "0.829167 | \n", + "
| 6 | \n", + "wav2vec2-large-xlsr-53-english | \n", + "0.3250 | \n", + "0.53 | \n", + "0.933333 | \n", + "