onnx-community

/

whisper-base

like 11

You can clone the repo with git using:

git lfs install
git clone https://huggingface.co/onnx-community/whisper-base

or if you want to download the files directly, you can navigate to "Files" and download the ones you need. For example, the model files are located at:

• https://huggingface.co/onnx-community/whisper-base/resolve/main/onnx/encoder_model.onnx?download=true
• https://huggingface.co/onnx-community/whisper-base/resolve/main/onnx/decoder_model_merged.onnx?download=true
• ...

thanks but i want to use this model for inference its possible in python? then how to do that in python give me some example please?

In python or javascript? You can use the javascript version with Transformers.js with:

import { pipeline } from '@xenova/transformers';

const transcriber = await pipeline('automatic-speech-recognition', 'Xenova/whisper-base');
const url = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/french-audio.mp3';
const output = await transcriber(url, { language: 'french', task: 'transcribe' });
// { text: " J'adore, j'aime, je n'aime pas, je déteste." }

Refer to the documentation for more information and usage instructions.

Error loading Whisper model: Unexpected token '<' when using pipeline in React JS app

Hey team,

I'm experiencing an issue when trying to load the Whisper model for automatic speech recognition in my React JS application. Specifically, when I try to execute the following line:

javascript
const transcriber = await pipeline('automatic-speech-recognition', 'Xenova/whisper-base');

I get the following error:

Error loading Whisper model: SyntaxError: Unexpected token '<', "<DOCTYPE "... is not valid JSON
    at JSON.parse (<anonymous>)
    at getModelJSON (hub.js:581:1)
    at async AutoProcessor.from_pretrained (processors.js:2216:1)
    at async Promise.all (:3001/index 2)
    at async loadItems (pipelines.js:3279:1)
    at async pipeline (pipelines.js:3219:1)
    at async loadModel (SpeechRecognition.js:16:1)
loadModel @ SpeechRecognition.js:24
await in loadModel (async)
callCallback @ react-dom.development.js:4164
invokeGuardedCallbackDev @ react-dom.development.js:4213
invokeGuardedCallback @ react-dom.development.js:4277
invokeGuardedCallbackAndCatchFirstError @ react-dom.development.js:4291
executeDispatch @ react-dom.development.js:9041
processDispatchQueueItemsInOrder @ react-dom.development.js:9073
processDispatchQueue @ react-dom.development.js:9086
dispatchEventsForPlugins @ react-dom.development.js:9097
(anonymous) @ react-dom.development.js:9288
batchedUpdates$1 @ react-dom.development.js:26140
batchedUpdates @ react-dom.development.js:3991
dispatchEventForPluginEventSystem @ react-dom.development.js:9287
dispatchEventWithEnableCapturePhaseSelectiveHydrationWithoutDiscreteEventReplay @ react-dom.development.js:6465
dispatchEvent @ react-dom.development.js:6457
dispatchDiscreteEvent @ react-dom.development.js:6430

It seems like the error is related to JSON parsing, but I'm not sure what's causing it. I've checked the pipeline function, but I couldn't find any issues.

Has anyone else encountered this error when using the Whisper model in a React JS application? Any ideas on how to resolve it?

Thanks in advance for your help

Hi there 👋 In most cases, this in intentional and is because (like the python library) we check your local server first for the model files before downloading them from the Hugging Face Hub. If your server correctly returns the 404 status when the model file is not found, it will fallback to the server. If you want to avoid this local model check, you can add the following to the top of your code:

import { env } from '@xenova/transformers';
env.allowLocalModels=false;

You may also need to refresh your cache and try again with:

1. Open Devtools
2. Go to "Application" tab
3. Go to "Storage"
4. Click "Clear site data"

Hey team,

after running
import { pipeline, env } from '@xenova/transformers';
env.allowLocalModels=false;
const pipe = await pipeline('automatic-speech-recognition', 'onnx-community/whisper-base');

using version:
"@xenova/transformers": "^2.17.2",

I keep getting error:
RealtimeTransciption.tsx:44 Error loading model or processing audio: Error: Unsupported model type: whisper
at AutoModelForCTC.from_pretrained (@xenova_transformers.js?v=fafca126:24843:13)
at asyn

I'm not sure what I'm doing wrong here.

Has anyone else encountered this error when using the Whisper model in a React JS application? Any ideas on how to resolve it?

Thanks in advance for your help

It may be an ONNX version issue - can you use Transformers.js v3 via npm i @huggingface/transformers instead?

We will be releasing this as the main version shortly.