|
--- |
|
library_name: transformers |
|
tags: [] |
|
--- |
|
|
|
# Model Card for wav2vec2-large-xlsr-persian-fine-tuned |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
|
|
This model is a fine-tuned version of `facebook/wav2vec2-large-xlsr-53` on Persian language data from the Mozilla Common Voice Dataset. The model is fine-tuned for automatic speech recognition (ASR) tasks. |
|
|
|
- **Developed by:** Alireza Dastmalchi Saei |
|
- **Funded by:** - |
|
- **Shared by:** - |
|
- **Model type:** wav2vec2 |
|
- **Language(s) (NLP):** Persian |
|
- **License:** MIT |
|
- **Finetuned from model:** wav2vec2-large-xlsr-53 |
|
|
|
### Model Sources |
|
|
|
- **Repository:** [Model Repository](https://huggingface.co/AlirezaSaei/wav2vec2-large-xlsr-persian-fine-tuned) |
|
- **Paper:** - |
|
- **Demo:** - |
|
|
|
## Uses |
|
|
|
### Direct Use |
|
|
|
This model can be used directly for transcribing Persian speech to text but it needs to be further fine-tuned with data. |
|
|
|
### Downstream Use |
|
|
|
The model can be fine-tuned further for specific ASR tasks or integrated into larger speech-processing pipelines. |
|
|
|
### Out-of-Scope Use |
|
|
|
The model is not suitable for languages other than Persian and may not perform well on noisy audio or speech with heavy accents not represented in the training data. |
|
|
|
## Bias, Risks, and Limitations |
|
|
|
The model is trained on a dataset that may not cover all variations of the Persian language, leading to potential biases in recognizing less represented dialects or accents. |
|
|
|
### Recommendations |
|
|
|
Users should be aware of the biases, risks, and limitations. Further fine-tuning on diverse datasets is recommended to mitigate these biases. |
|
|
|
## How to Get Started with the Model |
|
|
|
```python |
|
from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC |
|
import torch |
|
import torchaudio |
|
|
|
# Load processor and model |
|
processor = Wav2Vec2Processor.from_pretrained("AlirezaSaei/wav2vec2-large-xlsr-persian-fine-tuned") |
|
model = Wav2Vec2ForCTC.from_pretrained("AlirezaSaei/wav2vec2-large-xlsr-persian-fine-tuned") |
|
|
|
# Load audio file |
|
audio_input, _ = torchaudio.load("path_to_audio.wav") |
|
|
|
# Preprocess and predict |
|
inputs = processor(audio_input, sampling_rate=16000, return_tensors="pt", padding=True) |
|
logits = model(**inputs).logits |
|
predicted_ids = torch.argmax(logits, dim=-1) |
|
transcription = processor.batch_decode(predicted_ids) |
|
|
|
print("Transcription:", transcription) |
|
``` |
|
|
|
## Training Details |
|
|
|
### Training Data |
|
|
|
The model is fine-tuned on the Mozilla Common Voice Dataset. The training data includes Persian speech samples, with lengths filtered between 4 and 6 seconds for training and up to 15 seconds for testing. |
|
|
|
### Training Procedure |
|
|
|
The audio is resampled from 48000 Hz to 16000 Hz. The tokenizer, feature extractor, and processor are defined using the `Wav2Vec2CTCTokenizer`, `Wav2Vec2FeatureExtractor`, and `Wav2Vec2Processor` classes. |
|
|
|
#### Training Hyperparameters |
|
|
|
- **Training regime:** fp16 mixed precision |
|
- **Batch Size:** 12 |
|
- **Num Epochs:** 5 |
|
- **Learning Rate:** 1e-4 |
|
- **Gradient Accumulation Steps:** 2 |
|
- **Warmup Steps:** 1000 |
|
|
|
### Speeds, Sizes, Times |
|
|
|
- **Training Files:** 2217 |
|
- **Testing Files:** 5212 |
|
- **Training Time (minutes):** 19.67 |
|
- **Total Parameters:** 315,479,720 |
|
- **Trainable Parameters:** 311,269,544 |
|
- **WER:** 1.0 |
|
|
|
## Evaluation |
|
|
|
### Testing Data, Factors & Metrics |
|
|
|
#### Testing Data |
|
|
|
The model is evaluated on a subset of the Mozilla Common Voice Dataset. |
|
|
|
#### Factors |
|
|
|
Evaluation is disaggregated by different lengths of audio samples. |
|
|
|
#### Metrics |
|
|
|
Word Error Rate (WER) is used as the evaluation metric. It measures the percentage of words that are incorrectly predicted. |
|
|
|
### Results |
|
|
|
The model achieves a WER of 1.0 on the test data. |
|
|
|
## Environmental Impact |
|
|
|
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). |
|
|
|
- **Hardware Type:** Colab T4 GPU |
|
|
|
## Technical Specifications |
|
|
|
### Model Architecture and Objective |
|
|
|
The model uses the Wav2Vec2 architecture, which is designed for automatic speech recognition. |
|
|
|
### Compute Infrastructure |
|
|
|
#### Hardware |
|
|
|
Colab T4 GPU |
|
|
|
#### Software |
|
|
|
Python Notebook (.ipynb) |
|
|
|
## Model Card Contact |
|
|
|
For further information, contact me. |
