File size: 3,187 Bytes
a6cfd45 503cc5a a6cfd45 09fe49a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 |
---
library_name: transformers
tags:
- text-to-speech
- annotation
license: apache-2.0
language:
- kbd
pipeline_tag: text-to-speech
inference: false
datasets:
- anzorq/kbd_speech
- anzorq/kbd_speech_murat-tagged-for-parler-tts
---
# Parler-TTS Fine-tuned for Kabardian Language (Murat)
This model is a fine-tuned version of the Parler-TTS model trained on a dataset of Kabardian speech from the speaker Murat Sokhov.
## Model Details:
* **Model:** ParlerTTSForConditionalGeneration
* **Base Model:** Parler-TTS mini v0.1
* **Training Data:** Kabardian speech dataset from "Murat" (anzorq/kbd_speech_murat)
* **Training Configuration:**
* `--train_dataset_name`: "anzorq/kbd_speech_murat"
* `--train_metadata_dataset_name`: "anzorq/kbd_speech_murat-tagged-for-parler-tts"
* `--num_train_epochs`: 4
* `--gradient_accumulation_steps`: 18
* `--gradient_checkpointing`: True
* `--per_device_train_batch_size`: 2
* `--learning_rate`: 0.00008
* `--lr_scheduler_type`: "constant_with_warmup"
* `--warmup_steps`: 50
* `--logging_steps`: 2
* `--freeze_text_encoder`: True
* `--dtype`: "float16"
* `--seed`: 456
## Usage:
### Installation:
```bash
pip install git+https://github.com/huggingface/parler-tts.git
```
### Inference:
```python
from parler_tts import ParlerTTSForConditionalGeneration
from transformers import AutoTokenizer
import torch
import soundfile as sf
device = "cuda" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if device != "cpu" else torch.float32
model = ParlerTTSForConditionalGeneration.from_pretrained("anzorq/parler-tts-mini-kbd-Murat", torch_dtype=torch_dtype).to(device)
tokenizer = AutoTokenizer.from_pretrained("anzorq/parler-tts-mini-kbd-Murat")
prompt = "Уэшх нэужьым къиуа псы утхъуар, къэгубжьа хуэдэ, къыпэщӏэхуэр ирихьэхыну хьэзыру йожэх"
description = "Murat's voice is very clear, but it is very confined in terms of pacing and delivery"
# Simple transliteration since the original tokenizer used in Parler-TTS does not support Cyrillic symbols
def transliterate(text):
char_map = {
'а': 'a', 'б': 'b', 'в': 'v', 'г': 'g', 'д': 'd', 'е': 'e', 'ж': 'zh', 'з': 'z', 'и': 'i', 'й': 'j',
'к': 'k', 'л': 'l', 'м': 'm', 'н': 'n', 'о': 'o', 'п': 'p', 'р': 'r', 'с': 's', 'т': 't', 'у': 'u',
'ф': 'f', 'х': 'x', 'ц': 'c', 'ч': 'ch', 'ш': 'sh', 'щ': 'sx', 'ъ': '2', 'ы': 'y', 'ь': '3', 'э': '4',
'я': 'ya', 'ӏ': '1'
}
for cyrillic_char, latin_char in char_map.items():
text = text.replace(cyrillic_char, latin_char)
return text
transliterated_prompt = transliterate(prompt)
# Generate audio
input_ids = tokenizer(description, return_tensors="pt").input_ids.to(device)
prompt_input_ids = tokenizer(transliterated_prompt, return_tensors="pt").input_ids.to(device)
generation = model.generate(input_ids=input_ids, prompt_input_ids=prompt_input_ids).to(torch.float32)
audio_arr = generation.cpu().numpy().squeeze()
# Save the audio to a file
sf.write("parler_tts_out.wav", audio_arr, model.config.sampling_rate)
``` |