Update README.md
Browse files
README.md
CHANGED
|
@@ -3,4 +3,6 @@ language:
|
|
| 3 |
- en
|
| 4 |
---
|
| 5 |
This tokenizer was trained on a small corpus of concatenated ARPAbet pronunciation tokens + punctuation from the python g2p_en library computed over the entire `synthbot/pony-speech` dataset and 240k lines from `generics_kb_best`, from `community-datasets/generics_kb`.
|
|
|
|
|
|
|
| 6 |
i.e. `One on one, let's clean it.` -> `BAH1T WAH1N AA1N WAH1N , LEH1TS KLIY1N IH1T .` Uses BPE with vocab size of 384.
|
|
|
|
| 3 |
- en
|
| 4 |
---
|
| 5 |
This tokenizer was trained on a small corpus of concatenated ARPAbet pronunciation tokens + punctuation from the python g2p_en library computed over the entire `synthbot/pony-speech` dataset and 240k lines from `generics_kb_best`, from `community-datasets/generics_kb`.
|
| 6 |
+
|
| 7 |
+
|
| 8 |
i.e. `One on one, let's clean it.` -> `BAH1T WAH1N AA1N WAH1N , LEH1TS KLIY1N IH1T .` Uses BPE with vocab size of 384.
|