File size: 614 Bytes
cbf648c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
'''
Defines the set of symbols used in text input to the model.
'''


# cjehd_cleaners:
_pad         = '_'
_punctuation = ',.!?…~'
_letters     = ' #*=AEINOQU^`abdefghijklmnopqrstuvwxyzãæçéðøĭŋœɐɑɔəɛɡɥɦɪɫɯɱɸɹɽɾʀʁʃʊʏʑʒʔʦʧʰˀˈˌːˑ̩̯̃͜͡βθχ⁼↑→↓šđǩḱ-ă,ś'


# German_cleaners:
_pad         = '_'
_punctuation =',.!?…~;:'
_letters     ="'*^_abdefghijklmnopstuvxyzçõøĭŋɐɘəɚɱɹɽɾʀʁʃʋʏʔʥʰʷˌːˑχ↓ⱼ"


# Export all symbols:
symbols = [_pad] + list(_punctuation) + list(_letters)

# Special symbol ids
SPACE_ID = symbols.index(" ")