FYP_ARABIZI / vocab.json
ali-issa's picture
add tokenizer
0e37e93
raw
history blame contribute delete
305 Bytes
{"2": 1, "3": 2, "5": 3, "7": 4, "8": 5, "I": 6, "a": 7, "b": 8, "c": 9, "d": 10, "e": 11, "f": 12, "g": 13, "h": 14, "i": 15, "j": 16, "k": 17, "l": 18, "m": 19, "n": 20, "o": 21, "p": 22, "r": 23, "s": 24, "t": 25, "u": 26, "v": 27, "w": 28, "y": 29, "z": 30, "ه": 31, "|": 0, "[UNK]": 32, "[PAD]": 33}