timaeus
/

GPT2-tokenizer

Model card Files Files and versions Community

Ctrl+K

Ctrl+K

2 contributors

History: 1 commit

svwingerden's picture

georgeyw's picture

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx

138dbbe verified 7 months ago

.gitattributes

1.52 kB

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx 7 months ago
added_tokens.json

29 Bytes

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx 7 months ago
merges.txt

456 kB

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx 7 months ago
special_tokens_map.json

131 Bytes

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx 7 months ago
tokenizer.json

2.11 MB

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx 7 months ago
tokenizer_config.json

559 Bytes

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx 7 months ago
vocab.json

798 kB

Duplicate from georgeyw/GPT2-tokenizer-8k-ctx 7 months ago