Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
sakharamg
/
NMTKD
like
0
Model card
Files
Files and versions
Community
main
NMTKD
/
translation
/
tools
/
mosesdecoder
/
scripts
/
tokenizer
Ctrl+K
Ctrl+K
1 contributor
History:
1 commit
sakharamg
Uploading all files
158b61b
about 2 years ago
mosestokenizer
Uploading all files
about 2 years ago
basic-protected-patterns
267 Bytes
Uploading all files
about 2 years ago
deescape-special-chars-PTB.perl
Safe
760 Bytes
Uploading all files
about 2 years ago
deescape-special-chars.perl
729 Bytes
Uploading all files
about 2 years ago
delete-long-words.perl
Safe
310 Bytes
Uploading all files
about 2 years ago
detokenizer.perl
12.5 kB
Uploading all files
about 2 years ago
escape-special-chars.perl
847 Bytes
Uploading all files
about 2 years ago
lowercase.perl
Safe
383 Bytes
Uploading all files
about 2 years ago
normalize-punctuation.perl
Safe
1.91 kB
Uploading all files
about 2 years ago
pre-tok-clean.perl
1.43 kB
Uploading all files
about 2 years ago
pre-tokenizer.perl
967 Bytes
Uploading all files
about 2 years ago
pre_tokenize_cleaning.py
2.96 kB
Uploading all files
about 2 years ago
remove-non-printing-char.perl
Safe
549 Bytes
Uploading all files
about 2 years ago
replace-unicode-punctuation.perl
Safe
872 Bytes
Uploading all files
about 2 years ago
tokenizer.perl
18.1 kB
Uploading all files
about 2 years ago
tokenizer_PTB.perl
12.3 kB
Uploading all files
about 2 years ago
