Hugging Face's logo Hugging Face
  • Models
  • Datasets
  • Spaces
  • Docs
  • Enterprise
  • Pricing

  • Log In
  • Sign Up

sakharamg
/
NMTKD

Model card Files Files and versions Community
NMTKD / translation /tools /mosesdecoder /scripts /tokenizer
Ctrl+K
Ctrl+K
  • 1 contributor
History: 1 commit
sakharamg's picture
sakharamg
Uploading all files
158b61b almost 2 years ago
  • mosestokenizer
    Uploading all files almost 2 years ago
  • basic-protected-patterns
    267 Bytes
    Uploading all files almost 2 years ago
  • deescape-special-chars-PTB.perl
    760 Bytes
    Uploading all files almost 2 years ago
  • deescape-special-chars.perl
    729 Bytes
    Uploading all files almost 2 years ago
  • delete-long-words.perl
    310 Bytes
    Uploading all files almost 2 years ago
  • detokenizer.perl
    12.5 kB
    Uploading all files almost 2 years ago
  • escape-special-chars.perl
    847 Bytes
    Uploading all files almost 2 years ago
  • lowercase.perl
    383 Bytes
    Uploading all files almost 2 years ago
  • normalize-punctuation.perl
    1.91 kB
    Uploading all files almost 2 years ago
  • pre-tok-clean.perl
    1.43 kB
    Uploading all files almost 2 years ago
  • pre-tokenizer.perl
    967 Bytes
    Uploading all files almost 2 years ago
  • pre_tokenize_cleaning.py
    2.96 kB
    Uploading all files almost 2 years ago
  • remove-non-printing-char.perl
    549 Bytes
    Uploading all files almost 2 years ago
  • replace-unicode-punctuation.perl
    872 Bytes
    Uploading all files almost 2 years ago
  • tokenizer.perl
    18.1 kB
    Uploading all files almost 2 years ago
  • tokenizer_PTB.perl
    12.3 kB
    Uploading all files almost 2 years ago