Hugging Face's logo

ouasdg
/
fokentoken

Model card Files Community
5

Upload tokenizer

#3
by ouasdg - opened
base: refs/heads/main
←
from: refs/pr/3
Discussion Files changed
+24913
-25039
Files changed (3) hide show
  1. merges.txt +0 -63
  2. tokenizer.json +0 -0
  3. vocab.json +0 -0
merges.txt CHANGED
@@ -24742,66 +24742,3 @@ uc ric
24742
  Ġoga s
24743
  Ġtem leH
24744
  arts er
24745
- Ġnoisse fnoc
24746
- Ġbo m
24747
- Ġegats oh
24748
- Ġesac wohs
24749
- Ġretirw neercs
24750
- Ġsnai na
24751
- Ġslli B
24752
- Ġsecna veirg
24753
- Ġylgni droccA
24754
- Ġtni ts
24755
- Ġtni rps
24756
- Ġretse c
24757
- Ġtoo F
24758
- Ġsrek roW
24759
- Ġdengised er
24760
- Ġtrops sap
24761
- Ġyca L
24762
- Ġ07 21
24763
- Ġdesserp moc
24764
- Ġnais yalaM
24765
- Ġgnu ws
24766
- Ġgnikat rednu
24767
- Ġdelle per
24768
- Ġgnim m
24769
- Ġgnim rifnoc
24770
- Ġslai rub
24771
- Ġak sah
24772
- Ġtnia f
24773
- Ġpee J
24774
- Ġcita uqa
24775
- Ġais potareC
24776
- Ġea lli
24777
- Ġea von
24778
- Ġ83 3
24779
- Ġezi sahpme
24780
- Ġara gana
24781
- Ġdrawrof thgiarts
24782
- Ġsegnahc retni
24783
- Ġ09 5
24784
- Ġ09 31
24785
- zz A
24786
- Ġnek nus
24787
- Ġcitsi mitpo
24788
- Ġgnisi nagro
24789
- Ġevitca ni
24790
- Ġav A
24791
- Ġgnihca oC
24792
- mag la
24793
- ĠÃ «oZ
24794
- Ġelliv u
24795
- Ġsg iwt
24796
- Ġnosrep sekops
24797
- Ġpihsre weiv
24798
- Ġgnib bom
24799
- Ġgnib ircseD
24800
- Ġseira id
24801
- Ġseira noissim
24802
- Ġseira noitulover
24803
- Ġlairo tirreT
24804
- Ġpe elS
24805
- Ġpe elsa
24806
- Ġthgi led
24807
- Ġesop mi
 
24742
  Ġoga s
24743
  Ġtem leH
24744
  arts er
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
tokenizer.json CHANGED
The diff for this file is too large to render. See raw diff
 
vocab.json CHANGED
The diff for this file is too large to render. See raw diff