KoichiYasuoka commited on
Commit
a6bc6a0
1 Parent(s): 673be97

emulate tokenize_chinese_chars=True

Browse files
Files changed (3) hide show
  1. mergeout.sh +3 -0
  2. merges.txt +0 -0
  3. oldmerges.txt +0 -0
mergeout.sh ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ #! /usr/bin/egrep -vf
2
+ ^ã[^ĢģĤĥĦħĨĩĪīĬĭĮįİı ][^ ]
3
+ ^[ãäåæçèé][^ ][^ ]
merges.txt CHANGED
The diff for this file is too large to render. See raw diff
 
oldmerges.txt ADDED
The diff for this file is too large to render. See raw diff