rpii2023 commited on
Commit
ecf32ba
·
1 Parent(s): ec609cb

Upload processor

Browse files
Files changed (5) hide show
  1. added_tokens.json +5 -5
  2. merges.txt +0 -0
  3. tokenizer.json +0 -0
  4. tokenizer_config.json +0 -0
  5. vocab.json +0 -0
added_tokens.json CHANGED
@@ -1,7 +1,7 @@
1
  {
2
- "</s>": 53622,
3
- "<mask>": 53625,
4
- "<pad>": 53624,
5
- "<s>": 53621,
6
- "<unk>": 53623
7
  }
 
1
  {
2
+ "</s>": 36745,
3
+ "<mask>": 36748,
4
+ "<pad>": 36747,
5
+ "<s>": 36744,
6
+ "<unk>": 36746
7
  }
merges.txt CHANGED
The diff for this file is too large to render. See raw diff
 
tokenizer.json CHANGED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json CHANGED
The diff for this file is too large to render. See raw diff
 
vocab.json CHANGED
The diff for this file is too large to render. See raw diff