jed351 commited on
Commit
e0200cb
·
1 Parent(s): 8f88c67

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -4
README.md CHANGED
@@ -10,11 +10,9 @@ My forked and modified version: https://github.com/jedcheng/bert-tokenizer-canto
10
 
11
  After identifying the missing characters, the huggingface library provides very high level API to modify the tokenizer and embeddings.
12
 
13
- """
14
  Download your model from the Huggingface library
15
 
16
  tokenizer.add_tokens("your new tokens")
17
  model.resize_token_embeddings(len(tokenizer))
18
-
19
-
20
- """
 
10
 
11
  After identifying the missing characters, the huggingface library provides very high level API to modify the tokenizer and embeddings.
12
 
13
+ ```
14
  Download your model from the Huggingface library
15
 
16
  tokenizer.add_tokens("your new tokens")
17
  model.resize_token_embeddings(len(tokenizer))
18
+ ```