Update README.md
Browse files
README.md
CHANGED
@@ -18,6 +18,7 @@ tags:
|
|
18 |
- Baseline for ablation studies and comparisons with more advanced models
|
19 |
|
20 |
## Use
|
|
|
21 |
|
22 |
```python
|
23 |
from transformers import PreTrainedTokenizerFast, AutoModelForCausalLM
|
|
|
18 |
- Baseline for ablation studies and comparisons with more advanced models
|
19 |
|
20 |
## Use
|
21 |
+
Since this model doesn't use a proper GPT2 format tokenizer, special tokens still need to be set up manually (next experiment will use a proper one ofc):
|
22 |
|
23 |
```python
|
24 |
from transformers import PreTrainedTokenizerFast, AutoModelForCausalLM
|