SystemGemma2-2b-it / tokenizer_config.json

Commit History

fixes system prompt duplication #1
86cac43
verified

Piotr Zalewski commited on

enabled system prompt
131bbb8

lunahr commited on

Update tokenizer files (#15)
e48216d
verified

megrisdal Xenova HF staff commited on

Update tokenizer files (#7)
1b694f5
verified

Xenova HF staff commited on

Set `<start_of_turn` and `<end_of_turn>` to be special tokens (#2)
2b6ac3f
verified

Xenova HF staff commited on

Update tokenizer_config.json
d8d6c73
verified

osanseviero commited on

Upload tokenizer
59b7de6
verified

osanseviero commited on