thai-tokenizer

Running

thai-tokenizer / README.md

Xenova HF staff

Update README.md

618db5f verified 8 months ago

284 Bytes

	---
	title: The Tokenizer Playground
	emoji: 📝
	colorFrom: indigo
	colorTo: purple
	sdk: static
	pinned: false
	short_description: Experiment with and compare different tokenizers
	---

	Link to source code: https://github.com/xenova/transformers.js/tree/main/examples/tokenizer-playground.