File size: 515 Bytes
f5f3bf3 a6eaabf f5f3bf3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
---
title: Marathi BPE Tokenizer
emoji: 🔤
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 4.7.1
app_file: app.py
pinned: false
---
# Marathi Tokenizer Demo
A web interface for the Marathi BPE tokenizer. This demo allows you to:
- Enter Marathi text
- See the encoded tokens
- Verify the decoding
- Check token counts
## Usage
Simply enter Marathi text in the input box and see the tokenization results!
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference |