|
--- |
|
title: Marathi BPE Tokenizer |
|
emoji: 🔤 |
|
colorFrom: blue |
|
colorTo: purple |
|
sdk: gradio |
|
sdk_version: 4.7.1 |
|
app_file: app.py |
|
pinned: false |
|
--- |
|
|
|
# Marathi Tokenizer Demo |
|
|
|
A web interface for the Marathi BPE tokenizer. This demo allows you to: |
|
- Enter Marathi text |
|
- See the encoded tokens |
|
- Verify the decoding |
|
- Check token counts |
|
|
|
## Usage |
|
Simply enter Marathi text in the input box and see the tokenization results! |
|
|
|
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference |
