license: apache-2.0
language:
- bn
- en
library_name: transformers
tags:
- BanglaT5
- Transformers
- Transliteration
BanglishToBanglaTransliteration
Model Description:
The BanglishToBanglaTransliteration model is designed to transliterate text written in Banglish (Bengali language written using the Latin script) into proper Bengali script. This model is useful for processing and converting text that has been transliterated into Latin characters, which is common in digital communication among Bengali speakers.
Model Details
Model Type: Transformer-based Encoder-Decoder Model
Languages: Bengali (Bangla)
Training Data: The model was trained on a dataset of Banglish and corresponding Bengali sentences.
Usage
Installation
To use this model, you need to install the transformers library by Hugging Face:
pip install transformers
How to Use
Here is an example of how to use the BanglishToBanglaTransliteration model with the transformers library:
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("kazalbrur/BanglishToBanglaTransliteration")
model = AutoModelForSeq2SeqLM.from_pretrained("kazalbrur/BanglishToBanglaTransliteration")
banglish_text = "apni kemon achen?"
inputs = tokenizer.encode(banglish_text, return_tensors="pt")
outputs = model.generate(inputs)
bangla_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(bangla_text)
Training Details
Hyperparameters:
- Embedding Dimension: 256
- Fully Connected Dimension: 512
- Number of Layers: 4
- Number of Attention Heads: 8
- Dropout Rate: 0.1
- Epochs: 200
- Batch Size: 256
Optimizer: Adam with a custom learning rate schedule.
Training Procedure
The model was trained using a Transformer architecture, incorporating positional encodings, multi-head attention mechanisms, and feed-forward neural networks. It was optimized using a custom learning rate schedule and Adam optimizer.
Loss Function: Sparse Categorical Crossentropy, masked to ignore padding tokens.
Accuracy Function: Calculated based on the exact match of tokens, excluding padding tokens.
Evaluation
The model's performance was evaluated using standard NLP metrics. The accuracy and loss during training were tracked and can be plotted to visualize model performance.
Limitations
The model may not perform well on out-of-vocabulary words or phrases that were not included in the training data. Additionally, transliteration errors can occur if the input Banglish text is not phonetically accurate.
Future Work
Future improvements may include expanding the training dataset, fine-tuning on more diverse datasets, and integrating with broader NLP tasks.
Citation
If you use this model, please cite it as follows:
@misc{banglishtobanglatransliteration,
author = {Kazal Chandra Barman},
title = {BanglishToBanglaTransliteration},
year = {2024},
url = {https://huggingface.co/kazalbrur/BanglishToBanglaTransliteration}
}
Acknowledgements
Special thanks to [https://www.kaggle.com/datasets/kazalnext/banglish-to-bangla-dataset] for providing the dataset and [csebuetnlp/banglat5] for facilitating the development of this model.