tags:
- embeddings
- glove
- cc100
license: cc-by-sa-4.0
CC100 GloVe Embeddings for BN_ROM Language
Model Description
- Language: bn_rom
- Embedding Algorithm: GloVe (Global Vectors for Word Representation)
- Vocabulary Size: 626162
- Vector Dimensions: 300
- Training Data: CC100 dataset
Training Information
We trained GloVe embeddings using the original C code. The model was trained by stochastically sampling nonzero elements from the co-occurrence matrix, over 100 iterations, to produce 300-dimensional vectors. We used a context window of ten words to the left and ten words to the right. Words with fewer than 5 co-occurrences were excluded for languages with over 1 million tokens in the training data, and the threshold was set to 2 for languages with smaller datasets.
We used data from CC100 for training the static word embeddings. We set xmax = 100, α = 3/4, and used AdaGrad optimization with an initial learning rate of 0.05.
Usage
These embeddings can be used for various NLP tasks such as text classification, named entity recognition, and as input features for neural networks.
Citation
If you use these embeddings in your research, please cite:
@misc{gurgurov2024lowremrepositorywordembeddings,
title={LowREm: A Repository of Word Embeddings for 87 Low-Resource Languages Enhanced with Multilingual Graph Knowledge},
author={Daniil Gurgurov and Rishu Kumar and Simon Ostermann},
year={2024},
eprint={2409.18193},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2409.18193},
}
License
These embeddings are released under the CC-BY-SA 4.0 License.