|
|
--- |
|
|
library_name: transformers |
|
|
license: mit |
|
|
datasets: |
|
|
- Sp1786/multiclass-sentiment-analysis-dataset |
|
|
base_model: |
|
|
- bigscience/bloom |
|
|
pipeline_tag: text-classification |
|
|
--- |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Model Description |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- **Developed by:** Aman Prakash |
|
|
- **Model type:** Decoder only LLM |
|
|
- **Language(s) (NLP):** All suported by Bloom-1.7B, but primarily finetuned on English Dataset. |
|
|
- **License:** MIT |
|
|
- **Finetuned from model:** Bloom-1.7B |
|
|
|
|
|
|
|
|
|
|
|
## Uses |
|
|
|
|
|
For Sentiment Analysis Classification with 3 class - 0 - negative, 1 - neutral, 2 - positive. |
|
|
|
|
|
|
|
|
### Downstream Use |
|
|
|
|
|
Sentiment Analysis Classification |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Training Details |
|
|
|
|
|
### FineTuning Data |
|
|
|
|
|
[Sp1786/multiclass-sentiment-analysis-dataset](https://huggingface.co/datasets/Sp1786/multiclass-sentiment-analysis-dataset) |
|
|
|
|
|
[More Information Needed] |
|
|
|
|
|
### Training Procedure |
|
|
|
|
|
LoRA with rank 8. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#### Training Hyperparameters |
|
|
|
|
|
per_device_train_batch_size=4, |
|
|
per_device_eval_batch_size=4, |
|
|
gradient_accumulation_steps=4, |
|
|
eval_strategy="steps", |
|
|
eval_steps=200, |
|
|
logging_steps=100, |
|
|
num_train_epochs=2, |
|
|
learning_rate=2e-4, |
|
|
fp16=True, |
|
|
save_steps=500, |
|
|
save_total_limit=2, |
|
|
|
|
|
|
|
|
|
