RoBERTa-Base Quantized Model for Sentiment Analysis

This repository hosts a quantized version of the RoBERTa model, fine-tuned for sentiment-analysis-twitter-tweets. The model has been optimized for efficient deployment while maintaining high accuracy, making it suitable for resource-constrained environments.

Model Details

Model Architecture: RoBERTa Base
Task: Sentiment Analysis
Dataset: Twitter Sentiment Analysis
Quantization: Float16
Fine-tuning Framework: Hugging Face Transformers

Usage

Installation

pip install transformers torch

Loading the Model

 
from transformers import RobertaTokenizerFast, RobertaForSequenceClassification, Trainer, TrainingArguments
import torch


 
# Load tokenizer

tokenizer = RobertaTokenizerFast.from_pretrained("roberta-base")
 
# Define a test sentence

test_sentence = "The food was absolutely delicious and the service was amazing!"
 
# Tokenize input

inputs = tokenizer(test_sentence, return_tensors="pt", padding=True, truncation=True, max_length=128)
 
# Ensure input tensors are in correct dtype

inputs["input_ids"] = inputs["input_ids"].long()  # Convert to long type

inputs["attention_mask"] = inputs["attention_mask"].long()  # Convert to long type

 
# Make prediction

with torch.no_grad():

    outputs = quantized_model(**inputs)
 
# Get predicted class

predicted_class = torch.argmax(outputs.logits, dim=1).item()

print(f"Predicted Class: {predicted_class}")
 
 
label_mapping = {0: "Negative", 1: "Neutral", 2: "Positive"}   

#Example
 
predicted_label = label_mapping[predicted_class]

print(f"Predicted Label: {predicted_label}")

Performance Metrics

Accuracy: 0.913237
Precision: 0.913336
Recall: 0.913568
F1: 0.913237

Fine-Tuning Details

Dataset

The dataset is taken from Kaggle .

Training

Number of epochs: 3
Batch size: 16
Evaluation strategy: epoch
Learning rate: 2e-5

Quantization

Post-training quantization was applied using PyTorch's built-in quantization framework to reduce the model size and improve inference efficiency.

Repository Structure


.
├── config.json
├── tokenizer_config.json    
├── special_tokens_map.json 
├── tokenizer.json        
├── model.safetensors    # Fine Tuned Model
├── README.md            # Model documentation

Limitations

The model may not generalize well to domains outside the fine-tuning dataset.
Quantization may result in minor accuracy degradation compared to full-precision models.

Contributing

Contributions are welcome! Feel free to open an issue or submit a pull request if you have suggestions or improvements.